perf(proot): accelerate and cache glibc syscall relays (#1687)

Libc-internal file operations bypass the existing preload wrappers and
still stop in PRoot. Patch recognized ARM64 SVC sites in libc and the
runtime loader, reuse the current translator for 20 path syscall
numbers, and return declined calls to PRoot. Enabled by default after
the tracer-key check; `PROOT_FP_PATCH_SVC=0` remains the troubleshooting
opt-out.

The hardened implementation patches 47 sites on Thor's glibc 2.43.
Relays preserve general/SIMD registers, flags, FP state and errno, and
provide unwind metadata back to the original libc frame. Text is patched
through `/proc/self/mem` after relay pages become executable.
Initialization skips multiple live threads and SVE/SME. Alternate signal
stacks and insufficient remaining stack use PRoot; main-stack growth
respects the current limit. No proroot code or binaries are imported.

Robust-list set/get sites deliberately remain with PRoot/the kernel.
Device tests reproduced stale state when the earlier private-TLS
implementation mixed patched and raw callers. Removing that shortcut
fixes coherence and preserves the host's actual return policy; the
earlier thread-creation gain is retired. SDK forwarding still enables
faccessat2 ENOSYS fallback and clone3 ENOSYS only before Android 15.
Successful clone3 is never executed from the C relay. Existing fsync
handling is unchanged. No in-process RET_TRAP filter is added, so
SIGSYS-mask/handler machinery is not applicable.

Version-2 build-ID/API tables cache relative patch sites in `/tmp`.
Every hit revalidates executable bounds, alignment, instructions and
syscall numbers. Invalid/unsafe files rescan; publication uses a private
temporary file and atomic rename. The PRoot dependency fetch also gains
a mirror matching the existing pinned checksum, resolving upstream
availability failures without changing versions.

Five alternating pairs using the final bundled library on Thor/Android
13:

| operation | wrappers only | hardened relays |
|---|---:|---:|
| libc-internal stat | 49.6 us | 6.0 us |
| libc-internal open/close | 50.4 us | 7.8 us |
| libc faccessat | 49.7 us | 22.0 us |
| libc getcwd | 54.9 us | 5.0 us |
| thread create/join | 136 us | 144 us |
| complete tiny exec/wait | 4.44 ms | 5.60 ms |

An earlier five-pair hardened-library run measured stat 50.1 to 5.9 us
and open/close 51.9 to 7.1 us. The metadata gains survive the safety
checks. Neither run shows a thread-creation gain, and tiny execs retain
roughly 0.9-1.2 ms overhead.

Three full Steam startup pairs yielded Home-ready medians of 47.02 s off
/ 45.25 s on, with overlapping 43-50 s ranges. Steam/CEF-only PSS
medians were 1,223.1 / 1,233.9 MiB; the roughly 11 MiB difference is
smaller than observed variation, but does not establish zero memory
cost. Startup/memory used one unchanged signed bundled-library APK
without overrides; the final stack-gap guard only increases distance
reserved above other mappings. No reliable whole-client startup
improvement is claimed.

Earlier fresh Half-Life SD installs took 350/341 s off and 326/327 s on.
These are historical results from the preceding robust-list revision,
not a post-hardening download retest. Network/shader scheduling varied,
allocation did not reliably improve, and the repeat baseline committed
faster. This PR demonstrates metadata acceleration; it does not
demonstrate a reliable fix for Steam's SD allocation/commit bottleneck.

Validation on `2c3c135b` (production code unchanged since `4998b046`):

- Local release build, runtime/native-library checks, release signing
and four focused Android tests pass. [Final-head
CI](https://github.com/Droid-Deck/DroidDeck/actions/runs/40704837477)
passed: 1,431 Python/native tests (49 skips), 2,648 Android tests (zero
failures/skips), and 162 Rust tests. CodeRabbit is green on the same
head. The installed production APK forwards API 33 and patches 47 sites
with an unset enable flag and no override; Steam Home renders.
- Embedded and shared-PIC matrices pass on glibc 2.43; shared-PIC also
passes on separately loaded glibc 2.39. Coverage includes raw
results/errno/registers/FP state, path/flag/error and protected-pointer
cases, exact readlink boundaries, mixed raw/relay robust state,
concurrent threads, fork, thread churn, alternate-stack signals,
deferred/FIFO cancellation, signal siglongjmp recovery, forced unwind
with caller cleanup, and limited remaining stack.
- Four device test-process filters deny the memory patch file, relay
mapping, executable permission, or patch writes. Every case leaves
original syscall execution working. Host injection also verifies partial
installation after an individual site fails.
- Concurrent atomic symlink replacement passes 3,000 internal
stat/readlink iterations at default/zero TTL. Eight focused host tests
pass, covering instruction/branch limits, stack/limit fallback,
untrusted cache files and NDEBUG checks. API selection gives 45/47/46
sites for unknown/33/simulated-35; unset and explicit 1 agree, and 0
leaves sites untouched. This does not replace Android 15 hardware
testing.
- All bench tools compile with -DNDEBUG. Relative/absolute fixture
workloads agree, and thread/ABI/failure checks pass on Thor with those
builds.
- Earlier internal fts/realpath/NSS/dlopen and normalized read-path
equivalence pass. The write suite retains the existing wrapper SD
Unix-socket mismatch against PRoot.

CodeRabbit disposition: fixed relative fixture paths, prevented
benchmark checks disappearing under NDEBUG, corrected the
limited-fallback description, and narrowed memory-file denial so the
thread check still succeeds. The default-off recommendation was
addressed by fixing/testing the underlying guard and unwind issue while
retaining the requested default-on behavior. The readlink count
suggestion is intentionally not adopted: device boundary comparisons
reproduce current PRoot behavior, so changing that inherited policy
belongs in a separate correctness change.

This is not consequence-free instruction patching: constructor time,
private text copies and stack checks have costs. Android 15+ ARM64 and
16 KiB-page hardware, arbitrary asynchronous cancellation/non-local
exits, user-managed stacks, and attributable tracer-counter measurements
remain gaps. Loader calls before the constructor and arbitrary guest ELF
syscall sites remain outside scope.

Thor was restored to its original APK and bundled library, with app data
preserved and test settings, overrides, temporary files and version-2
test caches cleared.

Detailed evidence and revision boundaries: [proroot
analysis](https://github.com/Droid-Deck/DroidDeck/blob/2c3c135b52c3dd949ef5a99a67d8c12eba24406c/docs/development/proroot-analysis.md).
This commit is contained in:
Kurt Himebauch authored and GitHub committed 2026-10-28 10:53:26 -04:00
commit 9294a60172
19 files changed
+2407 -60

No files matched your search

+10
View File
@@ -0,0 +1,10 @@
#ifndef DROIDDECK_BENCH_CHECKS_H
#define DROIDDECK_BENCH_CHECKS_H
/* Benchmark assertions execute the operations being measured. */
#ifdef NDEBUG
#undef NDEBUG
#endif
#include <assert.h>
#endif
+453
View File
@@ -0,0 +1,453 @@
#if defined(__aarch64__)
static void interruption_hook(const long *);
#define FP_SVC_TEST_HOOK interruption_hook
#endif
#include "../fastpath/fastpath.c"
#include "checks.h"
#include <sys/wait.h>
#include <linux/futex.h>
#include <signal.h>
#include <setjmp.h>
#if defined(__aarch64__)
static void (*interrupt_path)(void);
static void interruption_hook(const long *r) {
(void)r;
if (interrupt_path) {
void (*hook)(void) = interrupt_path;
interrupt_path = NULL;
hook();
}
}
struct snapshot { long x[32]; unsigned char q[32][16]; unsigned long nzcv, fpcr, fpsr; };
extern uint32_t svc_check_site;
extern void svc_register_probe(struct snapshot *, const long *);
__asm__(
".text\n.p2align 4\n.type svc_register_probe,%function\nsvc_register_probe:\n"
"sub sp, sp, #1072\n"
"stp x19, x20, [sp]\nstp x21, x22, [sp, #16]\nstp x23, x24, [sp, #32]\n"
"stp x25, x26, [sp, #48]\nstp x27, x28, [sp, #64]\nstp x29, x30, [sp, #80]\n"
"stp q8, q9, [sp, #96]\nstp q10, q11, [sp, #128]\n"
"stp q12, q13, [sp, #160]\nstp q14, q15, [sp, #192]\n"
"stp x0, x1, [sp, #224]\n"
".irp n,1,2,3,4,5,6,7,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
"mov x\\n, #\\n\n.endr\n"
".irp n,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31\n"
"movi v\\n\\().16b, #0x5a\n.endr\n"
"mov x0, #0xa0000000\nmsr nzcv, x0\n"
"ldr x0, [sp, #232]\nldr x8, [x0, #48]\n"
"ldp x4, x5, [x0, #32]\nldp x2, x3, [x0, #16]\nldp x0, x1, [x0]\n"
".global svc_check_site\nsvc_check_site:\nsvc #0\n"
".irp n,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
"str x\\n, [sp, #(256 + \\n * 8)]\n.endr\n"
".irp n,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31\n"
"str q\\n, [sp, #(512 + \\n * 16)]\n.endr\n"
"mrs x0, nzcv\nstr x0, [sp, #1024]\nmrs x0, fpcr\nstr x0, [sp, #1032]\n"
"mrs x0, fpsr\nstr x0, [sp, #1040]\n"
"ldr x0, [sp, #224]\nadd x1, sp, #256\nmov x2, #792\nbl memcpy\n"
"ldp q8, q9, [sp, #96]\nldp q10, q11, [sp, #128]\n"
"ldp q12, q13, [sp, #160]\nldp q14, q15, [sp, #192]\n"
"ldp x19, x20, [sp]\nldp x21, x22, [sp, #16]\nldp x23, x24, [sp, #32]\n"
"ldp x25, x26, [sp, #48]\nldp x27, x28, [sp, #64]\nldp x29, x30, [sp, #80]\n"
"add sp, sp, #1072\nret\n.size svc_register_probe, .-svc_register_probe\n"
);
static long probe(const long *args) {
struct snapshot out = {0};
unsigned long saved_fpcr, saved_fpsr;
__asm__ volatile("mrs %0, fpcr\nmrs %1, fpsr" : "=r"(saved_fpcr), "=r"(saved_fpsr));
unsigned long fpcr = saved_fpcr ^ (1UL << 22), fpsr = saved_fpsr | (1UL << 27);
__asm__ volatile("msr fpcr, %0\nmsr fpsr, %1" : : "r"(fpcr), "r"(fpsr));
errno = 123;
svc_register_probe(&out, args);
assert(errno == 123);
for (int i = 1; i <= 30; i++) {
long expected = i <= 5 ? args[i] : i == 8 ? args[6] : i;
if (out.x[i] != expected) { fprintf(stderr, "x%d: %lx != %lx\n", i, out.x[i], expected); abort(); }
}
for (int i = 0; i < 32; i++)
for (int j = 0; j < 16; j++) assert(out.q[i][j] == 0x5a);
assert(out.nzcv == 0xa0000000UL);
assert(out.fpcr == fpcr && out.fpsr == fpsr);
__asm__ volatile("msr fpcr, %0\nmsr fpsr, %1" : : "r"(saved_fpcr), "r"(saved_fpsr));
return out.x[0];
}
static void check(const long *args, long want) {
long got = probe(args);
if (got != want) { fprintf(stderr, "syscall %ld: got %ld, expected %ld\n", args[6], got, want); abort(); }
}
static long raw(long nr, long a, long b, long c, long d, long e, long f) {
long args[] = {a, b, c, d, e, f, nr};
return probe(args);
}
static void fault(long nr, long a, long b, long c, long d, long e, long f) {
long args[] = {a, b, c, d, e, f, 0, 0, nr};
long want = svc_original(args);
long call[] = {a, b, c, d, e, f, nr};
check(call, want);
}
static void readlink_boundaries(const char *path) {
char expected[PATH_MAX + 8], actual[PATH_MAX + 8];
long args[] = {AT_FDCWD, (long)path, (long)expected, PATH_MAX, 0, 0, 0, 0, SYS_readlinkat};
long length = svc_original(args);
assert(length > 0 && length < PATH_MAX - 2);
for (long size = length - 1; size <= length + 2; size++) {
memset(expected, 0x5a, sizeof expected); memset(actual, 0x5a, sizeof actual);
args[3] = size;
long want = svc_original(args);
long got = raw(SYS_readlinkat, AT_FDCWD, (long)path, (long)actual, size, 0, 0);
if (got != want || (want > 0 && memcmp(expected, actual, (size_t)want)))
fprintf(stderr, "readlink %s size %ld: relay %ld, proot %ld\n", path, size, got, want);
assert(got == want && (want <= 0 || !memcmp(expected, actual, (size_t)want)));
for (size_t i = (size_t)size; i < sizeof actual; i++) assert(actual[i] == 0x5a);
}
}
static void path_checks(void) {
char work[] = "/tmp/proot-expanded-check-XXXXXX";
assert(mkdtemp(work));
int cwd = open(".", O_RDONLY | O_DIRECTORY);
assert(cwd >= 0 && chdir(work) == 0);
assert(raw(SYS_mkdirat, AT_FDCWD, (long)"dir", 0700, 0, 0, 0) == 0);
long fd = raw(SYS_openat, AT_FDCWD, (long)"dir/file", O_CREAT | O_RDWR | O_EXCL, 0600, 0, 0);
assert(fd >= 0 && write((int)fd, "hello", 5) == 5);
struct stat st;
struct statx sx;
assert(raw(SYS_newfstatat, AT_FDCWD, (long)"dir/file", (long)&st, 0, 0, 0) == 0 && st.st_size == 5);
assert(raw(SYS_statx, AT_FDCWD, (long)"dir/file", 0, STATX_BASIC_STATS, (long)&sx, 0) == 0 && sx.stx_size == 5);
assert(raw(SYS_faccessat, AT_FDCWD, (long)"dir/file", R_OK, 0, 0, 0) == 0);
assert(raw(SYS_symlinkat, (long)"file", AT_FDCWD, (long)"dir/link", 0, 0, 0) == 0);
readlink_boundaries("dir/link");
readlink_boundaries("/proc/self/exe");
char text[PATH_MAX];
memset(text, '#', sizeof text);
assert(raw(SYS_readlinkat, AT_FDCWD, (long)"dir/link", (long)text, sizeof text, 0, 0) == 4);
assert(!memcmp(text, "file#", 5));
assert(raw(SYS_fchmodat, AT_FDCWD, (long)"dir/file", 0640, 0, 0, 0) == 0);
struct timespec times[] = {{1000, 0}, {2000, 0}};
assert(raw(SYS_truncate, (long)"dir/file", 11, 0, 0, 0, 0) == 0);
assert(raw(SYS_utimensat, AT_FDCWD, (long)"dir/file", (long)times, 0, 0, 0) == 0);
assert(raw(SYS_newfstatat, AT_FDCWD, (long)"dir/file", (long)&st, 0, 0, 0) == 0);
assert(st.st_size == 11 && (st.st_mode & 0777) == 0640 && st.st_mtime == 2000);
assert(raw(SYS_renameat, AT_FDCWD, (long)"dir/file", AT_FDCWD, (long)"dir/moved", 0, 0) == 0);
assert(raw(SYS_renameat2, AT_FDCWD, (long)"dir/moved", AT_FDCWD, (long)"dir/file", RENAME_NOREPLACE, 0) == 0);
struct statfs fs;
assert(raw(SYS_statfs, (long)"dir/file", (long)&fs, 0, 0, 0, 0) == 0);
long x = raw(SYS_getxattr, (long)"dir/file", (long)"user.proot-missing", (long)text, sizeof text, 0, 0);
assert(x == -ENODATA || x == -EOPNOTSUPP);
x = raw(SYS_lgetxattr, (long)"dir/link", (long)"user.proot-missing", (long)text, sizeof text, 0, 0);
assert(x == -ENODATA || x == -EOPNOTSUPP);
assert(raw(SYS_listxattr, (long)"dir/file", (long)text, sizeof text, 0, 0, 0) >= 0);
assert(raw(SYS_llistxattr, (long)"dir/link", (long)text, sizeof text, 0, 0, 0) >= 0);
int in = inotify_init1(IN_CLOEXEC);
assert(in >= 0 && raw(SYS_inotify_add_watch, in, (long)"dir", IN_CREATE, 0, 0, 0) >= 0);
close(in);
long n = raw(SYS_getcwd, (long)text, sizeof text, 0, 0, 0, 0);
assert(n == (long)strlen(work) + 1 && !strcmp(text, work));
size_t page = sysconf(_SC_PAGESIZE);
char *guard = mmap(NULL, page * 2, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
assert(guard != MAP_FAILED && mprotect(guard + page, page, PROT_NONE) == 0);
strcpy(guard + page - 2, ".");
assert(raw(SYS_newfstatat, AT_FDCWD, (long)(guard + page - 2), (long)&st, 0, 0, 0) == 0);
long bad = (long)(guard + page);
fault(SYS_openat, AT_FDCWD, bad, O_RDONLY, 0, 0, 0);
fault(SYS_newfstatat, AT_FDCWD, (long)"dir/file", bad, 0, 0, 0);
fault(SYS_statx, AT_FDCWD, (long)"dir/file", 0, STATX_BASIC_STATS, bad, 0);
fault(SYS_readlinkat, AT_FDCWD, (long)"dir/link", bad, 16, 0, 0);
fault(SYS_getcwd, bad, PATH_MAX, 0, 0, 0, 0);
fault(SYS_utimensat, AT_FDCWD, (long)"missing", bad, 0, 0, 0);
fault(SYS_inotify_add_watch, -1, (long)"missing", IN_CREATE, 0, 0, 0);
fault(SYS_renameat2, AT_FDCWD, (long)"dir/file", AT_FDCWD, (long)"dir/nope", 0x80000000, 0);
assert(munmap(guard, page * 2) == 0);
close((int)fd);
assert(raw(SYS_unlinkat, AT_FDCWD, (long)"dir/link", 0, 0, 0, 0) == 0);
assert(raw(SYS_unlinkat, AT_FDCWD, (long)"dir/file", 0, 0, 0, 0) == 0);
assert(raw(SYS_unlinkat, AT_FDCWD, (long)"dir", AT_REMOVEDIR, 0, 0, 0) == 0);
assert(fchdir(cwd) == 0 && close(cwd) == 0 && rmdir(work) == 0);
puts("PASS: path families, dirfds, flags, symlinks, protected pointers, page boundary");
}
static void robust_checks(void) {
if (!svc_android_api) return;
long original = 0;
size_t size = 0;
assert(raw(SYS_get_robust_list, 0, (long)&original, (long)&size, 0, 0, 0) == 0);
struct robust_list_head head = {0};
long set_result = raw(SYS_set_robust_list, (long)&head, sizeof head, 0, 0, 0, 0);
assert(set_result == 0 || set_result == -ENOSYS);
long raw_head = 0;
long direct[] = {0, (long)&raw_head, (long)&size, 0, 0, 0, 0, 0, SYS_get_robust_list};
assert(svc_original(direct) == 0 && raw_head == (long)&head);
long got = 0;
assert(raw(SYS_get_robust_list, gettid(), (long)&got, (long)&size, 0, 0, 0) == 0);
assert(got == (long)&head && size == sizeof head);
assert(raw(SYS_get_robust_list, 0, 1, (long)&size, 0, 0, 0) == -EFAULT);
assert(raw(SYS_set_robust_list, 0, sizeof head, 0, 0, 0, 0) == set_result);
long cleared = raw(SYS_get_robust_list, 0, (long)&got, (long)&size, 0, 0, 0);
assert(cleared == -ENOSYS || (cleared == 0 && got == 0));
direct[0] = original; direct[1] = sizeof head; direct[8] = SYS_set_robust_list;
assert(svc_original(direct) == set_result);
assert(raw(SYS_get_robust_list, 0, (long)&got, (long)&size, 0, 0, 0) == 0 && original == got);
assert(raw(SYS_faccessat2, AT_FDCWD, (long)"/usr/lib/libc.so.6", R_OK, 0, 0, 0) == -ENOSYS);
if (svc_android_api < 35) assert(raw(SYS_clone3, 0, 0, 0, 0, 0, 0) == -ENOSYS);
int api = svc_android_api;
svc_android_api = 35;
assert(!svc_supported(SYS_clone3));
svc_android_api = 0;
assert(!svc_supported(SYS_clone3) && !svc_supported(SYS_set_robust_list));
svc_android_api = api;
assert(!svc_supported(SYS_set_robust_list) && !svc_supported(SYS_get_robust_list));
puts("PASS: mixed relay/raw robust-list callers, errors, faccessat2, clone3 platform guard");
}
static void *thread_check(void *unused) {
(void)unused;
struct stat st;
if (svc_android_api) {
long head;
size_t size;
assert(raw(SYS_get_robust_list, 0, (long)&head, (long)&size, 0, 0, 0) == 0 && head && size == 3 * sizeof(long));
struct robust_list_head private = {0};
long result = raw(SYS_set_robust_list, (long)&private, sizeof private, 0, 0, 0, 0);
assert(result == 0 || result == -ENOSYS);
for (int i = 0; i < 10; i++) {
long got;
sched_yield();
assert(raw(SYS_get_robust_list, 0, (long)&got, (long)&size, 0, 0, 0) == 0 && got == (long)&private);
}
assert(raw(SYS_set_robust_list, head, sizeof private, 0, 0, 0, 0) == result);
}
const long args[] = {AT_FDCWD, (long)"/usr/lib/libc.so.6", (long)&st, 0, 0, 0, SYS_newfstatat};
for (int i = 0; i < 100; i++) check(args, 0);
return NULL;
}
static pthread_barrier_t barrier;
static void *thread_probe(void *unused) {
pthread_barrier_wait(&barrier);
thread_check(unused);
pthread_barrier_wait(&barrier);
return NULL;
}
static volatile sig_atomic_t signals_seen;
static void on_signal(int signal) {
(void)signal;
int saved = errno;
struct stat st;
assert(raw(SYS_newfstatat, AT_FDCWD, (long)"/usr/lib/libc.so.6", (long)&st, 0, 0, 0) == 0);
signals_seen++;
errno = saved;
}
static sigjmp_buf interrupted;
static void jump_signal(int signal) { (void)signal; siglongjmp(interrupted, 1); }
static void jump_path(void) { assert(raise(SIGUSR2) == 0); abort(); }
static void exit_path(void) { pthread_exit(PTHREAD_CANCELED); }
static void mark_cleanup(void *arg) { *(int *)arg = 1; }
struct unwind_check { int cleaned; int (*stat)(int, const char *, struct stat *, int); };
static void *unwind_worker(void *arg) {
struct unwind_check *test = arg;
struct stat st;
pthread_cleanup_push(mark_cleanup, &test->cleaned);
interrupt_path = exit_path;
test->stat(AT_FDCWD, "/usr/lib/libc.so.6", &st, 0);
pthread_cleanup_pop(1);
abort();
}
static void interruption_checks(void) {
void *libc = dlopen("libc.so.6", RTLD_NOW);
struct unwind_check test = {.stat = dlsym(libc, "fstatat")};
assert(test.stat);
struct sigaction old, action = {.sa_handler = jump_signal};
sigemptyset(&action.sa_mask);
assert(sigaction(SIGUSR2, &action, &old) == 0);
sigset_t deliver, old_mask;
sigemptyset(&deliver); sigaddset(&deliver, SIGUSR2);
assert(pthread_sigmask(SIG_UNBLOCK, &deliver, &old_mask) == 0);
if (!sigsetjmp(interrupted, 1)) {
struct stat st;
interrupt_path = jump_path;
test.stat(AT_FDCWD, "/usr/lib/libc.so.6", &st, 0);
abort();
}
assert(svc_busy);
struct stat st;
unsigned long before = svc_fast;
assert(test.stat(AT_FDCWD, "/usr/lib/libc.so.6", &st, 0) == 0);
assert(!svc_busy && svc_fast > before);
assert(sigaction(SIGUSR2, &old, NULL) == 0 && pthread_sigmask(SIG_SETMASK, &old_mask, NULL) == 0);
pthread_t thread;
void *result;
assert(pthread_create(&thread, NULL, unwind_worker, &test) == 0);
assert(pthread_join(thread, &result) == 0 && result == PTHREAD_CANCELED && test.cleaned);
assert(dlclose(libc) == 0);
puts("PASS: signal siglongjmp recovery, forced unwind through patched libc with caller cleanup");
}
static void *small_stack_worker(void *unused) {
size_t consume = (size_t)(uintptr_t)unused - 48 * 1024;
volatile char *padding = __builtin_alloca(consume);
padding[0] = padding[consume - 1] = 1;
struct stat st;
unsigned long before = svc_fast;
for (int i = 0; i < 20; i++)
assert(raw(SYS_newfstatat, AT_FDCWD, (long)"/usr/lib/libc.so.6", (long)&st, 0, 0, 0) == 0);
assert(svc_fast == before && !svc_busy);
return NULL;
}
static void small_stack_checks(void) {
pthread_attr_t attr;
pthread_t thread;
size_t size = PTHREAD_STACK_MIN;
if (size < 128 * 1024) size = 128 * 1024;
assert(pthread_attr_init(&attr) == 0 && pthread_attr_setstacksize(&attr, size) == 0);
assert(pthread_create(&thread, &attr, small_stack_worker, (void *)(uintptr_t)size) == 0);
assert(pthread_join(thread, NULL) == 0 && pthread_attr_destroy(&attr) == 0);
puts("PASS: limited remaining thread stack delegates path calls to PRoot");
}
struct signal_sender { pthread_t target; int done; };
static void *send_signals(void *arg) {
struct signal_sender *sender = arg;
const struct timespec delay = {0, 100000};
for (int i = 0; i < 100; i++) {
assert(pthread_kill(sender->target, SIGUSR1) == 0);
nanosleep(&delay, NULL);
}
__atomic_store_n(&sender->done, 1, __ATOMIC_RELEASE);
return NULL;
}
static void *cancel_worker(void *unused) {
(void)unused;
struct stat st;
for (;;) {
assert(raw(SYS_newfstatat, AT_FDCWD, (long)"/usr/lib/libc.so.6", (long)&st, 0, 0, 0) == 0);
pthread_testcancel();
}
}
struct cancel_open { const char *path; int (*open)(const char *, int, ...); int ready, cleaned; };
static void cancel_cleanup(void *arg) { ((struct cancel_open *)arg)->cleaned = 1; }
static void *cancel_fifo(void *arg) {
struct cancel_open *test = arg;
pthread_cleanup_push(cancel_cleanup, test);
__atomic_store_n(&test->ready, 1, __ATOMIC_RELEASE);
int fd = test->open(test->path, O_RDONLY);
if (fd >= 0) close(fd);
pthread_cleanup_pop(1);
return NULL;
}
static void signal_checks(void) {
stack_t old_stack, stack = {.ss_sp = malloc(SIGSTKSZ), .ss_size = SIGSTKSZ};
assert(stack.ss_sp && sigaltstack(&stack, &old_stack) == 0);
struct sigaction old, action = {.sa_handler = on_signal, .sa_flags = SA_ONSTACK};
sigemptyset(&action.sa_mask);
assert(sigaction(SIGUSR1, &action, &old) == 0);
sigset_t deliver, old_mask;
sigemptyset(&deliver);
sigaddset(&deliver, SIGUSR1);
assert(pthread_sigmask(SIG_UNBLOCK, &deliver, &old_mask) == 0);
struct signal_sender sender = {.target = pthread_self()};
pthread_t thread;
assert(pthread_create(&thread, NULL, send_signals, &sender) == 0);
struct stat st;
for (int i = 0; i < 2000 || !__atomic_load_n(&sender.done, __ATOMIC_ACQUIRE); i++)
assert(raw(SYS_newfstatat, AT_FDCWD, (long)"/usr/lib/libc.so.6", (long)&st, 0, 0, 0) == 0);
assert(pthread_join(thread, NULL) == 0 && signals_seen > 0);
assert(sigaction(SIGUSR1, &old, NULL) == 0 && sigaltstack(&old_stack, NULL) == 0);
assert(pthread_sigmask(SIG_SETMASK, &old_mask, NULL) == 0);
free(stack.ss_sp);
void *result;
assert(pthread_create(&thread, NULL, cancel_worker, NULL) == 0);
assert(pthread_cancel(thread) == 0 && pthread_join(thread, &result) == 0 && result == PTHREAD_CANCELED);
char fifo[] = "/tmp/proot-expanded-fifo-XXXXXX";
int fd = mkstemp(fifo);
assert(fd >= 0 && close(fd) == 0 && unlink(fifo) == 0 && mkfifo(fifo, 0600) == 0);
void *libc = dlopen("libc.so.6", RTLD_NOW);
struct cancel_open test = {.path = fifo, .open = dlsym(libc, "open")};
assert(test.open && pthread_create(&thread, NULL, cancel_fifo, &test) == 0);
while (!__atomic_load_n(&test.ready, __ATOMIC_ACQUIRE)) sched_yield();
struct timespec deadline = {0, 20000000};
nanosleep(&deadline, NULL);
assert(pthread_cancel(thread) == 0 && clock_gettime(CLOCK_REALTIME, &deadline) == 0);
deadline.tv_sec += 2;
assert(pthread_timedjoin_np(thread, &result, &deadline) == 0 && result == PTHREAD_CANCELED && test.cleaned);
assert(unlink(fifo) == 0 && dlclose(libc) == 0);
for (int i = 0; i < 32; i++) {
assert(pthread_create(&thread, NULL, thread_check, NULL) == 0);
assert(pthread_join(thread, NULL) == 0);
}
puts("PASS: signal reentry on alternate stack, deferred/FIFO cancellation with cleanup, thread churn");
}
int main(void) {
assert(fp_ready());
const char *mode = getenv("PROOT_FP_PATCH_SVC");
if (!svc_sites && mode && !strcmp(mode, "0")) {
assert(setenv("PROOT_FP_PATCH_SVC", "1", 1) == 0);
svc_patch_init();
}
printf("platform: android %d, patched %u sites\n", svc_android_api, svc_sites);
assert(svc_sites > 0);
assert(svc_single_thread());
size_t page = sysconf(_SC_PAGESIZE);
uint32_t *relay = svc_relay_page((uintptr_t)&svc_check_site, page), branch;
assert(relay && svc_emit(relay, &svc_check_site, &branch));
__builtin___clear_cache((char *)relay, (char *)relay + 40);
assert(mprotect(relay, page, PROT_READ | PROT_EXEC) == 0);
int mem = open("/proc/self/mem", O_RDWR | O_CLOEXEC);
assert(mem >= 0 && pwrite(mem, &branch, 4, (off_t)(uintptr_t)&svc_check_site) == 4);
close(mem);
__builtin___clear_cache((char *)&svc_check_site, (char *)(&svc_check_site + 1));
long args[] = {0, 1, 2, 3, 4, 5, SYS_getpid};
check(args, getpid());
struct stat st;
args[0] = AT_FDCWD; args[1] = (long)"/usr/lib/libc.so.6"; args[2] = (long)&st;
args[3] = 0; args[6] = SYS_newfstatat;
check(args, 0);
assert(st.st_size > 0);
args[1] = (long)"/usr/lib/proot-svc-check-does-not-exist";
check(args, -ENOENT);
args[1] = 0;
check(args, -EFAULT);
args[1] = (long)"/usr/lib/libc.so.6"; args[2] = 0;
check(args, -EFAULT);
args[1] = (long)"/proc/self/exe"; args[2] = (long)&st;
check(args, 0);
args[3] = 0x40000000;
check(args, -EINVAL);
path_checks();
robust_checks();
pthread_t threads[4];
assert(pthread_barrier_init(&barrier, NULL, 5) == 0);
for (int i = 0; i < 4; i++) assert(pthread_create(&threads[i], NULL, thread_probe, NULL) == 0);
pthread_barrier_wait(&barrier);
assert(!svc_single_thread());
pthread_barrier_wait(&barrier);
for (int i = 0; i < 4; i++) assert(pthread_join(threads[i], NULL) == 0);
assert(pthread_barrier_destroy(&barrier) == 0);
pid_t pid = fork();
assert(pid >= 0);
if (!pid) { thread_check(NULL); _exit(0); }
int status;
assert(waitpid(pid, &status, 0) == pid && WIFEXITED(status) && WEXITSTATUS(status) == 0);
signal_checks();
interruption_checks();
small_stack_checks();
puts("PASS: raw results, errno, x1-x30, q0-q31, NZCV, FPCR/FPSR, 4 threads, fork");
return 0;
}
#else
int main(void) { return 77; }
#endif
+39
View File
@@ -0,0 +1,39 @@
#include "../fastpath/fastpath.c"
#include "checks.h"
#include <linux/filter.h>
#include <linux/seccomp.h>
#include <sys/prctl.h>
int main(int argc, char **argv) {
#if defined(__aarch64__)
assert(argc == 2 && fp_ready() && !svc_sites && svc_single_thread());
int nr = !strcmp(argv[1], "mem") ? SYS_openat : !strcmp(argv[1], "write") ? SYS_pwrite64 :
!strcmp(argv[1], "rx") ? SYS_mprotect : SYS_mmap;
unsigned offset = nr == SYS_mprotect || nr == SYS_openat ? offsetof(struct seccomp_data, args[2]) : offsetof(struct seccomp_data, args[3]);
unsigned mask = nr == SYS_mprotect ? PROT_EXEC : nr == SYS_mmap ? MAP_FIXED_NOREPLACE : nr == SYS_openat ? O_RDWR : ~0U;
struct sock_filter code[] = {
BPF_STMT(BPF_LD | BPF_W | BPF_ABS, offsetof(struct seccomp_data, nr)),
BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, nr, 0, 3),
BPF_STMT(BPF_LD | BPF_W | BPF_ABS, offset),
BPF_JUMP(BPF_JMP | BPF_JSET | BPF_K, mask, 0, 1),
BPF_STMT(BPF_RET | BPF_K, SECCOMP_RET_ERRNO | EACCES),
BPF_STMT(BPF_RET | BPF_K, SECCOMP_RET_ALLOW),
};
if (nr == SYS_pwrite64) code[2] = (struct sock_filter)BPF_STMT(BPF_LD | BPF_IMM, 1);
struct sock_fprog filter = {.len = sizeof code / sizeof *code, .filter = code};
assert(setenv("PROOT_FP_PATCH_SVC", "1", 1) == 0);
if (sc(SYS_prctl, PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0) ||
sc(SYS_prctl, PR_SET_SECCOMP, SECCOMP_MODE_FILTER, (long)&filter, 0, 0)) return 77;
assert(svc_single_thread());
svc_patch_init();
assert(svc_sites == 0 && svc_pages == 0);
struct stat st;
long args[] = {AT_FDCWD, (long)"/etc/os-release", (long)&st, 0, 0, 0, 0, 0, SYS_newfstatat};
assert(svc_original(args) == 0 && S_ISREG(st.st_mode));
puts("PASS: denied patch installation leaves the original libc syscall usable");
return 0;
#else
(void)argc; (void)argv;
return 77;
#endif
}
+93
View File
@@ -0,0 +1,93 @@
#define _GNU_SOURCE
#include <errno.h>
#include <fcntl.h>
#include <link.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>
#include <sys/mman.h>
#include <unistd.h>
static int mov_sysnum(uint32_t insn) {
if ((insn & 0xffe0001fU) == 0xd2800008U ||
(insn & 0xffe0001fU) == 0x52800008U)
return (insn >> 5) & 0xffff;
return -1;
}
static int scan(struct dl_phdr_info *info, size_t size, void *unused) {
(void)size;
(void)unused;
const char *name = strrchr(info->dlpi_name, '/');
name = name ? name + 1 : info->dlpi_name;
if (strcmp(name, "libc.so.6") && strcmp(name, "ld-linux-aarch64.so.1")) return 0;
unsigned total = 0, adjacent = 0, nearby = 0, counts[512] = {0};
for (int i = 0; i < info->dlpi_phnum; i++) {
const ElfW(Phdr) *ph = &info->dlpi_phdr[i];
if (ph->p_type != PT_LOAD || !(ph->p_flags & PF_X)) continue;
const uint32_t *text = (const uint32_t *)(info->dlpi_addr + ph->p_vaddr);
for (size_t j = 0; j < ph->p_memsz / 4; j++) {
if (text[j] != 0xd4000001U) continue;
total++;
int nr = j ? mov_sysnum(text[j - 1]) : -1;
if (nr >= 0) {
adjacent++;
if (nr < 512) counts[nr]++;
} else {
for (size_t k = 2; k <= 8 && k <= j; k++)
if (mov_sysnum(text[j - k]) >= 0) { nearby++; break; }
}
}
}
printf("{\"object\":\"%s\",\"svc\":%u,\"adjacent_mov_x8\":%u,\"nearby_only\":%u,\"adjacent_syscalls\":{",
name, total, adjacent, nearby);
int comma = 0;
for (unsigned nr = 0; nr < 512; nr++)
if (counts[nr]) { printf("%s\"%u\":%u", comma ? "," : "", nr, counts[nr]); comma = 1; }
puts("}}");
return 0;
}
int main(void) {
long page = sysconf(_SC_PAGESIZE);
int fd = open("/usr/lib/libc.so.6", O_RDONLY | O_CLOEXEC);
if (fd < 0) { perror("libc"); return 1; }
unsigned char *p = mmap(NULL, page, PROT_READ | PROT_EXEC, MAP_PRIVATE, fd, 0);
if (p == MAP_FAILED) { perror("file mmap RX"); close(fd); return 1; }
unsigned char before = p[0], changed = before ^ 1;
errno = 0;
int rwx = mprotect(p, page, PROT_READ | PROT_WRITE | PROT_EXEC), rwx_errno = errno;
if (rwx == 0) {
p[0] = changed;
p[0] = before;
if (mprotect(p, page, PROT_READ | PROT_EXEC)) { perror("restore RX"); return 1; }
}
int mem = open("/proc/self/mem", O_RDWR | O_CLOEXEC), mem_errno = mem < 0 ? errno : 0;
ssize_t written = -1;
int write_errno = 0, verified = 0, restored = 0;
if (mem >= 0) {
errno = 0;
written = pwrite(mem, &changed, 1, (off_t)(uintptr_t)p);
write_errno = errno;
verified = written == 1 && *(volatile unsigned char *)p == changed;
restored = pwrite(mem, &before, 1, (off_t)(uintptr_t)p) == 1 &&
*(volatile unsigned char *)p == before;
close(mem);
}
unsigned char disk = 0;
int file_unchanged = pread(fd, &disk, 1, 0) == 1 && disk == before;
munmap(p, page);
close(fd);
void *anon = mmap(NULL, page, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
errno = 0;
int execmem = anon == MAP_FAILED ? -1 : mprotect(anon, page, PROT_READ | PROT_EXEC);
int execmem_errno = errno;
if (anon != MAP_FAILED) munmap(anon, page);
printf("{\"page_size\":%ld,\"rwx\":%d,\"rwx_errno\":%d,\"mem_open_errno\":%d,"
"\"mem_written\":%ld,\"mem_write_errno\":%d,\"mem_verified\":%d,\"mem_restored\":%d,"
"\"file_unchanged\":%d,\"execmem\":%d,\"execmem_errno\":%d}\n",
page, rwx, rwx_errno, mem_errno, (long)written, write_errno, verified, restored,
file_unchanged, execmem, execmem_errno);
dl_iterate_phdr(scan, NULL);
return 0;
}
+49
View File
@@ -0,0 +1,49 @@
#define _GNU_SOURCE
#include "checks.h"
#include <dlfcn.h>
#include <errno.h>
#include <fcntl.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <unistd.h>
int main(void) {
char work[] = "/tmp/proot-svc-race-XXXXXX";
assert(mkdtemp(work));
int cwd = open(".", O_RDONLY | O_DIRECTORY);
assert(cwd >= 0 && chdir(work) == 0);
for (int i = 0; i < 2; i++) {
int fd = open(i ? "b" : "a", O_CREAT | O_EXCL | O_WRONLY, 0600);
assert(fd >= 0 && ftruncate(fd, i + 1) == 0 && close(fd) == 0);
}
assert(symlink("a", "live") == 0);
void *libc = dlopen("libc.so.6", RTLD_NOW);
int (*stat)(int, const char *, struct stat *, int) = dlsym(libc, "fstatat");
ssize_t (*link)(int, const char *, char *, size_t) = dlsym(libc, "readlinkat");
assert(stat && link);
pid_t writer = fork();
assert(writer >= 0);
if (!writer) {
for (int i = 0; i < 3000; i++) {
assert(symlink(i & 1 ? "b" : "a", "next") == 0);
assert(rename("next", "live") == 0);
}
_exit(0);
}
for (int i = 0; i < 3000; i++) {
struct stat st;
assert(stat(AT_FDCWD, "live", &st, 0) == 0 && S_ISREG(st.st_mode) && (st.st_size == 1 || st.st_size == 2));
char text[8];
assert(link(AT_FDCWD, "live", text, sizeof text) == 1 && (text[0] == 'a' || text[0] == 'b'));
}
int status;
assert(waitpid(writer, &status, 0) == writer && WIFEXITED(status) && WEXITSTATUS(status) == 0);
assert(unlink("live") == 0 && unlink("a") == 0 && unlink("b") == 0);
assert(dlclose(libc) == 0 && fchdir(cwd) == 0 && close(cwd) == 0 && rmdir(work) == 0);
puts("PASS: concurrent atomic symlink replacement, internal stat/readlink legal results");
return 0;
}
+74
View File
@@ -0,0 +1,74 @@
#define _GNU_SOURCE
#include "checks.h"
#include <dlfcn.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/resource.h>
#include <sys/stat.h>
#include <time.h>
#include <unistd.h>
static double now(void) {
struct timespec t;
assert(clock_gettime(CLOCK_MONOTONIC, &t) == 0);
return t.tv_sec + t.tv_nsec / 1e9;
}
int main(int argc, char **argv) {
assert(argc == 2 && mkdir(argv[1], 0700) == 0);
void *libc = dlopen("libc.so.6", RTLD_NOW);
assert(libc);
int (*internal_open)(const char *, int, ...) = dlsym(libc, "__open64_nocancel");
int (*internal_stat)(int, const char *, struct stat *, int) = dlsym(libc, "fstatat");
assert(internal_open && internal_stat);
struct rusage before, after;
assert(getrusage(RUSAGE_SELF, &before) == 0);
char paths[64][4096], stream[4096];
double creation = 0, extension = 0, closing = 0, total = now();
for (int i = 0; i < 64; i++) {
snprintf(paths[i], sizeof paths[i], "%s/asset-with-a-longish-steam-depot-filename-%03d.dat", argv[1], i);
double start = now();
int fd = internal_open(paths[i], O_WRONLY | O_CREAT | O_EXCL | O_CLOEXEC, 0600);
assert(fd >= 0);
creation += now() - start;
start = now();
assert(ftruncate(fd, 262144) == 0);
extension += now() - start;
start = now();
assert(close(fd) == 0);
closing += now() - start;
}
double allocation = now() - total;
struct stat st;
double start = now();
for (int i = 0; i < 1000; i++) assert(internal_stat(AT_FDCWD, paths[i % 64], &st, 0) == 0);
double metadata = now() - start;
snprintf(stream, sizeof stream, "%s/stream.bin", argv[1]);
start = now();
int fd = internal_open(stream, O_WRONLY | O_CREAT | O_EXCL | O_CLOEXEC, 0600);
assert(fd >= 0);
double stream_create = now() - start;
static unsigned char block[1024 * 1024];
memset(block, 0x5a, sizeof block);
start = now();
for (int i = 0; i < 64; i++) assert(write(fd, block, sizeof block) == sizeof block);
double writing = now() - start;
start = now();
assert(fdatasync(fd) == 0);
double syncing = now() - start;
assert(close(fd) == 0);
assert(getrusage(RUSAGE_SELF, &after) == 0);
printf("{\"files\":64,\"bytes_per_file\":262144,\"create_s\":%.6f,\"extend_s\":%.6f,"
"\"close_s\":%.6f,\"allocation_s\":%.6f,\"stat1000_s\":%.6f,"
"\"stream_bytes\":67108864,\"stream_create_s\":%.6f,\"write_s\":%.6f,"
"\"fdatasync_s\":%.6f,\"voluntary_switches\":%ld}\n",
creation, extension, closing, allocation, metadata, stream_create, writing,
syncing, after.ru_nvcsw - before.ru_nvcsw);
fflush(stdout);
for (int i = 0; i < 64; i++) assert(unlink(paths[i]) == 0);
assert(unlink(stream) == 0 && rmdir(argv[1]) == 0);
assert(dlclose(libc) == 0);
return 0;
}
+47
View File
@@ -0,0 +1,47 @@
#define _GNU_SOURCE
#include "checks.h"
#include <dlfcn.h>
#include <fcntl.h>
#include <pthread.h>
#include <stdio.h>
#include <sys/resource.h>
#include <time.h>
#include <unistd.h>
static double now(void) {
struct timespec t;
assert(clock_gettime(CLOCK_MONOTONIC, &t) == 0);
return t.tv_sec * 1e9 + t.tv_nsec;
}
static void *thread(void *arg) { return arg; }
int main(void) {
void *libc = dlopen("libc.so.6", RTLD_NOW);
assert(libc);
int (*access)(int, const char *, int, int) = dlsym(libc, "faccessat");
char *(*cwd)(char *, size_t) = dlsym(libc, "getcwd");
assert(access && cwd);
struct rusage before, after;
assert(getrusage(RUSAGE_SELF, &before) == 0);
const int threads = 128, calls = 2000;
double start = now();
for (int i = 0; i < threads; i++) {
pthread_t id;
void *value;
assert(pthread_create(&id, NULL, thread, (void *)1) == 0);
assert(pthread_join(id, &value) == 0 && value == (void *)1);
}
double thread_us = (now() - start) / threads / 1000;
start = now();
for (int i = 0; i < calls; i++) assert(access(AT_FDCWD, "/etc/os-release", R_OK, 0) == 0);
double access_ns = (now() - start) / calls;
char path[4096];
start = now();
for (int i = 0; i < calls; i++) assert(cwd(path, sizeof path));
double cwd_ns = (now() - start) / calls;
assert(getrusage(RUSAGE_SELF, &after) == 0 && dlclose(libc) == 0);
printf("{\"thread_us\":%.0f,\"access_ns\":%.0f,\"cwd_ns\":%.0f,\"switches\":%ld}\n",
thread_us, access_ns, cwd_ns, after.ru_nvcsw - before.ru_nvcsw);
return 0;
}
+92
View File
@@ -0,0 +1,92 @@
#define _GNU_SOURCE
#include "checks.h"
#include <dlfcn.h>
#include <errno.h>
#include <fcntl.h>
#include <fts.h>
#include <pwd.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/resource.h>
#include <sys/stat.h>
#include <time.h>
#include <unistd.h>
static double now(void) {
struct timespec t;
assert(clock_gettime(CLOCK_MONOTONIC, &t) == 0);
return t.tv_sec * 1e9 + t.tv_nsec;
}
int main(int argc, char **argv) {
assert(argc >= 2);
void *libc = dlopen("libc.so.6", RTLD_NOW);
assert(libc);
int (*internal_stat)(int, const char *, struct stat *, int) = dlsym(libc, "fstatat");
int (*internal_open)(const char *, int, ...) = dlsym(libc, "__open64_nocancel");
assert(internal_stat && internal_open);
char path[4096], link[4096], missing[4096];
snprintf(path, sizeof path, "%s/file", argv[1]);
snprintf(link, sizeof link, "%s/link", argv[1]);
snprintf(missing, sizeof missing, "%s/missing", argv[1]);
const char *paths[] = {path, link, missing, "/etc/os-release", "/lib/libc.so.6", "/proc/self/exe", "/dev/null", "", NULL};
struct stat st;
for (size_t i = 0; i < sizeof paths / sizeof *paths; i++) {
errno = 0;
int r = internal_stat(AT_FDCWD, paths[i], &st, 0);
printf("stat %zu: %d %d", i, r, r ? errno : 0);
if (!r) printf(" %o %ld", st.st_mode & S_IFMT, S_ISREG(st.st_mode) && i != 5 ? st.st_size : 0L);
putchar('\n');
errno = 0;
int fd = internal_open(paths[i], O_RDONLY | O_CLOEXEC);
int err = fd < 0 ? errno : 0;
printf("open %zu: %d %d\n", i, fd < 0 ? -1 : 0, err);
if (fd >= 0) close(fd);
}
errno = 0;
int r = internal_stat(AT_FDCWD, path, &st, 0x40000000);
printf("flags: %d %d\n", r, r ? errno : 0);
char *tree[] = {argv[1], NULL};
FTS *fts = fts_open(tree, FTS_PHYSICAL | FTS_NOCHDIR, NULL);
assert(fts);
unsigned files = 0, dirs = 0, links = 0;
FTSENT *ent;
while ((ent = fts_read(fts))) {
if (ent->fts_info == FTS_F) files++;
if (ent->fts_info == FTS_D) dirs++;
if (ent->fts_info == FTS_SL) links++;
}
assert(fts_close(fts) == 0);
printf("fts: %u files %u dirs %u links\n", files, dirs, links);
char resolved[4096], canonical[4096];
assert(realpath(path, canonical));
assert(realpath(link, resolved) && !strcmp(resolved, canonical));
struct passwd pw, *found;
char buf[16384];
r = getpwnam_r("root", &pw, buf, sizeof buf, &found);
printf("nss: %d %d\n", r, found != NULL);
void *libm = dlopen("libm.so.6", RTLD_NOW | RTLD_LOCAL);
assert(libm && dlsym(libm, "cos"));
assert(dlclose(libm) == 0);
puts("realpath/dlopen: PASS");
if (argc > 2) {
const int loops = 5000;
struct rusage before, after;
assert(getrusage(RUSAGE_SELF, &before) == 0);
double start = now();
for (int i = 0; i < loops; i++) assert(internal_stat(AT_FDCWD, path, &st, 0) == 0);
double stat_ns = (now() - start) / loops;
start = now();
for (int i = 0; i < loops; i++) {
int fd = internal_open(path, O_RDONLY | O_CLOEXEC);
assert(fd >= 0 && close(fd) == 0);
}
double open_ns = (now() - start) / loops;
assert(getrusage(RUSAGE_SELF, &after) == 0);
printf("bench: stat %.0f ns, open+close %.0f ns, voluntary switches %ld\n",
stat_ns, open_ns, after.ru_nvcsw - before.ru_nvcsw);
}
assert(dlclose(libc) == 0);
return 0;
}
+14 -5
View File
@@ -23,13 +23,22 @@ fi
WORK=$(mktemp -d)
trap 'rm -rf "$WORK"' EXIT
fetch() {
curl -fsSL --retry 3 -o "$WORK/$1" "$2"
echo "$3 $WORK/$1" | shasum -a 256 -c -
mkdir -p "$WORK/${1%%.*}"
tar -xzf "$WORK/$1" -C "$WORK/${1%%.*}" --strip-components=1
local name=$1 primary=$2 sha=$3 url fetched=0
shift 3
for url in "$primary" "$@"; do
if curl -fsSL --connect-timeout 15 --max-time 90 --retry 3 --retry-max-time 120 -o "$WORK/$name" "$url" &&
printf '%s %s\n' "$sha" "$WORK/$name" | shasum -a 256 -c -; then
fetched=1
break
fi
done
[[ $fetched == 1 ]] || return 1
mkdir -p "$WORK/${name%%.*}"
tar -xzf "$WORK/$name" -C "$WORK/${name%%.*}" --strip-components=1
}
fetch proot.tar.gz "https://github.com/termux/proot/archive/${PROOT_COMMIT}.tar.gz" "$PROOT_SHA256"
fetch talloc.tar.gz "https://www.samba.org/ftp/talloc/talloc-${TALLOC_VERSION}.tar.gz" "$TALLOC_SHA256"
fetch talloc.tar.gz "https://www.samba.org/ftp/talloc/talloc-${TALLOC_VERSION}.tar.gz" "$TALLOC_SHA256" \
"https://distfiles.macports.org/talloc/talloc-${TALLOC_VERSION}.tar.gz"
cp "$HERE/statx_mnt_id.h" "$WORK/proot/src/tracee/statx_mnt_id.h"
cp "$HERE/profile.h" "$WORK/proot/src/syscall/profile.h"
for patch in "$HERE"/patches/*.patch; do
+92 -54
View File
@@ -931,29 +931,26 @@ static long statx_call(int dirfd, const char *path, int flags, unsigned mask, st
return 0;
}
int statx(int dirfd, const char *path, int flags, unsigned mask, struct statx *out) {
REAL(statx, int (*)(int, const char *, int, unsigned, struct statx *));
static long fp_statx(int dirfd, const char *path, int flags, unsigned mask, struct statx *out) {
struct res r;
long rs;
if (mask & STATX__RESERVED) { errno = EINVAL; return -1; }
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wnonnull-compare"
if (!out) { errno = EFAULT; return -1; }
if (!path) {
if (!(flags & AT_EMPTY_PATH)) { errno = EFAULT; return -1; }
return real_statx ? real_statx(dirfd, path, flags, mask, out) : ret(sc(SYS_statx, dirfd, (long)path, flags, mask, (long)out));
}
#pragma GCC diagnostic pop
if ((flags & AT_EMPTY_PATH) && !path[0] && fp_ready()) {
hits++;
return ret(statx_call(dirfd, "", flags, mask, out));
}
if (mask & STATX__RESERVED) return -EINVAL;
if (!out) return -EFAULT;
if (!path) return flags & AT_EMPTY_PATH ? FP_SLOW : -EFAULT;
if ((flags & AT_EMPTY_PATH) && !path[0] && fp_ready())
return statx_call(dirfd, "", flags, mask, out);
rs = (flags & ~(AT_SYMLINK_NOFOLLOW | AT_NO_AUTOMOUNT | AT_STATX_SYNC_TYPE)) ? FP_SLOW
: resolve(dirfd, path, (flags & AT_SYMLINK_NOFOLLOW ? 0 : 1) | MAY_BE_UNSEEN, &r);
if (rs == 1) rs = -ENOENT;
if (rs == 0 || rs == UNSEEN)
rs = statx_call(AT_FDCWD, r.host, flags | (rs == 0 && !r.magic ? AT_SYMLINK_NOFOLLOW : 0), mask, out);
ANSWER(rs, dirfd, path, real_statx(dirfd, path, flags, mask, out));
return rs;
}
int statx(int dirfd, const char *path, int flags, unsigned mask, struct statx *out) {
REAL(statx, int (*)(int, const char *, int, unsigned, struct statx *));
ANSWER(fp_statx(dirfd, path, flags, mask, out), dirfd, path,
real_statx ? real_statx(dirfd, path, flags, mask, out) : ret(sc(SYS_statx, dirfd, (long)path, flags, mask, (long)out)));
}
@@ -961,8 +958,7 @@ static int effective_is_real(void) {
return !getauxval(AT_SECURE) && getuid() == geteuid() && getgid() == getegid();
}
static int access_common(int dirfd, const char *path, int mode, int flags) {
REAL(faccessat, int (*)(int, const char *, int, int));
static long fp_access(int dirfd, const char *path, int mode, int flags) {
struct res r;
if ((flags & AT_EACCESS) && effective_is_real()) flags &= ~AT_EACCESS;
long rs = (flags & ~AT_SYMLINK_NOFOLLOW) == 0
@@ -970,6 +966,12 @@ static int access_common(int dirfd, const char *path, int mode, int flags) {
if (rs == 1) rs = -ENOENT;
if (rs == 0 && S_ISLNK(r.st.st_mode)) rs = FP_SLOW;
if (rs == 0 || rs == UNSEEN) rs = sc(SYS_faccessat, AT_FDCWD, (long)r.host, mode, 0, 0);
return rs;
}
static int access_common(int dirfd, const char *path, int mode, int flags) {
REAL(faccessat, int (*)(int, const char *, int, int));
long rs = fp_access(dirfd, path, mode, flags);
if (rs != FP_SLOW) { hits++; return ret(rs); }
missed(__func__, dirfd, path);
if (flags == 0) return (int)syscall(SYS_faccessat, dirfd, path, mode);
@@ -997,34 +999,36 @@ static long link_text(const struct res *r, char *text, size_t size, int *changed
return (long)strlen(text);
}
static ssize_t readlink_common(int dirfd, const char *path, char *buf, size_t size) {
REAL(readlinkat, ssize_t (*)(int, const char *, char *, size_t));
static long fp_readlink(int dirfd, const char *path, char *buf, size_t size, size_t *written) {
struct res r;
char text[PATH_MAX];
int changed;
long rs = size ? resolve(dirfd, path, 0, &r) : FP_SLOW;
if (rs == 1) rs = -ENOENT;
if (rs == 0) rs = link_text(&r, text, size, &changed);
if (rs == FP_SLOW) {
missed(__func__, dirfd, path);
return real_readlinkat(dirfd, path, buf, size);
}
hits++;
if (rs < 0) return ret(rs);
if (rs < 0) return rs;
size_t n = rs;
if (!changed) {
n = n < size ? n : size;
memcpy(buf, text, n);
if (written) *written = n;
return n;
}
size_t max = size < PATH_MAX ? size : PATH_MAX;
if (n + 1 < max) {
memcpy(buf, text, n + 1);
if (written) *written = n + 1;
return n;
}
memcpy(buf, text, max);
if (written) *written = max;
return max;
}
static ssize_t readlink_common(int dirfd, const char *path, char *buf, size_t size) {
REAL(readlinkat, ssize_t (*)(int, const char *, char *, size_t));
ANSWER(fp_readlink(dirfd, path, buf, size, NULL), dirfd, path, real_readlinkat(dirfd, path, buf, size));
}
ssize_t readlinkat(int dirfd, const char *path, char *buf, size_t size) { return readlink_common(dirfd, path, buf, size); }
ssize_t readlink(const char *path, char *buf, size_t size) { return readlink_common(AT_FDCWD, path, buf, size); }
@@ -1086,12 +1090,15 @@ char *__getcwd_chk(char *buf, size_t size, size_t room) {
}
int mkdirat(int dirfd, const char *path, mode_t mode) {
REAL(mkdirat, int (*)(int, const char *, mode_t));
static long fp_mkdir(int dirfd, const char *path, mode_t mode) {
struct res r;
long rs = resolve(dirfd, path, MAY_BE_UNSEEN, &r);
if (rs == 0 || rs == 1 || rs == UNSEEN) rs = sc(SYS_mkdirat, AT_FDCWD, (long)r.host, mode, 0, 0);
ANSWER(rs, dirfd, path, real_mkdirat(dirfd, path, mode));
return rs;
}
int mkdirat(int dirfd, const char *path, mode_t mode) {
REAL(mkdirat, int (*)(int, const char *, mode_t));
ANSWER(fp_mkdir(dirfd, path, mode), dirfd, path, real_mkdirat(dirfd, path, mode));
}
int mkdir(const char *path, mode_t mode) { return mkdirat(AT_FDCWD, path, mode); }
@@ -1137,8 +1144,7 @@ int unlinkat(int dirfd, const char *path, int flags) { return unlink_common(dirf
int unlink(const char *path) { return unlink_common(AT_FDCWD, path, 0); }
int rmdir(const char *path) { return unlink_common(AT_FDCWD, path, AT_REMOVEDIR); }
static int rename_common(int olddirfd, const char *old, int newdirfd, const char *new, unsigned flags) {
REAL(renameat2, int (*)(int, const char *, int, const char *, unsigned));
static long fp_rename(int olddirfd, const char *old, int newdirfd, const char *new, unsigned flags) {
struct res from, to;
long rs = flags & ~RENAME_NOREPLACE || trailing_slash(old) || trailing_slash(new) ? FP_SLOW : resolve(olddirfd, old, 0, &from);
if (rs == 1) rs = -ENOENT;
@@ -1153,6 +1159,11 @@ static int rename_common(int olddirfd, const char *old, int newdirfd, const char
if (rs == 0) forget();
}
}
return rs;
}
static int rename_common(int olddirfd, const char *old, int newdirfd, const char *new, unsigned flags) {
REAL(renameat2, int (*)(int, const char *, int, const char *, unsigned));
long rs = fp_rename(olddirfd, old, newdirfd, new, flags);
if (rs != FP_SLOW) { hits++; return ret(rs); }
missed(__func__, olddirfd, old);
if (!flags) return (int)syscall(SYS_renameat, olddirfd, old, newdirfd, new);
@@ -1164,30 +1175,35 @@ int renameat2(int olddirfd, const char *old, int newdirfd, const char *new, unsi
int renameat(int olddirfd, const char *old, int newdirfd, const char *new) { return rename_common(olddirfd, old, newdirfd, new, 0); }
int rename(const char *old, const char *new) { return rename_common(AT_FDCWD, old, AT_FDCWD, new, 0); }
int symlinkat(const char *target, int dirfd, const char *path) {
REAL(symlinkat, int (*)(const char *, int, const char *));
static long fp_symlink(const char *target, int dirfd, const char *path) {
struct res r;
long rs = trailing_slash(path) ? FP_SLOW : resolve(dirfd, path, MAY_BE_UNSEEN, &r);
if (rs == 0 || rs == 1 || rs == UNSEEN) {
rs = sc(SYS_symlinkat, (long)target, AT_FDCWD, (long)r.host, 0, 0);
if (rs == 0) forget();
}
ANSWER(rs, dirfd, path, real_symlinkat(target, dirfd, path));
return rs;
}
int symlinkat(const char *target, int dirfd, const char *path) {
REAL(symlinkat, int (*)(const char *, int, const char *));
ANSWER(fp_symlink(target, dirfd, path), dirfd, path, real_symlinkat(target, dirfd, path));
}
int symlink(const char *target, const char *path) { return symlinkat(target, AT_FDCWD, path); }
int fchmodat(int dirfd, const char *path, mode_t mode, int flags) {
REAL(fchmodat, int (*)(int, const char *, mode_t, int));
static long fp_chmod(int dirfd, const char *path, mode_t mode, int flags) {
struct res r;
long rs = flags ? FP_SLOW : resolve(dirfd, path, 1 | MAY_BE_UNSEEN, &r);
if (rs == 1) rs = -ENOENT;
if (rs == 0 || rs == UNSEEN) rs = sc(SYS_fchmodat, AT_FDCWD, (long)r.host, mode, 0, 0);
ANSWER(rs, dirfd, path, real_fchmodat(dirfd, path, mode, flags));
return rs;
}
int fchmodat(int dirfd, const char *path, mode_t mode, int flags) {
REAL(fchmodat, int (*)(int, const char *, mode_t, int));
ANSWER(fp_chmod(dirfd, path, mode, flags), dirfd, path, real_fchmodat(dirfd, path, mode, flags));
}
int chmod(const char *path, mode_t mode) { return fchmodat(AT_FDCWD, path, mode, 0); }
static int utimens_common(int dirfd, const char *path, const struct timespec times[2], int flags) {
REAL(utimensat, int (*)(int, const char *, const struct timespec[2], int));
static long fp_utimens(int dirfd, const char *path, const struct timespec times[2], int flags) {
struct res r;
long rs;
if (!path && fp_ready()) {
@@ -1197,7 +1213,11 @@ static int utimens_common(int dirfd, const char *path, const struct timespec tim
if (rs == 1) rs = -ENOENT;
if (rs == 0 || rs == UNSEEN) rs = sc(SYS_utimensat, AT_FDCWD, (long)r.host, (long)times, flags, 0);
}
ANSWER(rs, dirfd, path, real_utimensat(dirfd, path, times, flags));
return rs;
}
static int utimens_common(int dirfd, const char *path, const struct timespec times[2], int flags) {
REAL(utimensat, int (*)(int, const char *, const struct timespec[2], int));
ANSWER(fp_utimens(dirfd, path, times, flags), dirfd, path, real_utimensat(dirfd, path, times, flags));
}
int utimensat(int dirfd, const char *path, const struct timespec times[2], int flags) {
return utimens_common(dirfd, path, times, flags);
@@ -1224,23 +1244,29 @@ int utime(const char *path, const struct utimbuf *times) {
return utimens_common(AT_FDCWD, path, ts, 0);
}
int truncate(const char *path, off_t length) {
REAL(truncate, int (*)(const char *, off_t));
static long fp_truncate(const char *path, off_t length) {
struct res r;
long rs = resolve(AT_FDCWD, path, 1 | MAY_BE_UNSEEN, &r);
if (rs == 1) rs = -ENOENT;
if (rs == 0 || rs == UNSEEN) rs = sc(SYS_truncate, (long)r.host, length, 0, 0, 0);
ANSWER(rs, AT_FDCWD, path, real_truncate(path, length));
return rs;
}
int truncate(const char *path, off_t length) {
REAL(truncate, int (*)(const char *, off_t));
ANSWER(fp_truncate(path, length), AT_FDCWD, path, real_truncate(path, length));
}
int truncate64(const char *path, off_t length) __attribute__((alias("truncate")));
static int statfs_common(const char *path, struct statfs *buf, int (*real)(const char *, struct statfs *)) {
static long fp_statfs(const char *path, struct statfs *buf) {
struct res r;
long rs = resolve(AT_FDCWD, path, 1, &r);
if (rs == 1) rs = -ENOENT;
if (rs == 0 && under(r.guest, "/dev/shm", 8)) rs = FP_SLOW;
if (rs == 0) rs = sc(SYS_statfs, (long)r.host, (long)buf, 0, 0, 0);
ANSWER(rs, AT_FDCWD, path, real(path, buf));
return rs;
}
static int statfs_common(const char *path, struct statfs *buf, int (*real)(const char *, struct statfs *)) {
ANSWER(fp_statfs(path, buf), AT_FDCWD, path, real(path, buf));
}
int statfs(const char *path, struct statfs *buf) {
REAL(statfs, int (*)(const char *, struct statfs *));
@@ -1288,12 +1314,16 @@ int statvfs64(const char *path, struct statvfs64 *buf) {
return statvfs_common(path, (struct statvfs *)buf, real_statvfs64);
}
static ssize_t getxattr_common(const char *path, const char *name, void *value, size_t size, int follow,
ssize_t (*real)(const char *, const char *, void *, size_t)) {
static long fp_getxattr(const char *path, const char *name, void *value, size_t size, int follow) {
struct res r;
long rs = resolve(AT_FDCWD, path, follow | MAY_BE_UNSEEN, &r);
if (rs == 1) rs = -ENOENT;
if (rs == 0 || rs == UNSEEN) rs = sc(follow ? SYS_getxattr : SYS_lgetxattr, (long)r.host, (long)name, (long)value, size, 0);
return rs;
}
static ssize_t getxattr_common(const char *path, const char *name, void *value, size_t size, int follow,
ssize_t (*real)(const char *, const char *, void *, size_t)) {
long rs = fp_getxattr(path, name, value, size, follow);
if (rs != FP_SLOW) { hits++; return rs < 0 ? ret(rs) : rs; }
missed(__func__, AT_FDCWD, path);
return real(path, name, value, size);
@@ -1307,12 +1337,16 @@ ssize_t lgetxattr(const char *path, const char *name, void *value, size_t size)
return getxattr_common(path, name, value, size, 0, real_lgetxattr);
}
static ssize_t listxattr_common(const char *path, char *list, size_t size, int follow,
ssize_t (*real)(const char *, char *, size_t)) {
static long fp_listxattr(const char *path, char *list, size_t size, int follow) {
struct res r;
long rs = resolve(AT_FDCWD, path, follow | MAY_BE_UNSEEN, &r);
if (rs == 1) rs = -ENOENT;
if (rs == 0 || rs == UNSEEN) rs = sc(follow ? SYS_listxattr : SYS_llistxattr, (long)r.host, (long)list, size, 0, 0);
return rs;
}
static ssize_t listxattr_common(const char *path, char *list, size_t size, int follow,
ssize_t (*real)(const char *, char *, size_t)) {
long rs = fp_listxattr(path, list, size, follow);
if (rs != FP_SLOW) { hits++; return rs < 0 ? ret(rs) : rs; }
missed(__func__, AT_FDCWD, path);
return real(path, list, size);
@@ -1326,13 +1360,15 @@ ssize_t llistxattr(const char *path, char *list, size_t size) {
return listxattr_common(path, list, size, 0, real_llistxattr);
}
int inotify_add_watch(int fd, const char *path, uint32_t mask) {
REAL(inotify_add_watch, int (*)(int, const char *, uint32_t));
static long fp_inotify(int fd, const char *path, uint32_t mask) {
struct res r;
long rs = resolve(AT_FDCWD, path, (mask & IN_DONT_FOLLOW ? 0 : 1) | MAY_BE_UNSEEN, &r);
if (rs == 1) rs = -ENOENT;
if (rs == 0 || rs == UNSEEN) rs = sc(SYS_inotify_add_watch, fd, (long)r.host, mask, 0, 0);
ANSWER(rs, AT_FDCWD, path, real_inotify_add_watch(fd, path, mask));
if (rs == 0 || rs == 1 || rs == UNSEEN) rs = sc(SYS_inotify_add_watch, fd, (long)r.host, mask, 0, 0);
return rs;
}
int inotify_add_watch(int fd, const char *path, uint32_t mask) {
REAL(inotify_add_watch, int (*)(int, const char *, uint32_t));
ANSWER(fp_inotify(fd, path, mask), AT_FDCWD, path, real_inotify_add_watch(fd, path, mask));
}
@@ -1828,3 +1864,5 @@ struct passwd *getpwuid(uid_t uid) {
missed(__func__, AT_FDCWD, "(passwd)");
return real_getpwuid(uid);
}
#include "svc-patch.h"
+90
View File
@@ -0,0 +1,90 @@
struct svc_cached_sites {
uint64_t magic;
uint32_t api, count;
uint64_t offset[256];
};
static unsigned svc_cache_hits, svc_cache_misses;
static int svc_segment(const struct dl_phdr_info *info, uint64_t at, size_t length, unsigned flags) {
for (int i = 0; i < info->dlpi_phnum; i++) {
const ElfW(Phdr) *p = &info->dlpi_phdr[i];
if (p->p_type == PT_LOAD && (p->p_flags & flags) == flags &&
at >= p->p_vaddr && at - p->p_vaddr <= p->p_memsz &&
length <= p->p_memsz - (at - p->p_vaddr)) return i + 1;
}
return 0;
}
static int svc_cache_name(const struct dl_phdr_info *info, char *path, size_t size) {
const char *enabled = getenv("PROOT_FP_SVC_CACHE");
if (enabled && strcmp(enabled, "1")) return 0;
for (int i = 0; i < info->dlpi_phnum; i++) {
const ElfW(Phdr) *p = &info->dlpi_phdr[i];
if (p->p_type != PT_NOTE || !svc_segment(info, p->p_vaddr, p->p_memsz, PF_R)) continue;
const unsigned char *note = (const unsigned char *)(info->dlpi_addr + p->p_vaddr);
for (size_t at = 0; p->p_memsz - at >= sizeof(ElfW(Nhdr));) {
ElfW(Nhdr) h;
memcpy(&h, note + at, sizeof h);
at += sizeof h;
size_t name = ((size_t)h.n_namesz + 3) & ~(size_t)3;
size_t desc = ((size_t)h.n_descsz + 3) & ~(size_t)3;
if (name > p->p_memsz - at || desc > p->p_memsz - at - name) break;
if (h.n_type == NT_GNU_BUILD_ID && h.n_namesz == 4 && !memcmp(note + at, "GNU", 4) &&
h.n_descsz && h.n_descsz <= 64) {
char id[129];
static const char hex[] = "0123456789abcdef";
for (size_t j = 0; j < h.n_descsz; j++) {
unsigned char b = note[at + name + j];
id[2 * j] = hex[b >> 4];
id[2 * j + 1] = hex[b & 15];
}
id[2 * h.n_descsz] = 0;
int n = snprintf(path, size, "/tmp/.droiddeck-svc-v2-%d-%s", svc_android_api, id);
return n > 0 && (size_t)n < size;
}
at += name + desc;
}
}
return 0;
}
static int svc_cache_read(const struct dl_phdr_info *info, const char *path) {
long fd = fp_open(AT_FDCWD, path, O_RDONLY | O_NOFOLLOW | O_CLOEXEC | O_NONBLOCK, 0);
if (fd < 0) return 0;
struct stat st;
struct svc_cached_sites table;
int valid = sc(SYS_fstat, fd, (long)&st, 0, 0, 0) == 0 && S_ISREG(st.st_mode) &&
st.st_uid == (uid_t)sc(SYS_getuid, 0, 0, 0, 0, 0) && !(st.st_mode & 0077) &&
st.st_size == sizeof table && sc(SYS_read, fd, (long)&table, sizeof table, 0, 0) == sizeof table;
sc(SYS_close, fd, 0, 0, 0, 0);
if (!valid || table.magic != 0x324356534444ULL || table.api != (unsigned)svc_android_api ||
!table.count || table.count > 256 || table.count > 256 - svc_planned) return 0;
for (unsigned i = 0; i < table.count; i++) {
uint64_t offset = table.offset[i];
int segment = svc_segment(info, offset, 4, PF_R | PF_X);
if ((offset & 3) || (i && offset <= table.offset[i - 1]) || !segment) return 0;
const ElfW(Phdr) *p = &info->dlpi_phdr[segment - 1];
const uint32_t *text = (const uint32_t *)(info->dlpi_addr + p->p_vaddr);
size_t at = (offset - p->p_vaddr) / 4;
if (text[at] != 0xd4000001U || !svc_supported(svc_site_number(text, at))) return 0;
}
for (unsigned i = 0; i < table.count; i++)
svc_plan[svc_planned++].site = (uint32_t *)(info->dlpi_addr + table.offset[i]);
return 1;
}
static void svc_cache_write(const struct dl_phdr_info *info, const char *path, size_t first) {
if (svc_planned == first || svc_planned == 256) return;
struct svc_cached_sites table = {.magic = 0x324356534444ULL, .api = svc_android_api, .count = svc_planned - first};
for (unsigned i = 0; i < table.count; i++)
table.offset[i] = (uintptr_t)svc_plan[first + i].site - info->dlpi_addr;
char temporary[256];
int n = snprintf(temporary, sizeof temporary, "%s.part-%d", path, getpid());
if (n <= 0 || n >= (int)sizeof temporary) return;
long fd = fp_open(AT_FDCWD, temporary, O_WRONLY | O_CREAT | O_EXCL | O_NOFOLLOW | O_CLOEXEC, 0600);
if (fd < 0) return;
long written = sc(SYS_write, fd, (long)&table, sizeof table, 0, 0);
sc(SYS_close, fd, 0, 0, 0, 0);
if (written == sizeof table) fp_rename(AT_FDCWD, temporary, AT_FDCWD, path, 0);
fp_unlink(AT_FDCWD, temporary, 0);
}
+128
View File
@@ -0,0 +1,128 @@
static int svc_android_api;
static int svc_host_android_api(void) {
const char *value = getenv("PROOT_FP_ANDROID_API");
if (!value || !*value) return 0;
char *end;
long api = strtol(value, &end, 10);
return !*end && api > 0 && api < 1000 ? (int)api : 0;
}
static long svc_memory(void *local, long remote, size_t size, int write) {
if (!size) return 0;
struct iovec here = {local, size}, there = {(void *)remote, size};
return sc(write ? SYS_process_vm_writev : SYS_process_vm_readv, getpid(),
(long)&here, 1, (long)&there, 1);
}
static int svc_path(long pointer, char *path) {
long n = svc_memory(path, pointer, PATH_MAX, 0);
return n > 0 && memchr(path, 0, (size_t)n) != NULL;
}
static long svc_output(long result, void *value, long out, size_t size) {
if (result < 0) return result;
return svc_memory(value, out, size, 1) == (long)size ? result : FP_SLOW;
}
static int svc_supported(int nr) {
switch (nr) {
case SYS_openat: case SYS_newfstatat: case SYS_statx: case SYS_readlinkat:
case SYS_faccessat: case SYS_mkdirat: case SYS_unlinkat: case SYS_renameat:
case SYS_renameat2: case SYS_symlinkat: case SYS_fchmodat: case SYS_utimensat:
case SYS_truncate: case SYS_statfs: case SYS_getxattr: case SYS_lgetxattr:
case SYS_listxattr: case SYS_llistxattr: case SYS_inotify_add_watch: case SYS_getcwd:
return 1;
case SYS_faccessat2:
return svc_android_api > 0;
case SYS_clone3:
return svc_android_api > 0 && svc_android_api < 35;
default: return 0;
}
}
static long svc_blocked(const long *r) {
if (!svc_android_api) return FP_SLOW;
if (r[8] == SYS_faccessat2 || (r[8] == SYS_clone3 && svc_android_api < 35)) return -ENOSYS;
return FP_SLOW;
}
static long svc_path_call(const long *r) {
#ifdef FP_SVC_TEST_HOOK
FP_SVC_TEST_HOOK(r);
#endif
char path[PATH_MAX], other[PATH_MAX];
struct timespec times[2];
const struct timespec *when = (const struct timespec *)r[2];
int first = 1, second = -1;
switch (r[8]) {
case SYS_getcwd: {
if (own_cwd(path) == FP_SLOW) return FP_SLOW;
size_t n = strlen(path) + 1;
if ((unsigned long)r[1] < n) return -ERANGE;
return svc_output(n, path, r[0], n);
}
case SYS_truncate: case SYS_statfs: case SYS_getxattr: case SYS_lgetxattr:
case SYS_listxattr: case SYS_llistxattr: first = 0; break;
case SYS_renameat: case SYS_renameat2: second = 3; break;
case SYS_symlinkat: first = 0; second = 2; break;
case SYS_utimensat:
if (when) {
if (svc_memory(times, r[2], sizeof times, 0) != sizeof times) return FP_SLOW;
for (int i = 0; i < 2; i++)
if (times[i].tv_nsec != UTIME_NOW && times[i].tv_nsec != UTIME_OMIT &&
(times[i].tv_nsec < 0 || times[i].tv_nsec >= 1000000000L)) return FP_SLOW;
when = times;
}
if (!r[1]) return fp_utimens((int)r[0], NULL, when, (int)r[3]);
break;
case SYS_inotify_add_watch: {
char link[64], target[64];
snprintf(link, sizeof link, "/proc/self/fd/%d", (int)r[0]);
long n = sc(SYS_readlinkat, AT_FDCWD, (long)link, (long)target, sizeof target - 1, 0);
if (n <= 0) return FP_SLOW;
target[n] = 0;
if (strcmp(target, "anon_inode:inotify")) return FP_SLOW;
break;
}
default:
if (!svc_supported((int)r[8])) return FP_SLOW;
}
if (!svc_path(r[first], path) || (second >= 0 && !svc_path(r[second], other))) return FP_SLOW;
switch (r[8]) {
case SYS_openat: return fp_open((int)r[0], path, (int)r[2], (mode_t)r[3]);
case SYS_newfstatat: {
struct stat st;
return svc_output(fp_stat((int)r[0], path, (int)r[3], &st), &st, r[2], sizeof st);
}
case SYS_statx: {
struct statx st;
return svc_output(fp_statx((int)r[0], path, (int)r[2], (unsigned)r[3], &st), &st, r[4], sizeof st);
}
case SYS_readlinkat: {
char text[PATH_MAX];
size_t written = 0;
long result = fp_readlink((int)r[0], path, text, (size_t)r[3], &written);
return svc_output(result, text, r[2], written);
}
case SYS_faccessat: return fp_access((int)r[0], path, (int)r[2], 0);
case SYS_mkdirat: return fp_mkdir((int)r[0], path, (mode_t)r[2]);
case SYS_unlinkat: return fp_unlink((int)r[0], path, (int)r[2]);
case SYS_renameat: case SYS_renameat2:
return fp_rename((int)r[0], path, (int)r[2], other, r[8] == SYS_renameat2 ? (unsigned)r[4] : 0);
case SYS_symlinkat: return fp_symlink(path, (int)r[1], other);
case SYS_fchmodat: return fp_chmod((int)r[0], path, (mode_t)r[2], 0);
case SYS_utimensat: return fp_utimens((int)r[0], path, when, (int)r[3]);
case SYS_truncate: return fp_truncate(path, (off_t)r[1]);
case SYS_statfs: {
struct statfs st;
return svc_output(fp_statfs(path, &st), &st, r[1], sizeof st);
}
case SYS_getxattr: case SYS_lgetxattr:
return fp_getxattr(path, (const char *)r[1], (void *)r[2], (size_t)r[3], r[8] == SYS_getxattr);
case SYS_listxattr: case SYS_llistxattr:
return fp_listxattr(path, (char *)r[1], (size_t)r[2], r[8] == SYS_listxattr);
case SYS_inotify_add_watch: return fp_inotify((int)r[0], path, (uint32_t)r[2]);
default: return FP_SLOW;
}
}
+269
View File
@@ -0,0 +1,269 @@
static inline int svc_number(uint32_t insn) {
if ((insn & 0xffe0001fU) == 0xd2800008U ||
(insn & 0xffe0001fU) == 0x52800008U)
return (insn >> 5) & 0xffff;
return -1;
}
static inline int svc_argument_instruction(uint32_t insn) {
if (insn == 0xd503201fU) return 1;
if ((insn & 0xffc00000U) == 0x29000000U ||
(insn & 0xffc00000U) == 0xa9000000U ||
(insn & 0xffc00000U) == 0xb9000000U ||
(insn & 0xffc00000U) == 0xf9000000U) return 1;
if ((insn & 31) == 8) return 0;
return (insn & 0xffe0ffe0U) == 0xaa0003e0U ||
(insn & 0xffe0ffe0U) == 0x2a0003e0U ||
(insn & 0xfffffc00U) == 0x93407c00U ||
(insn & 0x7f800000U) == 0x52800000U ||
(insn & 0x7f800000U) == 0x12800000U ||
(insn & 0xffc00000U) == 0xb9400000U ||
(insn & 0xffc00000U) == 0xf9400000U ||
(insn & 0xffc00000U) == 0xb9800000U ||
(insn & 0xff000000U) == 0x91000000U ||
(insn & 0xff000000U) == 0xd1000000U ||
(insn & 0xff000000U) == 0x11000000U ||
(insn & 0xff000000U) == 0x51000000U;
}
static inline int svc_site_number(const uint32_t *text, size_t at) {
for (size_t back = 1; back <= 8 && back <= at; back++) {
uint32_t insn = text[at - back];
int nr = svc_number(insn);
if (nr >= 0) return nr;
if (!svc_argument_instruction(insn)) break;
}
return -1;
}
static inline int svc_branch(uintptr_t from, uintptr_t to, uint32_t *out) {
int64_t delta = (int64_t)to - (int64_t)from;
if ((delta & 3) || delta < -0x8000000LL || delta >= 0x8000000LL) return 0;
*out = 0x14000000U | ((uint32_t)(delta / 4) & 0x03ffffffU);
return 1;
}
#if defined(__aarch64__) && !defined(FP_SVC_ENCODINGS_ONLY)
#include <asm/hwcap.h>
#include <link.h>
#include <sys/uio.h>
#include <signal.h>
static __thread uintptr_t svc_busy __attribute__((tls_model("initial-exec")));
static unsigned svc_sites, svc_pages;
static unsigned long svc_fast, svc_slow;
static unsigned long long svc_scan_ns, svc_patch_ns;
#include "svc-dispatch.h"
#include "svc-stack.h"
static long svc_original(const long *r) {
register long x0 __asm__("x0") = r[0];
register long x1 __asm__("x1") = r[1];
register long x2 __asm__("x2") = r[2];
register long x3 __asm__("x3") = r[3];
register long x4 __asm__("x4") = r[4];
register long x5 __asm__("x5") = r[5];
register long x8 __asm__("x8") = r[8];
__asm__ volatile("svc #0" : "+r"(x0) : "r"(x1), "r"(x2), "r"(x3), "r"(x4), "r"(x5), "r"(x8) : "memory", "cc");
return x0;
}
__attribute__((used, visibility("hidden")))
long fp_svc_dispatch(const long *r) {
int saved_errno = errno;
long result = svc_blocked(r);
uintptr_t frame = (uintptr_t)r;
if (result == FP_SLOW && (!svc_busy || frame >= svc_busy)) {
uintptr_t previous = svc_busy;
svc_busy = frame;
if (svc_stack_room(frame)) { result = svc_path_call(r); svc_busy = 0; }
else svc_busy = previous;
}
if (result == FP_SLOW) {
__atomic_fetch_add(&svc_slow, 1, __ATOMIC_RELAXED);
result = svc_original(r);
} else __atomic_fetch_add(&svc_fast, 1, __ATOMIC_RELAXED);
errno = saved_errno;
return result;
}
void fp_svc_entry(void) __attribute__((visibility("hidden")));
/* A replaced svc may have live values in every register, including NZCV and SIMD. */
__asm__(
".text\n.p2align 4\n.hidden fp_svc_entry\n.type fp_svc_entry,%function\nfp_svc_entry:\n"
".cfi_startproc\n.cfi_def_cfa sp, 16\n.cfi_return_column 16\n"
"bti j\nsub sp, sp, #800\n.cfi_def_cfa_offset 816\n"
"stp x0, x1, [sp, #0]\nstp x2, x3, [sp, #16]\n"
"stp x4, x5, [sp, #32]\nstp x6, x7, [sp, #48]\n"
"stp x8, x9, [sp, #64]\nstp x10, x11, [sp, #80]\n"
"stp x12, x13, [sp, #96]\nstp x14, x15, [sp, #112]\n"
"stp x18, x19, [sp, #128]\nstp x20, x21, [sp, #144]\n"
"stp x22, x23, [sp, #160]\nstp x24, x25, [sp, #176]\n"
"stp x26, x27, [sp, #192]\nstp x28, x29, [sp, #208]\n"
"stp x30, x17, [sp, #224]\nmrs x0, nzcv\nstr x0, [sp, #240]\n"
"ldr x0, [x17, #8]\nstr x0, [sp, #784]\n"
".cfi_offset 16, -32\n.cfi_offset 19, -680\n.cfi_offset 20, -672\n"
".cfi_offset 21, -664\n.cfi_offset 22, -656\n.cfi_offset 23, -648\n"
".cfi_offset 24, -640\n.cfi_offset 25, -632\n.cfi_offset 26, -624\n"
".cfi_offset 27, -616\n.cfi_offset 28, -608\n.cfi_offset 29, -600\n.cfi_offset 30, -592\n"
".cfi_offset 72, -432\n.cfi_offset 73, -416\n.cfi_offset 74, -400\n.cfi_offset 75, -384\n"
".cfi_offset 76, -368\n.cfi_offset 77, -352\n.cfi_offset 78, -336\n.cfi_offset 79, -320\n"
"mrs x0, fpcr\nstr x0, [sp, #768]\nmrs x0, fpsr\nstr x0, [sp, #776]\n"
"stp q0, q1, [sp, #256]\nstp q2, q3, [sp, #288]\n"
"stp q4, q5, [sp, #320]\nstp q6, q7, [sp, #352]\n"
"stp q8, q9, [sp, #384]\nstp q10, q11, [sp, #416]\n"
"stp q12, q13, [sp, #448]\nstp q14, q15, [sp, #480]\n"
"stp q16, q17, [sp, #512]\nstp q18, q19, [sp, #544]\n"
"stp q20, q21, [sp, #576]\nstp q22, q23, [sp, #608]\n"
"stp q24, q25, [sp, #640]\nstp q26, q27, [sp, #672]\n"
"stp q28, q29, [sp, #704]\nstp q30, q31, [sp, #736]\n"
"mov x0, sp\nbl fp_svc_dispatch\nstr x0, [sp]\n"
"ldp q0, q1, [sp, #256]\nldp q2, q3, [sp, #288]\n"
"ldp q4, q5, [sp, #320]\nldp q6, q7, [sp, #352]\n"
"ldp q8, q9, [sp, #384]\nldp q10, q11, [sp, #416]\n"
"ldp q12, q13, [sp, #448]\nldp q14, q15, [sp, #480]\n"
"ldp q16, q17, [sp, #512]\nldp q18, q19, [sp, #544]\n"
"ldp q20, q21, [sp, #576]\nldp q22, q23, [sp, #608]\n"
"ldp q24, q25, [sp, #640]\nldp q26, q27, [sp, #672]\n"
"ldp q28, q29, [sp, #704]\nldp q30, q31, [sp, #736]\n"
"ldr x0, [sp, #768]\nmsr fpcr, x0\nldr x0, [sp, #776]\nmsr fpsr, x0\n"
"ldr x0, [sp, #240]\nmsr nzcv, x0\n"
"ldp x0, x1, [sp, #0]\nldp x2, x3, [sp, #16]\n"
"ldp x4, x5, [sp, #32]\nldp x6, x7, [sp, #48]\n"
"ldp x8, x9, [sp, #64]\nldp x10, x11, [sp, #80]\n"
"ldp x12, x13, [sp, #96]\nldp x14, x15, [sp, #112]\n"
"ldp x18, x19, [sp, #128]\nldp x20, x21, [sp, #144]\n"
"ldp x22, x23, [sp, #160]\nldp x24, x25, [sp, #176]\n"
"ldp x26, x27, [sp, #192]\nldp x28, x29, [sp, #208]\n"
"ldr x30, [sp, #224]\nldr x16, [sp, #232]\n"
"add sp, sp, #800\n.cfi_def_cfa_offset 16\nbr x16\n.cfi_endproc\n.size fp_svc_entry, .-fp_svc_entry\n"
);
static int svc_emit(uint32_t *code, uint32_t *site, uint32_t *branch) {
uint32_t back;
if (!svc_branch((uintptr_t)site, (uintptr_t)code, branch) ||
!svc_branch((uintptr_t)(code + 5), (uintptr_t)(site + 1), &back)) return 0;
code[0] = 0xa9bf47f0U; /* stp x16, x17, [sp, #-16]! */
code[1] = 0x580000f0U; /* ldr x16, literal at +32 */
code[2] = 0x10000051U; /* adr x17, +8 */
code[3] = 0xd61f0200U; /* br x16 */
code[4] = 0xa8c147f0U; /* ldp x16, x17, [sp], #16 */
code[5] = back;
uintptr_t entry = (uintptr_t)fp_svc_entry;
uintptr_t resume = (uintptr_t)(site + 1);
memcpy(code + 6, &resume, sizeof resume);
memcpy(code + 8, &entry, sizeof entry);
return 1;
}
struct svc_site { uint32_t *site; uint32_t branch; };
static struct svc_site svc_plan[256];
static size_t svc_planned;
#include "svc-cache.h"
static int svc_object(struct dl_phdr_info *info, size_t size, void *unused) {
(void)size;
(void)unused;
const char *name = strrchr(info->dlpi_name, '/');
name = name ? name + 1 : info->dlpi_name;
if (strcmp(name, "libc.so.6") && strcmp(name, "ld-linux-aarch64.so.1")) return 0;
char cache[224];
int cached = svc_cache_name(info, cache, sizeof cache);
if (cached && svc_cache_read(info, cache)) { svc_cache_hits++; return 0; }
if (cached) svc_cache_misses++;
size_t first = svc_planned;
for (int i = 0; i < info->dlpi_phnum; i++) {
const ElfW(Phdr) *ph = &info->dlpi_phdr[i];
if (ph->p_type != PT_LOAD || !(ph->p_flags & PF_X)) continue;
uint32_t *text = (uint32_t *)(info->dlpi_addr + ph->p_vaddr);
for (size_t j = 1; j < ph->p_memsz / 4 && svc_planned < 256; j++) {
if (text[j] != 0xd4000001U) continue;
int nr = svc_site_number(text, j);
if (!svc_supported(nr)) continue;
svc_plan[svc_planned++].site = &text[j];
}
}
if (cached) svc_cache_write(info, cache, first);
return 0;
}
static uint32_t *svc_relay_page(uintptr_t site, size_t page) {
uintptr_t base = site & ~(uintptr_t)(page - 1);
for (uintptr_t delta = page; delta < 0x7ff0000UL; delta += 0x10000UL) {
for (int direction = -1; direction <= 1; direction += 2) {
uintptr_t at = direction < 0 ? base - delta : base + delta;
if (at < page) continue;
void *p = mmap((void *)at, page, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED_NOREPLACE, -1, 0);
if (p == (void *)at) return p;
if (p != MAP_FAILED) munmap(p, page);
}
}
return NULL;
}
#include "svc-publish.h"
static int svc_single_thread(void) {
struct entry { uint64_t ino; int64_t off; unsigned short len; unsigned char type; char name[]; };
char buf[4096] __attribute__((aligned(8)));
long fd = sc(SYS_openat, AT_FDCWD, (long)"/proc/self/task", O_RDONLY | O_DIRECTORY | O_CLOEXEC, 0, 0);
if (fd < 0) return 0;
int count = 0;
long n;
while ((n = sc(SYS_getdents64, fd, (long)buf, sizeof buf, 0, 0)) > 0) {
for (long at = 0; at < n;) {
const struct entry *e = (const struct entry *)(buf + at);
if (n - at < (long)offsetof(struct entry, name) + 1 ||
e->len < offsetof(struct entry, name) + 1 || e->len > n - at) { count = -1; goto done; }
if (e->name[0] >= '0' && e->name[0] <= '9' && ++count > 1) goto done;
at += e->len;
}
}
if (n < 0) count = -1;
done:
sc(SYS_close, fd, 0, 0, 0, 0);
return count == 1;
}
static void svc_stats(void) {
char line[256];
int n = snprintf(line, sizeof line, "fastpath-svc[%d]: %u sites, %u relay pages, %lu fast, %lu to proot; scan %llu us, patch %llu us, android %d, cache %u hit %u miss\n",
getpid(), svc_sites, svc_pages, svc_fast, svc_slow, svc_scan_ns / 1000, svc_patch_ns / 1000, svc_android_api, svc_cache_hits, svc_cache_misses);
if (n > 0 && write(2, line, n < (int)sizeof line ? n : (int)sizeof line - 1) < 0) {}
}
static unsigned long long svc_clock(void) {
struct timespec t;
if (sc(SYS_clock_gettime, CLOCK_MONOTONIC, (long)&t, 0, 0, 0) < 0) return 0;
return (unsigned long long)t.tv_sec * 1000000000ULL + t.tv_nsec;
}
static void svc_child(void) { svc_busy = 0; }
__attribute__((constructor)) static void svc_patch_init(void) {
const char *enabled = getenv("PROOT_FP_PATCH_SVC");
if ((enabled && strcmp(enabled, "1")) || !fp_ready()) return;
if (getauxval(AT_HWCAP) & HWCAP_SVE) return;
#ifdef HWCAP2_SME
if (getauxval(AT_HWCAP2) & HWCAP2_SME) return;
#endif
if (!svc_single_thread()) return;
if (pthread_atfork(NULL, NULL, svc_child)) return;
svc_android_api = svc_host_android_api();
size_t page = (size_t)sysconf(_SC_PAGESIZE);
if (page < 4096 || (page & (page - 1))) return;
int mem = (int)sc(SYS_openat, AT_FDCWD, (long)"/proc/self/mem", O_RDWR | O_CLOEXEC, 0, 0);
if (mem < 0) return;
unsigned long long start = svc_clock();
dl_iterate_phdr(svc_object, NULL);
svc_scan_ns = svc_clock() - start;
start = svc_clock();
svc_publish(mem, page);
sc(SYS_close, mem, 0, 0, 0, 0);
svc_patch_ns = svc_clock() - start;
if (getenv("PROOT_FP_STATS")) atexit(svc_stats);
}
#endif
+28
View File
@@ -0,0 +1,28 @@
static void svc_publish(int mem, size_t page) {
for (size_t first = 0; first < svc_planned;) {
uint32_t *relay = svc_relay_page((uintptr_t)svc_plan[first].site, page);
if (!relay) { first++; continue; }
size_t end = first;
for (; end < svc_planned && (end - first + 1) * 40 <= page; end++) {
uint32_t *code = relay + (end - first) * 10;
if (!svc_emit(code, svc_plan[end].site, &svc_plan[end].branch)) break;
}
__builtin___clear_cache((char *)relay, (char *)relay + page);
if (end == first || mprotect(relay, page, PROT_READ | PROT_EXEC)) {
munmap(relay, page);
first++;
continue;
}
unsigned patched = 0;
for (size_t i = first; i < end; i++) {
struct svc_site *s = &svc_plan[i];
long written = sc(SYS_pwrite64, mem, (long)&s->branch, sizeof s->branch, (long)s->site, 0);
if (written != sizeof s->branch) continue;
__builtin___clear_cache((char *)s->site, (char *)(s->site + 1));
patched++;
}
if (patched) { svc_sites += patched; svc_pages++; }
else munmap(relay, page);
first = end;
}
}
+83
View File
@@ -0,0 +1,83 @@
#include <sys/resource.h>
#include <string.h>
static __thread uintptr_t svc_stack_low __attribute__((tls_model("initial-exec")));
static __thread uintptr_t svc_stack_high __attribute__((tls_model("initial-exec")));
static __thread uintptr_t svc_stack_gap __attribute__((tls_model("initial-exec")));
static __thread int svc_stack_main __attribute__((tls_model("initial-exec")));
static int svc_hex(const char **text, uintptr_t *value) {
const char *p = *text;
uintptr_t n = 0;
int digits = 0;
for (;; p++) {
unsigned c = (unsigned char)*p;
unsigned d = c >= '0' && c <= '9' ? c - '0' : c >= 'a' && c <= 'f' ? c - 'a' + 10 : 16;
if (d == 16) break;
if (++digits > (int)(2 * sizeof n)) return 0;
n = (n << 4) | d;
}
*text = p;
*value = n;
return digits > 0;
}
static int svc_stack_line(const char *line, uintptr_t sp, uintptr_t *low, uintptr_t *high) {
uintptr_t a, b;
if (!svc_hex(&line, &a) || *line++ != '-' || !svc_hex(&line, &b) ||
*line++ != ' ' || line[0] != 'r' || line[1] != 'w' || a >= b || sp < a || sp >= b) return 0;
*low = a;
*high = b;
return 1;
}
static __attribute__((noinline)) int svc_stack_find(uintptr_t sp) {
char chunk[512], line[96];
size_t at = 0;
long fd = sc(SYS_openat, AT_FDCWD, (long)"/proc/self/maps", O_RDONLY | O_CLOEXEC, 0, 0);
if (fd < 0) return 0;
long n;
uintptr_t low = 0, high = 0, previous = 0;
int found = 0;
while (!found && (n = sc(SYS_read, fd, (long)chunk, sizeof chunk, 0, 0)) > 0) {
for (long i = 0; i < n; i++) {
if (chunk[i] == '\n') {
line[at] = 0;
found = svc_stack_line(line, sp, &low, &high);
if (!found) {
const char *p = line;
uintptr_t a, b;
if (svc_hex(&p, &a) && *p++ == '-' && svc_hex(&p, &b) && b <= sp) previous = b;
}
at = 0;
if (found) break;
} else if (at < sizeof line - 1) line[at++] = chunk[i];
}
}
sc(SYS_close, fd, 0, 0, 0, 0);
if (found) {
svc_stack_low = 0;
svc_stack_high = high;
svc_stack_main = strstr(line, "[stack]") != NULL;
svc_stack_gap = previous < low && low - previous > 16 * 1024 * 1024 ? previous + 16 * 1024 * 1024 : low;
svc_stack_low = low;
}
return found;
}
static int svc_stack_room(uintptr_t sp) {
stack_t alt;
if (sc(SYS_sigaltstack, 0, (long)&alt, 0, 0, 0) < 0 || (alt.ss_flags & SS_ONSTACK)) return 0;
if (!svc_stack_low || sp < svc_stack_low || sp >= svc_stack_high)
if (!svc_stack_find(sp)) return 0;
uintptr_t low = svc_stack_low;
if (svc_stack_main) {
struct rlimit limit;
if (sc(SYS_prlimit64, 0, RLIMIT_STACK, 0, (long)&limit, 0) < 0) return 0;
uintptr_t grow = limit.rlim_cur >= svc_stack_high ? 0 : svc_stack_high - limit.rlim_cur;
if (grow < svc_stack_gap) grow = svc_stack_gap;
if (grow < low) low = grow;
}
/* Path translation nests several PATH_MAX-sized frames. */
return sp - low >= 96 * 1024;
}
+290
View File
@@ -0,0 +1,290 @@
from pathlib import Path
import shutil
import subprocess
import tempfile
import unittest
HEADER = Path(__file__).resolve().parents[1] / "proot/fastpath/svc-patch.h"
class FastPathSvcEncodingTest(unittest.TestCase):
def compile_check(self, source, header):
if shutil.which("cc") is None:
self.skipTest("cc is not available")
with tempfile.TemporaryDirectory() as directory:
program = Path(directory) / "check.c"
program.write_text(source.replace('"HEADER"', '"' + (HEADER.parent / header).as_posix() + '"'))
output = Path(directory) / "check"
subprocess.run(["cc", "-O2", "-Wall", "-Wextra", "-Werror", str(program), "-o", str(output)], check=True)
subprocess.run([str(output)], check=True, timeout=10)
def test_benchmark_checks_execute_with_ndebug(self):
self.compile_check(r'''
#define NDEBUG
#include "HEADER"
int main(void) {
int calls = 0;
assert(++calls == 1);
return calls != 1;
}
''', "../bench/checks.h")
def test_stack_bounds_and_alternate_stack_fallback(self):
self.compile_check(r'''
#define _GNU_SOURCE
#include <assert.h>
#include <stdint.h>
#include <stddef.h>
#include <signal.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/syscall.h>
#include <sys/resource.h>
static rlim_t stack_limit = 1024 * 1024;
static long sc(long n, long a, long b, long c, long d, long e) {
if (n == SYS_prlimit64) {
if (!stack_limit) return -1;
((struct rlimit *)d)->rlim_cur = stack_limit;
return 0;
}
return syscall(n,a,b,c,d,e);
}
#include "HEADER"
int main(void) {
uintptr_t low, high;
assert(svc_stack_line("1000-9000 rw-p 00000000 00:00 0", 0x8000, &low, &high));
assert(low == 0x1000 && high == 0x9000);
assert(!svc_stack_line("1000-9000 r--p", 0x8000, &low, &high));
assert(!svc_stack_line("1000-9000 rw-p", 0x9000, &low, &high));
assert(!svc_stack_line("9000-1000 rw-p", 0x8000, &low, &high));
assert(!svc_stack_line("fffffffffffffffff-9000 rw-p", 0x8000, &low, &high));
assert(!svc_stack_line("", 0x8000, &low, &high));
char local;
assert(svc_stack_find((uintptr_t)&local));
assert((uintptr_t)&local >= svc_stack_low && (uintptr_t)&local < svc_stack_high);
svc_stack_main = 0; svc_stack_low = 0x1000; svc_stack_high = 0x100000;
assert(!svc_stack_room(0x1000 + 96 * 1024 - 1));
assert(svc_stack_room(0x1000 + 96 * 1024));
svc_stack_main = 1; svc_stack_low = 0xf0000; svc_stack_gap = 0x1000;
assert(svc_stack_room(0xff000));
stack_limit = 64 * 1024;
assert(!svc_stack_room(0xff000));
stack_limit = 0;
assert(!svc_stack_room(0xff000));
return 0;
}
''', "svc-stack.h")
def test_patch_publication_failure_and_partial_success(self):
self.compile_check(r'''
#define _GNU_SOURCE
#include <assert.h>
#include <stdint.h>
#include <stddef.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#include <sys/syscall.h>
static struct svc_site { uint32_t *site; uint32_t branch; } svc_plan[3];
static size_t svc_planned = 3;
static unsigned svc_sites, svc_pages;
static int fail_alloc, fail_emit, fail_rx, fail_write, writes, live, rx;
static uint32_t *svc_relay_page(uintptr_t site, size_t page) {
(void)site;
if (fail_alloc) return NULL;
live++; rx = 0;
return calloc(1, page);
}
static int svc_emit(uint32_t *code, uint32_t *site, uint32_t *branch) {
(void)code; (void)site;
*branch = 0x14000001;
return !fail_emit;
}
static int mock_rx(void *p, size_t size, int prot) {
(void)p; (void)size;
assert(prot == (PROT_READ | PROT_EXEC));
if (fail_rx) return -1;
rx = 1;
return 0;
}
static int mock_unmap(void *p, size_t size) { (void)size; free(p); live--; return 0; }
static long sc(long nr, long fd, long buf, long len, long at, long unused) {
(void)fd; (void)unused;
assert(nr == SYS_pwrite64 && len == 4 && rx);
if (++writes == fail_write) return -1;
memcpy((void *)at, (void *)buf, len);
return len;
}
#define mprotect mock_rx
#define munmap mock_unmap
#include "HEADER"
int main(void) {
static uint32_t sites[3];
for (int mode = 0; mode < 5; mode++) {
for (int i=0; i<3; i++) { sites[i]=0xd4000001; svc_plan[i].site=&sites[i]; }
svc_sites=svc_pages=writes=live=rx=0;
fail_alloc=mode==0; fail_emit=mode==1; fail_rx=mode==2; fail_write=mode==3 ? 2 : 0;
svc_publish(7,4096);
if (mode<3) assert(!svc_sites && !svc_pages && !writes && !live);
else assert(svc_sites==(mode==3 ? 2U : 3U) && svc_pages==1 && live==1);
for (int i=0; i<3; i++) assert(sites[i]==(mode<3 || (mode==3 && i==1) ? 0xd4000001 : 0x14000001));
}
return 0;
}
''', "svc-publish.h")
def test_instruction_decoding_and_branch_limits(self):
if shutil.which("cc") is None:
self.skipTest("cc is not available")
source = r'''
#include <assert.h>
#include <stdint.h>
#include <stddef.h>
#define FP_SVC_ENCODINGS_ONLY
#include "HEADER"
int main(void) {
assert(svc_number(0xd2800708) == 56);
assert(svc_number(0x528009e8) == 79);
assert(svc_number(0xd2800709) == -1);
assert(svc_number(0xd2a00708) == -1);
assert(svc_number(0xf2800708) == -1);
assert(svc_number(0xaa0003e8) == -1);
const uint32_t create[] = {0xd2800708, 0x29047fe0, 0x92800c60, 0xd4000001};
const uint32_t create_at[] = {0xd2800708, 0x29047fe3, 0xb9404be3, 0xd4000001};
const uint32_t xstat[] = {0xd28009e8, 0xaa0203e1, 0xaa0303e2, 0x93407c83, 0xd4000001};
const uint32_t overwritten[] = {0xd2800708, 0xaa0003e8, 0xd4000001};
const uint32_t loaded[] = {0xd2800708, 0xf94003e8, 0xd4000001};
const uint32_t branch[] = {0xd2800708, 0x14000001, 0xd4000001};
const uint32_t call[] = {0xd2800708, 0x94000001, 0xd4000001};
const uint32_t shift[] = {0xd2800708, 0xd2a00708, 0xd4000001};
const uint32_t robust[] = {0xd2800c68, 0xf9000260, 0xa9000261, 0xf9400260, 0x9100e260, 0xd4000001};
const uint32_t writeback[] = {0xd2800708, 0xa9bf07e0, 0xd4000001};
const uint32_t added[] = {0xd2800708, 0x91000408, 0xd4000001};
assert(svc_site_number(create, 3) == 56);
assert(svc_site_number(create_at, 3) == 56);
assert(svc_site_number(xstat, 4) == 79);
assert(svc_site_number(overwritten, 2) == -1);
assert(svc_site_number(loaded, 2) == -1);
assert(svc_site_number(branch, 2) == -1);
assert(svc_site_number(call, 2) == -1);
assert(svc_site_number(shift, 2) == -1);
assert(svc_site_number(robust, 5) == 99);
assert(svc_site_number(writeback, 2) == -1);
assert(svc_site_number(added, 2) == -1);
uint32_t out;
uintptr_t pc = 0x10000000;
assert(svc_branch(pc, pc + 4, &out) && out == 0x14000001);
assert(svc_branch(pc, pc - 4, &out) && out == 0x17ffffff);
assert(svc_branch(pc, pc + 0x7fffffc, &out) && out == 0x15ffffff);
assert(svc_branch(pc, pc - 0x8000000, &out) && out == 0x16000000);
assert(!svc_branch(pc, pc + 0x8000000, &out));
assert(!svc_branch(pc, pc - 0x8000004, &out));
assert(!svc_branch(pc, pc + 2, &out));
return 0;
}
'''
with tempfile.TemporaryDirectory() as directory:
work = Path(directory)
program = work / "check.c"
program.write_text(source.replace("HEADER", HEADER.as_posix()))
output = work / "check"
subprocess.run(["cc", "-O2", "-Wall", "-Wextra", "-Werror", str(program), "-o", str(output)], check=True)
subprocess.run([str(output)], check=True, timeout=10)
def test_cache_revalidates_image_offsets_and_untrusted_files(self):
if shutil.which("cc") is None:
self.skipTest("cc is not available")
source = r'''
#define _GNU_SOURCE
#include <assert.h>
#include <stdint.h>
#include <stddef.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>
#include <unistd.h>
#include <link.h>
#include <sys/stat.h>
#include <sys/syscall.h>
#define FP_SVC_ENCODINGS_ONLY
#include "HEADER"
static int svc_android_api = 33;
static struct { uint32_t *site; } svc_plan[256];
static size_t svc_planned;
static int svc_supported(int nr) { return nr == 56 || nr == 79; }
#define sc(n,a,b,c,d,e) syscall(n,a,b,c,d,e)
static long fp_open(int d, const char *p, int f, mode_t m) { return openat(d,p,f,m); }
static long fp_rename(int d, const char *p, int e, const char *q, int f) { (void)f; return renameat(d,p,e,q); }
static long fp_unlink(int d, const char *p, int f) { return unlinkat(d,p,f); }
#include "CACHE"
static void store(const char *path, struct svc_cached_sites *table) {
int fd = open(path, O_WRONLY | O_TRUNC);
assert(fd >= 0 && write(fd, table, sizeof *table) == sizeof *table && close(fd) == 0);
}
int main(void) {
uint32_t image[32] = {0xd2800708, 0xd4000001, 0xd28009e8, 0xd4000001};
ElfW(Phdr) ph[] = {{.p_type=PT_LOAD, .p_flags=PF_R|PF_X, .p_memsz=sizeof image},
{.p_type=PT_NOTE, .p_vaddr=64, .p_memsz=24}};
struct dl_phdr_info info = {.dlpi_addr=(uintptr_t)image, .dlpi_phdr=ph, .dlpi_phnum=2};
ElfW(Nhdr) note = {.n_type=NT_GNU_BUILD_ID, .n_namesz=4, .n_descsz=8};
memcpy((char *)image+64, &note, sizeof note);
memcpy((char *)image+76, "GNU", 4);
uint64_t id = getpid();
memcpy((char *)image+80, &id, 8);
char path[256], other[256];
assert(svc_cache_name(&info, path, sizeof path));
svc_android_api = 35;
assert(svc_cache_name(&info, other, sizeof other) && strcmp(path, other));
svc_android_api = 33;
setenv("PROOT_FP_SVC_CACHE", "0", 1);
assert(!svc_cache_name(&info, other, sizeof other));
unsetenv("PROOT_FP_SVC_CACHE");
ph[1].p_memsz = sizeof image;
assert(!svc_cache_name(&info, other, sizeof other));
ph[1].p_memsz = 24;
svc_plan[0].site = image+1; svc_plan[1].site = image+3; svc_planned=2;
svc_cache_write(&info, path, 0);
svc_planned=0;
assert(svc_cache_read(&info, path) && svc_planned==2 && svc_plan[1].site==image+3);
struct svc_cached_sites valid, table;
int fd = open(path, O_RDONLY);
assert(fd >= 0 && read(fd, &valid, sizeof valid)==sizeof valid && close(fd)==0);
#define REJECT(change) do { table=valid; change; store(path,&table); svc_planned=0; \
assert(!svc_cache_read(&info,path) && svc_planned==0); } while(0)
REJECT(table.magic++);
REJECT(table.api=35);
REJECT(table.count=0);
REJECT(table.count=257);
REJECT(table.offset[0]=3);
REJECT(table.offset[0]=sizeof image);
REJECT(table.offset[0]=UINT64_MAX-3);
REJECT(table.offset[1]=table.offset[0]);
REJECT(table.offset[0]=12; table.offset[1]=4);
store(path,&valid);
image[1]=0x14000001;
assert(!svc_cache_read(&info,path));
image[1]=0xd4000001; image[0]=0xd2800008;
assert(!svc_cache_read(&info,path));
image[0]=0xd2800708;
assert(chmod(path,0644)==0 && !svc_cache_read(&info,path));
assert(chmod(path,0600)==0 && truncate(path,8)==0 && !svc_cache_read(&info,path));
assert(unlink(path)==0 && symlink("/dev/zero",path)==0 && !svc_cache_read(&info,path));
assert(unlink(path)==0 && mkdir(path,0700)==0 && !svc_cache_read(&info,path));
assert(rmdir(path)==0);
svc_planned=0;
svc_cache_write(&info,path,0);
assert(access(path,F_OK)<0);
(void)svc_cache_hits; (void)svc_cache_misses;
return 0;
}
'''
with tempfile.TemporaryDirectory() as directory:
work = Path(directory)
program = work / "cache.c"
program.write_text(source.replace('"HEADER"', '"' + HEADER.as_posix() + '"')
.replace('"CACHE"', '"' + HEADER.with_name("svc-cache.h").as_posix() + '"'))
output = work / "cache"
subprocess.run(["cc", "-O2", "-Wall", "-Wextra", "-Werror", str(program), "-o", str(output)], check=True)
subprocess.run([str(output)], check=True, timeout=10)