diff --git a/docs/development/proroot-analysis.md b/docs/development/proroot-analysis.md index 1636206844..f3bdb0e393 100644 --- a/docs/development/proroot-analysis.md +++ b/docs/development/proroot-analysis.md @@ -287,7 +287,7 @@ The installed PRoot pack emitted no profiling start/counter records despite the queued profiling setting; the profiler report had `hasCounters: false`. That is a measurement gap, not evidence of zero remaining traps. -### Expanded relay coverage +### Expanded relay coverage (before the safety follow-up) The subsequent revision keeps patching enabled by default and shares the existing wrapper translators with 20 path syscall numbers: openat, newfstatat, statx, @@ -449,3 +449,87 @@ workflow in [proot-performance](proot-performance.md) for a fresh session. No ex environment setting is needed; use `PROOT_FP_PATCH_SVC=0` for the unpatched comparison. `svc-storage` takes a new, unused directory name; it creates and removes its own test files and directory. It writes 64 MiB plus 16 MiB of file extensions per run. + +### Safety follow-up (2026-10-28) + +Path patching remains enabled by default. Robust-list registration/query sites are +now deliberately excluded: a device regression test reproduced disagreement +between the relay's private TLS head and an unpatched raw get_robust_list call in +the preceding implementation. Both directions now use the existing PRoot/kernel +route, including cleared heads. This preserves the host's actual return policy: +the installed Thor runtime accepts set_robust_list with zero, whereas the removed +emulation returned ENOSYS. PRoot patch 0016 can instead return ENOSYS when Android +blocks registration; neither policy is replaced by private relay state. + +The build-ID cache is version 2 because the accepted syscall set changed. The +current glibc 2.43/API-33 build patches 47 sites rather than 49. faccessat2 and +pre-Android-15 clone3 fallback acceleration remain; droiddeck-fsync is unchanged. +The earlier 138-to-113-microsecond thread result belongs to the retired robust-list +implementation and must not be claimed for the current code. + +Relays now save the original post-SVC program counter and supply unwind metadata +for the saved caller registers. Device tests force an unwind from inside the +translator through a patched libc frame and verify caller cleanup. A signal handler +also exits with siglongjmp; a later call recovers the abandoned recursion guard +and resumes acceleration. These tests supplement deferred and blocked-FIFO +cancellation; they are not a claim of arbitrary asynchronous-cancellation safety. + +Compiler stack-usage output identified multiple nested PATH_MAX-sized frames. +Before entering them, a small dispatcher checks the active alternate signal stack +and cached writable stack mapping. Alternate-stack calls and calls with less than +96 KiB remaining use PRoot. Main-stack growth respects the current RLIMIT_STACK +and leaves space above the preceding mapping. The implementation adds only a few +TLS words, not a large private translation stack. Tests cover a restricted thread +stack, normal/alternate-stack signals, main-stack growth and a reduced/unavailable +stack limit. User-managed stacks and arbitrary mid-helper non-local exits remain +broader compatibility risks, not proven consequence-free cases. + +On Thor/Android 13, the embedded and shared-PIC matrices pass on glibc 2.43; +the shared-PIC matrix also passes with a separately loaded glibc 2.39. They cover register/FP/errno preservation, mixed raw +and relay robust calls, pointer and page boundaries, readlink buffers of target +length minus one through plus two, thread churn, fork, signal/non-local exits, +forced unwinding and limited stack space. Readlink preserves PRoot's existing +translated-link length behavior; bytes beyond the reported payload are unspecified. + +Test-process seccomp filters separately deny opening /proc/self/mem, nearby relay +mappings, executable mprotect and patch writes. All four device cases leave zero +patched sites and a working original libc syscall. Host injection additionally +checks that one failed site write permits other sites to install. Concurrent +atomic symlink replacement passes 3,000 internal stat/readlink iterations at both +zero and default cache TTLs. These are bounded tests; they do not prove every +possible filesystem race or reproduce a real SELinux execmod denial. + +Five new alternating pairs measured: + +| operation | wrappers only | hardened relays | +|---|---:|---:| +| libc-internal stat | 50.1 us | 5.9 us | +| libc-internal open/close | 51.9 us | 7.1 us | +| libc faccessat | 52.0 us | 20.2 us | +| libc getcwd | 51.7 us | 4.6 us | +| thread create/join | 129 us | 144 us | +| complete tiny exec/wait | 5.04 ms | 5.96 ms | + +The metadata gains survive the safety checks. This run shows no thread-creation +gain, and the roughly 0.9 ms tiny-exec cost remains. The earlier SD download and +flush results are historical measurements of the preceding build; no new claim +that SD media writes or whole Steam installs become reliably faster follows from +these syscall results. Android 15+ ARM64 and 16 KiB-page hardware remain untested. + +Three alternating pairs also measured session start through the Steam Big Picture +Home page becoming ready, using its DOM/Recent Games content rather than merely +the compositor's first frame. Off runs took 48.76/43.47/47.02 seconds; on runs took +43.43/45.25/50.40 seconds. The 47.02-to-45.25-second medians sit within overlapping +run-to-run variation and do not establish a reliable whole-client startup win. +These use the signed bundled-library build without overrides; the final stack-gap +guard only increases the reserved distance above other mappings. + +At Home-ready plus 20 seconds, three one-second-spaced process-memory samples per +run gave Steam/CEF-only median PSS of 1,223.1 MiB off and 1,233.9 MiB on. Including +their active Python/Wine descendants gave 1,243.7/1,254.6 MiB. The five Steam/CEF +processes were selected by executable name; descendants were included by parent +PID. Per-run Steam/CEF medians ranged 1,217.9–1,240.9 MiB off and +1,194.8–1,244.9 MiB on. The roughly 11 MiB median increase is smaller than this +variation, but it is not evidence of zero memory cost. PSS avoids counting shared +pages in full for each process. Startup and memory comparisons used one unchanged +APK; no downloads or games were initiated during them. diff --git a/tools/proot/bench/svc-check.c b/tools/proot/bench/svc-check.c index 36d439390b..664181d606 100644 --- a/tools/proot/bench/svc-check.c +++ b/tools/proot/bench/svc-check.c @@ -1,10 +1,25 @@ +#if defined(__aarch64__) +static void interruption_hook(const long *); +#define FP_SVC_TEST_HOOK interruption_hook +#endif #include "../fastpath/fastpath.c" #include #include #include #include +#include #if defined(__aarch64__) +static void (*interrupt_path)(void); +static void interruption_hook(const long *r) { + (void)r; + if (interrupt_path) { + void (*hook)(void) = interrupt_path; + interrupt_path = NULL; + hook(); + } +} + struct snapshot { long x[32]; unsigned char q[32][16]; unsigned long nzcv, fpcr, fpsr; }; extern uint32_t svc_check_site; extern void svc_register_probe(struct snapshot *, const long *); @@ -77,6 +92,23 @@ static void fault(long nr, long a, long b, long c, long d, long e, long f) { check(call, want); } +static void readlink_boundaries(const char *path) { + char expected[PATH_MAX + 8], actual[PATH_MAX + 8]; + long args[] = {AT_FDCWD, (long)path, (long)expected, PATH_MAX, 0, 0, 0, 0, SYS_readlinkat}; + long length = svc_original(args); + assert(length > 0 && length < PATH_MAX - 2); + for (long size = length - 1; size <= length + 2; size++) { + memset(expected, 0x5a, sizeof expected); memset(actual, 0x5a, sizeof actual); + args[3] = size; + long want = svc_original(args); + long got = raw(SYS_readlinkat, AT_FDCWD, (long)path, (long)actual, size, 0, 0); + if (got != want || (want > 0 && memcmp(expected, actual, (size_t)want))) + fprintf(stderr, "readlink %s size %ld: relay %ld, proot %ld\n", path, size, got, want); + assert(got == want && (want <= 0 || !memcmp(expected, actual, (size_t)want))); + for (size_t i = (size_t)size; i < sizeof actual; i++) assert(actual[i] == 0x5a); + } +} + static void path_checks(void) { char work[] = "/tmp/proot-expanded-check-XXXXXX"; assert(mkdtemp(work)); @@ -91,6 +123,8 @@ static void path_checks(void) { assert(raw(SYS_statx, AT_FDCWD, (long)"dir/file", 0, STATX_BASIC_STATS, (long)&sx, 0) == 0 && sx.stx_size == 5); assert(raw(SYS_faccessat, AT_FDCWD, (long)"dir/file", R_OK, 0, 0, 0) == 0); assert(raw(SYS_symlinkat, (long)"file", AT_FDCWD, (long)"dir/link", 0, 0, 0) == 0); + readlink_boundaries("dir/link"); + readlink_boundaries("/proc/self/exe"); char text[PATH_MAX]; memset(text, '#', sizeof text); assert(raw(SYS_readlinkat, AT_FDCWD, (long)"dir/link", (long)text, sizeof text, 0, 0) == 4); @@ -145,14 +179,21 @@ static void robust_checks(void) { size_t size = 0; assert(raw(SYS_get_robust_list, 0, (long)&original, (long)&size, 0, 0, 0) == 0); struct robust_list_head head = {0}; - assert(raw(SYS_set_robust_list, (long)&head, sizeof head, 0, 0, 0, 0) == -ENOSYS); + long set_result = raw(SYS_set_robust_list, (long)&head, sizeof head, 0, 0, 0, 0); + assert(set_result == 0 || set_result == -ENOSYS); + long raw_head = 0; + long direct[] = {0, (long)&raw_head, (long)&size, 0, 0, 0, 0, 0, SYS_get_robust_list}; + assert(svc_original(direct) == 0 && raw_head == (long)&head); long got = 0; assert(raw(SYS_get_robust_list, gettid(), (long)&got, (long)&size, 0, 0, 0) == 0); assert(got == (long)&head && size == sizeof head); assert(raw(SYS_get_robust_list, 0, 1, (long)&size, 0, 0, 0) == -EFAULT); - assert(raw(SYS_set_robust_list, 0, sizeof head, 0, 0, 0, 0) == -ENOSYS); - assert(raw(SYS_get_robust_list, 0, (long)&got, (long)&size, 0, 0, 0) == -ENOSYS); - assert(raw(SYS_set_robust_list, original, sizeof head, 0, 0, 0, 0) == -ENOSYS); + assert(raw(SYS_set_robust_list, 0, sizeof head, 0, 0, 0, 0) == set_result); + long cleared = raw(SYS_get_robust_list, 0, (long)&got, (long)&size, 0, 0, 0); + assert(cleared == -ENOSYS || (cleared == 0 && got == 0)); + direct[0] = original; direct[1] = sizeof head; direct[8] = SYS_set_robust_list; + assert(svc_original(direct) == set_result); + assert(raw(SYS_get_robust_list, 0, (long)&got, (long)&size, 0, 0, 0) == 0 && original == got); assert(raw(SYS_faccessat2, AT_FDCWD, (long)"/usr/lib/libc.so.6", R_OK, 0, 0, 0) == -ENOSYS); if (svc_android_api < 35) assert(raw(SYS_clone3, 0, 0, 0, 0, 0, 0) == -ENOSYS); int api = svc_android_api; @@ -161,7 +202,8 @@ static void robust_checks(void) { svc_android_api = 0; assert(!svc_supported(SYS_clone3) && !svc_supported(SYS_set_robust_list)); svc_android_api = api; - puts("PASS: robust-list TLS, errors, faccessat2, clone3 platform guard"); + assert(!svc_supported(SYS_set_robust_list) && !svc_supported(SYS_get_robust_list)); + puts("PASS: mixed relay/raw robust-list callers, errors, faccessat2, clone3 platform guard"); } static void *thread_check(void *unused) { @@ -172,13 +214,14 @@ static void *thread_check(void *unused) { size_t size; assert(raw(SYS_get_robust_list, 0, (long)&head, (long)&size, 0, 0, 0) == 0 && head && size == 3 * sizeof(long)); struct robust_list_head private = {0}; - assert(raw(SYS_set_robust_list, (long)&private, sizeof private, 0, 0, 0, 0) == -ENOSYS); + long result = raw(SYS_set_robust_list, (long)&private, sizeof private, 0, 0, 0, 0); + assert(result == 0 || result == -ENOSYS); for (int i = 0; i < 10; i++) { long got; sched_yield(); assert(raw(SYS_get_robust_list, 0, (long)&got, (long)&size, 0, 0, 0) == 0 && got == (long)&private); } - assert(raw(SYS_set_robust_list, head, sizeof private, 0, 0, 0, 0) == -ENOSYS); + assert(raw(SYS_set_robust_list, head, sizeof private, 0, 0, 0, 0) == result); } const long args[] = {AT_FDCWD, (long)"/usr/lib/libc.so.6", (long)&st, 0, 0, 0, SYS_newfstatat}; for (int i = 0; i < 100; i++) check(args, 0); @@ -203,6 +246,75 @@ static void on_signal(int signal) { errno = saved; } +static sigjmp_buf interrupted; +static void jump_signal(int signal) { (void)signal; siglongjmp(interrupted, 1); } +static void jump_path(void) { assert(raise(SIGUSR2) == 0); abort(); } +static void exit_path(void) { pthread_exit(PTHREAD_CANCELED); } +static void mark_cleanup(void *arg) { *(int *)arg = 1; } +struct unwind_check { int cleaned; int (*stat)(int, const char *, struct stat *, int); }; +static void *unwind_worker(void *arg) { + struct unwind_check *test = arg; + struct stat st; + pthread_cleanup_push(mark_cleanup, &test->cleaned); + interrupt_path = exit_path; + test->stat(AT_FDCWD, "/usr/lib/libc.so.6", &st, 0); + pthread_cleanup_pop(1); + abort(); +} + +static void interruption_checks(void) { + void *libc = dlopen("libc.so.6", RTLD_NOW); + struct unwind_check test = {.stat = dlsym(libc, "fstatat")}; + assert(test.stat); + struct sigaction old, action = {.sa_handler = jump_signal}; + sigemptyset(&action.sa_mask); + assert(sigaction(SIGUSR2, &action, &old) == 0); + sigset_t deliver, old_mask; + sigemptyset(&deliver); sigaddset(&deliver, SIGUSR2); + assert(pthread_sigmask(SIG_UNBLOCK, &deliver, &old_mask) == 0); + if (!sigsetjmp(interrupted, 1)) { + struct stat st; + interrupt_path = jump_path; + test.stat(AT_FDCWD, "/usr/lib/libc.so.6", &st, 0); + abort(); + } + assert(svc_busy); + struct stat st; + unsigned long before = svc_fast; + assert(test.stat(AT_FDCWD, "/usr/lib/libc.so.6", &st, 0) == 0); + assert(!svc_busy && svc_fast > before); + assert(sigaction(SIGUSR2, &old, NULL) == 0 && pthread_sigmask(SIG_SETMASK, &old_mask, NULL) == 0); + pthread_t thread; + void *result; + assert(pthread_create(&thread, NULL, unwind_worker, &test) == 0); + assert(pthread_join(thread, &result) == 0 && result == PTHREAD_CANCELED && test.cleaned); + assert(dlclose(libc) == 0); + puts("PASS: signal siglongjmp recovery, forced unwind through patched libc with caller cleanup"); +} + +static void *small_stack_worker(void *unused) { + size_t consume = (size_t)(uintptr_t)unused - 48 * 1024; + volatile char *padding = __builtin_alloca(consume); + padding[0] = padding[consume - 1] = 1; + struct stat st; + unsigned long before = svc_fast; + for (int i = 0; i < 20; i++) + assert(raw(SYS_newfstatat, AT_FDCWD, (long)"/usr/lib/libc.so.6", (long)&st, 0, 0, 0) == 0); + assert(svc_fast == before && !svc_busy); + return NULL; +} + +static void small_stack_checks(void) { + pthread_attr_t attr; + pthread_t thread; + size_t size = PTHREAD_STACK_MIN; + if (size < 128 * 1024) size = 128 * 1024; + assert(pthread_attr_init(&attr) == 0 && pthread_attr_setstacksize(&attr, size) == 0); + assert(pthread_create(&thread, &attr, small_stack_worker, (void *)(uintptr_t)size) == 0); + assert(pthread_join(thread, NULL) == 0 && pthread_attr_destroy(&attr) == 0); + puts("PASS: limited remaining thread stack delegates path calls to PRoot"); +} + struct signal_sender { pthread_t target; int done; }; static void *send_signals(void *arg) { struct signal_sender *sender = arg; @@ -237,7 +349,7 @@ static void *cancel_fifo(void *arg) { } static void signal_checks(void) { - stack_t old_stack, stack = {.ss_sp = malloc(128 * 1024), .ss_size = 128 * 1024}; + stack_t old_stack, stack = {.ss_sp = malloc(SIGSTKSZ), .ss_size = SIGSTKSZ}; assert(stack.ss_sp && sigaltstack(&stack, &old_stack) == 0); struct sigaction old, action = {.sa_handler = on_signal, .sa_flags = SA_ONSTACK}; sigemptyset(&action.sa_mask); @@ -292,7 +404,7 @@ int main(void) { size_t page = sysconf(_SC_PAGESIZE); uint32_t *relay = svc_relay_page((uintptr_t)&svc_check_site, page), branch; assert(relay && svc_emit(relay, &svc_check_site, &branch)); - __builtin___clear_cache((char *)relay, (char *)relay + 32); + __builtin___clear_cache((char *)relay, (char *)relay + 40); assert(mprotect(relay, page, PROT_READ | PROT_EXEC) == 0); int mem = open("/proc/self/mem", O_RDWR | O_CLOEXEC); assert(mem >= 0 && pwrite(mem, &branch, 4, (off_t)(uintptr_t)&svc_check_site) == 4); @@ -331,6 +443,8 @@ int main(void) { int status; assert(waitpid(pid, &status, 0) == pid && WIFEXITED(status) && WEXITSTATUS(status) == 0); signal_checks(); + interruption_checks(); + small_stack_checks(); puts("PASS: raw results, errno, x1-x30, q0-q31, NZCV, FPCR/FPSR, 4 threads, fork"); return 0; } diff --git a/tools/proot/bench/svc-fail.c b/tools/proot/bench/svc-fail.c new file mode 100644 index 0000000000..94430bad44 --- /dev/null +++ b/tools/proot/bench/svc-fail.c @@ -0,0 +1,38 @@ +#include "../fastpath/fastpath.c" +#include +#include +#include +#include + +int main(int argc, char **argv) { +#if defined(__aarch64__) + assert(argc == 2 && fp_ready() && !svc_sites); + int nr = !strcmp(argv[1], "mem") ? SYS_openat : !strcmp(argv[1], "write") ? SYS_pwrite64 : + !strcmp(argv[1], "rx") ? SYS_mprotect : SYS_mmap; + unsigned offset = nr == SYS_mprotect ? offsetof(struct seccomp_data, args[2]) : offsetof(struct seccomp_data, args[3]); + unsigned mask = nr == SYS_mprotect ? PROT_EXEC : nr == SYS_mmap ? MAP_FIXED_NOREPLACE : ~0U; + struct sock_filter code[] = { + BPF_STMT(BPF_LD | BPF_W | BPF_ABS, offsetof(struct seccomp_data, nr)), + BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, nr, 0, 3), + BPF_STMT(BPF_LD | BPF_W | BPF_ABS, offset), + BPF_JUMP(BPF_JMP | BPF_JSET | BPF_K, mask, 0, 1), + BPF_STMT(BPF_RET | BPF_K, SECCOMP_RET_ERRNO | EACCES), + BPF_STMT(BPF_RET | BPF_K, SECCOMP_RET_ALLOW), + }; + if (nr == SYS_openat || nr == SYS_pwrite64) code[2] = (struct sock_filter)BPF_STMT(BPF_LD | BPF_IMM, 1); + struct sock_fprog filter = {.len = sizeof code / sizeof *code, .filter = code}; + assert(setenv("PROOT_FP_PATCH_SVC", "1", 1) == 0); + if (sc(SYS_prctl, PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0) || + sc(SYS_prctl, PR_SET_SECCOMP, SECCOMP_MODE_FILTER, (long)&filter, 0, 0)) return 77; + svc_patch_init(); + assert(svc_sites == 0 && svc_pages == 0); + struct stat st; + long args[] = {AT_FDCWD, (long)"/etc/os-release", (long)&st, 0, 0, 0, 0, 0, SYS_newfstatat}; + assert(svc_original(args) == 0 && S_ISREG(st.st_mode)); + puts("PASS: denied patch installation leaves the original libc syscall usable"); + return 0; +#else + (void)argc; (void)argv; + return 77; +#endif +} diff --git a/tools/proot/bench/svc-race.c b/tools/proot/bench/svc-race.c new file mode 100644 index 0000000000..4198f4e36c --- /dev/null +++ b/tools/proot/bench/svc-race.c @@ -0,0 +1,49 @@ +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +int main(void) { + char work[] = "/tmp/proot-svc-race-XXXXXX"; + assert(mkdtemp(work)); + int cwd = open(".", O_RDONLY | O_DIRECTORY); + assert(cwd >= 0 && chdir(work) == 0); + for (int i = 0; i < 2; i++) { + int fd = open(i ? "b" : "a", O_CREAT | O_EXCL | O_WRONLY, 0600); + assert(fd >= 0 && ftruncate(fd, i + 1) == 0 && close(fd) == 0); + } + assert(symlink("a", "live") == 0); + void *libc = dlopen("libc.so.6", RTLD_NOW); + int (*stat)(int, const char *, struct stat *, int) = dlsym(libc, "fstatat"); + ssize_t (*link)(int, const char *, char *, size_t) = dlsym(libc, "readlinkat"); + assert(stat && link); + pid_t writer = fork(); + assert(writer >= 0); + if (!writer) { + for (int i = 0; i < 3000; i++) { + assert(symlink(i & 1 ? "b" : "a", "next") == 0); + assert(rename("next", "live") == 0); + } + _exit(0); + } + for (int i = 0; i < 3000; i++) { + struct stat st; + assert(stat(AT_FDCWD, "live", &st, 0) == 0 && S_ISREG(st.st_mode) && (st.st_size == 1 || st.st_size == 2)); + char text[8]; + assert(link(AT_FDCWD, "live", text, sizeof text) == 1 && (text[0] == 'a' || text[0] == 'b')); + } + int status; + assert(waitpid(writer, &status, 0) == writer && WIFEXITED(status) && WEXITSTATUS(status) == 0); + assert(unlink("live") == 0 && unlink("a") == 0 && unlink("b") == 0); + assert(dlclose(libc) == 0 && fchdir(cwd) == 0 && close(cwd) == 0 && rmdir(work) == 0); + puts("PASS: concurrent atomic symlink replacement, internal stat/readlink legal results"); + return 0; +} diff --git a/tools/proot/fastpath/svc-cache.h b/tools/proot/fastpath/svc-cache.h index 4a6f49b808..57c1be4bab 100644 --- a/tools/proot/fastpath/svc-cache.h +++ b/tools/proot/fastpath/svc-cache.h @@ -39,7 +39,7 @@ static int svc_cache_name(const struct dl_phdr_info *info, char *path, size_t si id[2 * j + 1] = hex[b & 15]; } id[2 * h.n_descsz] = 0; - int n = snprintf(path, size, "/tmp/.droiddeck-svc-v1-%d-%s", svc_android_api, id); + int n = snprintf(path, size, "/tmp/.droiddeck-svc-v2-%d-%s", svc_android_api, id); return n > 0 && (size_t)n < size; } at += name + desc; @@ -57,7 +57,7 @@ static int svc_cache_read(const struct dl_phdr_info *info, const char *path) { st.st_uid == (uid_t)sc(SYS_getuid, 0, 0, 0, 0, 0) && !(st.st_mode & 0077) && st.st_size == sizeof table && sc(SYS_read, fd, (long)&table, sizeof table, 0, 0) == sizeof table; sc(SYS_close, fd, 0, 0, 0, 0); - if (!valid || table.magic != 0x314356534444ULL || table.api != (unsigned)svc_android_api || + if (!valid || table.magic != 0x324356534444ULL || table.api != (unsigned)svc_android_api || !table.count || table.count > 256 || table.count > 256 - svc_planned) return 0; for (unsigned i = 0; i < table.count; i++) { uint64_t offset = table.offset[i]; @@ -75,7 +75,7 @@ static int svc_cache_read(const struct dl_phdr_info *info, const char *path) { static void svc_cache_write(const struct dl_phdr_info *info, const char *path, size_t first) { if (svc_planned == first || svc_planned == 256) return; - struct svc_cached_sites table = {.magic = 0x314356534444ULL, .api = svc_android_api, .count = svc_planned - first}; + struct svc_cached_sites table = {.magic = 0x324356534444ULL, .api = svc_android_api, .count = svc_planned - first}; for (unsigned i = 0; i < table.count; i++) table.offset[i] = (uintptr_t)svc_plan[first + i].site - info->dlpi_addr; char temporary[256]; diff --git a/tools/proot/fastpath/svc-dispatch.h b/tools/proot/fastpath/svc-dispatch.h index f8cdbbf76a..b8e87475fd 100644 --- a/tools/proot/fastpath/svc-dispatch.h +++ b/tools/proot/fastpath/svc-dispatch.h @@ -1,5 +1,4 @@ static int svc_android_api; -static __thread struct { long head, tid; } svc_robust __attribute__((tls_model("initial-exec"))); static int svc_host_android_api(void) { const char *value = getenv("PROOT_FP_ANDROID_API"); @@ -34,7 +33,7 @@ static int svc_supported(int nr) { case SYS_truncate: case SYS_statfs: case SYS_getxattr: case SYS_lgetxattr: case SYS_listxattr: case SYS_llistxattr: case SYS_inotify_add_watch: case SYS_getcwd: return 1; - case SYS_set_robust_list: case SYS_get_robust_list: case SYS_faccessat2: + case SYS_faccessat2: return svc_android_api > 0; case SYS_clone3: return svc_android_api > 0 && svc_android_api < 35; @@ -45,24 +44,13 @@ static int svc_supported(int nr) { static long svc_blocked(const long *r) { if (!svc_android_api) return FP_SLOW; if (r[8] == SYS_faccessat2 || (r[8] == SYS_clone3 && svc_android_api < 35)) return -ENOSYS; - if (r[8] == SYS_set_robust_list) { - svc_robust.head = r[0]; - svc_robust.tid = sc(SYS_gettid, 0, 0, 0, 0, 0); - return -ENOSYS; - } - if (r[8] == SYS_get_robust_list) { - long tid = sc(SYS_gettid, 0, 0, 0, 0, 0); - if ((r[0] && r[0] != tid) || svc_robust.tid != tid) return FP_SLOW; - if (!svc_robust.head) return -ENOSYS; - size_t size = 3 * sizeof(long); - if (svc_memory(&svc_robust.head, r[1], sizeof(long), 1) != sizeof(long) || - svc_memory(&size, r[2], sizeof size, 1) != sizeof size) return -EFAULT; - return 0; - } return FP_SLOW; } static long svc_path_call(const long *r) { +#ifdef FP_SVC_TEST_HOOK + FP_SVC_TEST_HOOK(r); +#endif char path[PATH_MAX], other[PATH_MAX]; struct timespec times[2]; const struct timespec *when = (const struct timespec *)r[2]; diff --git a/tools/proot/fastpath/svc-patch.h b/tools/proot/fastpath/svc-patch.h index dbd4488f09..300302f58e 100644 --- a/tools/proot/fastpath/svc-patch.h +++ b/tools/proot/fastpath/svc-patch.h @@ -47,13 +47,15 @@ static inline int svc_branch(uintptr_t from, uintptr_t to, uint32_t *out) { #include #include #include +#include -static __thread int svc_busy __attribute__((tls_model("initial-exec"))); +static __thread uintptr_t svc_busy __attribute__((tls_model("initial-exec"))); static unsigned svc_sites, svc_pages; static unsigned long svc_fast, svc_slow; static unsigned long long svc_scan_ns, svc_patch_ns; #include "svc-dispatch.h" +#include "svc-stack.h" static long svc_original(const long *r) { register long x0 __asm__("x0") = r[0]; @@ -71,10 +73,12 @@ __attribute__((used, visibility("hidden"))) long fp_svc_dispatch(const long *r) { int saved_errno = errno; long result = svc_blocked(r); - if (result == FP_SLOW && !svc_busy) { - svc_busy = 1; - result = svc_path_call(r); - svc_busy = 0; + uintptr_t frame = (uintptr_t)r; + if (result == FP_SLOW && (!svc_busy || frame >= svc_busy)) { + uintptr_t previous = svc_busy; + svc_busy = frame; + if (svc_stack_room(frame)) { result = svc_path_call(r); svc_busy = 0; } + else svc_busy = previous; } if (result == FP_SLOW) { __atomic_fetch_add(&svc_slow, 1, __ATOMIC_RELAXED); @@ -88,7 +92,8 @@ void fp_svc_entry(void) __attribute__((visibility("hidden"))); /* A replaced svc may have live values in every register, including NZCV and SIMD. */ __asm__( ".text\n.p2align 4\n.hidden fp_svc_entry\n.type fp_svc_entry,%function\nfp_svc_entry:\n" - "bti j\nsub sp, sp, #800\n" + ".cfi_startproc\n.cfi_def_cfa sp, 16\n.cfi_return_column 16\n" + "bti j\nsub sp, sp, #800\n.cfi_def_cfa_offset 816\n" "stp x0, x1, [sp, #0]\nstp x2, x3, [sp, #16]\n" "stp x4, x5, [sp, #32]\nstp x6, x7, [sp, #48]\n" "stp x8, x9, [sp, #64]\nstp x10, x11, [sp, #80]\n" @@ -97,6 +102,13 @@ __asm__( "stp x22, x23, [sp, #160]\nstp x24, x25, [sp, #176]\n" "stp x26, x27, [sp, #192]\nstp x28, x29, [sp, #208]\n" "stp x30, x17, [sp, #224]\nmrs x0, nzcv\nstr x0, [sp, #240]\n" + "ldr x0, [x17, #8]\nstr x0, [sp, #784]\n" + ".cfi_offset 16, -32\n.cfi_offset 19, -680\n.cfi_offset 20, -672\n" + ".cfi_offset 21, -664\n.cfi_offset 22, -656\n.cfi_offset 23, -648\n" + ".cfi_offset 24, -640\n.cfi_offset 25, -632\n.cfi_offset 26, -624\n" + ".cfi_offset 27, -616\n.cfi_offset 28, -608\n.cfi_offset 29, -600\n.cfi_offset 30, -592\n" + ".cfi_offset 72, -432\n.cfi_offset 73, -416\n.cfi_offset 74, -400\n.cfi_offset 75, -384\n" + ".cfi_offset 76, -368\n.cfi_offset 77, -352\n.cfi_offset 78, -336\n.cfi_offset 79, -320\n" "mrs x0, fpcr\nstr x0, [sp, #768]\nmrs x0, fpsr\nstr x0, [sp, #776]\n" "stp q0, q1, [sp, #256]\nstp q2, q3, [sp, #288]\n" "stp q4, q5, [sp, #320]\nstp q6, q7, [sp, #352]\n" @@ -125,7 +137,7 @@ __asm__( "ldp x22, x23, [sp, #160]\nldp x24, x25, [sp, #176]\n" "ldp x26, x27, [sp, #192]\nldp x28, x29, [sp, #208]\n" "ldr x30, [sp, #224]\nldr x16, [sp, #232]\n" - "add sp, sp, #800\nbr x16\n.size fp_svc_entry, .-fp_svc_entry\n" + "add sp, sp, #800\n.cfi_def_cfa_offset 16\nbr x16\n.cfi_endproc\n.size fp_svc_entry, .-fp_svc_entry\n" ); static int svc_emit(uint32_t *code, uint32_t *site, uint32_t *branch) { @@ -133,13 +145,15 @@ static int svc_emit(uint32_t *code, uint32_t *site, uint32_t *branch) { if (!svc_branch((uintptr_t)site, (uintptr_t)code, branch) || !svc_branch((uintptr_t)(code + 5), (uintptr_t)(site + 1), &back)) return 0; code[0] = 0xa9bf47f0U; /* stp x16, x17, [sp, #-16]! */ - code[1] = 0x580000b0U; /* ldr x16, literal at +24 */ + code[1] = 0x580000f0U; /* ldr x16, literal at +32 */ code[2] = 0x10000051U; /* adr x17, +8 */ code[3] = 0xd61f0200U; /* br x16 */ code[4] = 0xa8c147f0U; /* ldp x16, x17, [sp], #16 */ code[5] = back; uintptr_t entry = (uintptr_t)fp_svc_entry; - memcpy(code + 6, &entry, sizeof entry); + uintptr_t resume = (uintptr_t)(site + 1); + memcpy(code + 6, &resume, sizeof resume); + memcpy(code + 8, &entry, sizeof entry); return 1; } @@ -190,6 +204,8 @@ static uint32_t *svc_relay_page(uintptr_t site, size_t page) { return NULL; } +#include "svc-publish.h" + static int svc_single_thread(void) { struct entry { uint64_t ino; int64_t off; unsigned short len; unsigned char type; char name[]; }; char buf[4096] __attribute__((aligned(8))); @@ -245,32 +261,7 @@ __attribute__((constructor)) static void svc_patch_init(void) { dl_iterate_phdr(svc_object, NULL); svc_scan_ns = svc_clock() - start; start = svc_clock(); - for (size_t first = 0; first < svc_planned;) { - uint32_t *relay = svc_relay_page((uintptr_t)svc_plan[first].site, page); - if (!relay) { first++; continue; } - size_t end = first; - for (; end < svc_planned && (end - first + 1) * 32 <= page; end++) { - uint32_t *code = relay + (end - first) * 8; - if (!svc_emit(code, svc_plan[end].site, &svc_plan[end].branch)) break; - } - __builtin___clear_cache((char *)relay, (char *)relay + page); - if (end == first || mprotect(relay, page, PROT_READ | PROT_EXEC)) { - munmap(relay, page); - first++; - continue; - } - unsigned patched = 0; - for (size_t i = first; i < end; i++) { - struct svc_site *s = &svc_plan[i]; - long written = sc(SYS_pwrite64, mem, (long)&s->branch, sizeof s->branch, (long)s->site, 0); - if (written != sizeof s->branch) continue; - __builtin___clear_cache((char *)s->site, (char *)(s->site + 1)); - patched++; - } - if (patched) { svc_sites += patched; svc_pages++; } - else munmap(relay, page); - first = end; - } + svc_publish(mem, page); sc(SYS_close, mem, 0, 0, 0, 0); svc_patch_ns = svc_clock() - start; if (getenv("PROOT_FP_STATS")) atexit(svc_stats); diff --git a/tools/proot/fastpath/svc-publish.h b/tools/proot/fastpath/svc-publish.h new file mode 100644 index 0000000000..002e06c518 --- /dev/null +++ b/tools/proot/fastpath/svc-publish.h @@ -0,0 +1,28 @@ +static void svc_publish(int mem, size_t page) { + for (size_t first = 0; first < svc_planned;) { + uint32_t *relay = svc_relay_page((uintptr_t)svc_plan[first].site, page); + if (!relay) { first++; continue; } + size_t end = first; + for (; end < svc_planned && (end - first + 1) * 40 <= page; end++) { + uint32_t *code = relay + (end - first) * 10; + if (!svc_emit(code, svc_plan[end].site, &svc_plan[end].branch)) break; + } + __builtin___clear_cache((char *)relay, (char *)relay + page); + if (end == first || mprotect(relay, page, PROT_READ | PROT_EXEC)) { + munmap(relay, page); + first++; + continue; + } + unsigned patched = 0; + for (size_t i = first; i < end; i++) { + struct svc_site *s = &svc_plan[i]; + long written = sc(SYS_pwrite64, mem, (long)&s->branch, sizeof s->branch, (long)s->site, 0); + if (written != sizeof s->branch) continue; + __builtin___clear_cache((char *)s->site, (char *)(s->site + 1)); + patched++; + } + if (patched) { svc_sites += patched; svc_pages++; } + else munmap(relay, page); + first = end; + } +} diff --git a/tools/proot/fastpath/svc-stack.h b/tools/proot/fastpath/svc-stack.h new file mode 100644 index 0000000000..d3a4ff528b --- /dev/null +++ b/tools/proot/fastpath/svc-stack.h @@ -0,0 +1,83 @@ +#include +#include + +static __thread uintptr_t svc_stack_low __attribute__((tls_model("initial-exec"))); +static __thread uintptr_t svc_stack_high __attribute__((tls_model("initial-exec"))); +static __thread uintptr_t svc_stack_gap __attribute__((tls_model("initial-exec"))); +static __thread int svc_stack_main __attribute__((tls_model("initial-exec"))); + +static int svc_hex(const char **text, uintptr_t *value) { + const char *p = *text; + uintptr_t n = 0; + int digits = 0; + for (;; p++) { + unsigned c = (unsigned char)*p; + unsigned d = c >= '0' && c <= '9' ? c - '0' : c >= 'a' && c <= 'f' ? c - 'a' + 10 : 16; + if (d == 16) break; + if (++digits > (int)(2 * sizeof n)) return 0; + n = (n << 4) | d; + } + *text = p; + *value = n; + return digits > 0; +} + +static int svc_stack_line(const char *line, uintptr_t sp, uintptr_t *low, uintptr_t *high) { + uintptr_t a, b; + if (!svc_hex(&line, &a) || *line++ != '-' || !svc_hex(&line, &b) || + *line++ != ' ' || line[0] != 'r' || line[1] != 'w' || a >= b || sp < a || sp >= b) return 0; + *low = a; + *high = b; + return 1; +} + +static __attribute__((noinline)) int svc_stack_find(uintptr_t sp) { + char chunk[512], line[96]; + size_t at = 0; + long fd = sc(SYS_openat, AT_FDCWD, (long)"/proc/self/maps", O_RDONLY | O_CLOEXEC, 0, 0); + if (fd < 0) return 0; + long n; + uintptr_t low = 0, high = 0, previous = 0; + int found = 0; + while (!found && (n = sc(SYS_read, fd, (long)chunk, sizeof chunk, 0, 0)) > 0) { + for (long i = 0; i < n; i++) { + if (chunk[i] == '\n') { + line[at] = 0; + found = svc_stack_line(line, sp, &low, &high); + if (!found) { + const char *p = line; + uintptr_t a, b; + if (svc_hex(&p, &a) && *p++ == '-' && svc_hex(&p, &b) && b <= sp) previous = b; + } + at = 0; + if (found) break; + } else if (at < sizeof line - 1) line[at++] = chunk[i]; + } + } + sc(SYS_close, fd, 0, 0, 0, 0); + if (found) { + svc_stack_low = 0; + svc_stack_high = high; + svc_stack_main = strstr(line, "[stack]") != NULL; + svc_stack_gap = previous < low && low - previous > 16 * 1024 * 1024 ? previous + 16 * 1024 * 1024 : low; + svc_stack_low = low; + } + return found; +} + +static int svc_stack_room(uintptr_t sp) { + stack_t alt; + if (sc(SYS_sigaltstack, 0, (long)&alt, 0, 0, 0) < 0 || (alt.ss_flags & SS_ONSTACK)) return 0; + if (!svc_stack_low || sp < svc_stack_low || sp >= svc_stack_high) + if (!svc_stack_find(sp)) return 0; + uintptr_t low = svc_stack_low; + if (svc_stack_main) { + struct rlimit limit; + if (sc(SYS_prlimit64, 0, RLIMIT_STACK, 0, (long)&limit, 0) < 0) return 0; + uintptr_t grow = limit.rlim_cur >= svc_stack_high ? 0 : svc_stack_high - limit.rlim_cur; + if (grow < svc_stack_gap) grow = svc_stack_gap; + if (grow < low) low = grow; + } + /* Path translation nests several PATH_MAX-sized frames. */ + return sp - low >= 96 * 1024; +} diff --git a/tools/tests/test_fastpath_svc.py b/tools/tests/test_fastpath_svc.py index a0d84e9242..41c1be275d 100644 --- a/tools/tests/test_fastpath_svc.py +++ b/tools/tests/test_fastpath_svc.py @@ -9,6 +9,120 @@ HEADER = Path(__file__).resolve().parents[1] / "proot/fastpath/svc-patch.h" class FastPathSvcEncodingTest(unittest.TestCase): + def compile_check(self, source, header): + if shutil.which("cc") is None: + self.skipTest("cc is not available") + with tempfile.TemporaryDirectory() as directory: + program = Path(directory) / "check.c" + program.write_text(source.replace('"HEADER"', '"' + HEADER.with_name(header).as_posix() + '"')) + output = Path(directory) / "check" + subprocess.run(["cc", "-O2", "-Wall", "-Wextra", "-Werror", str(program), "-o", str(output)], check=True) + subprocess.run([str(output)], check=True, timeout=10) + + def test_stack_bounds_and_alternate_stack_fallback(self): + self.compile_check(r''' +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include +static rlim_t stack_limit = 1024 * 1024; +static long sc(long n, long a, long b, long c, long d, long e) { + if (n == SYS_prlimit64) { + if (!stack_limit) return -1; + ((struct rlimit *)d)->rlim_cur = stack_limit; + return 0; + } + return syscall(n,a,b,c,d,e); +} +#include "HEADER" +int main(void) { + uintptr_t low, high; + assert(svc_stack_line("1000-9000 rw-p 00000000 00:00 0", 0x8000, &low, &high)); + assert(low == 0x1000 && high == 0x9000); + assert(!svc_stack_line("1000-9000 r--p", 0x8000, &low, &high)); + assert(!svc_stack_line("1000-9000 rw-p", 0x9000, &low, &high)); + assert(!svc_stack_line("9000-1000 rw-p", 0x8000, &low, &high)); + assert(!svc_stack_line("fffffffffffffffff-9000 rw-p", 0x8000, &low, &high)); + assert(!svc_stack_line("", 0x8000, &low, &high)); + char local; + assert(svc_stack_find((uintptr_t)&local)); + assert((uintptr_t)&local >= svc_stack_low && (uintptr_t)&local < svc_stack_high); + svc_stack_main = 0; svc_stack_low = 0x1000; svc_stack_high = 0x100000; + assert(!svc_stack_room(0x1000 + 96 * 1024 - 1)); + assert(svc_stack_room(0x1000 + 96 * 1024)); + svc_stack_main = 1; svc_stack_low = 0xf0000; svc_stack_gap = 0x1000; + assert(svc_stack_room(0xff000)); + stack_limit = 64 * 1024; + assert(!svc_stack_room(0xff000)); + stack_limit = 0; + assert(!svc_stack_room(0xff000)); + return 0; +} +''', "svc-stack.h") + + def test_patch_publication_failure_and_partial_success(self): + self.compile_check(r''' +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +static struct svc_site { uint32_t *site; uint32_t branch; } svc_plan[3]; +static size_t svc_planned = 3; +static unsigned svc_sites, svc_pages; +static int fail_alloc, fail_emit, fail_rx, fail_write, writes, live, rx; +static uint32_t *svc_relay_page(uintptr_t site, size_t page) { + (void)site; + if (fail_alloc) return NULL; + live++; rx = 0; + return calloc(1, page); +} +static int svc_emit(uint32_t *code, uint32_t *site, uint32_t *branch) { + (void)code; (void)site; + *branch = 0x14000001; + return !fail_emit; +} +static int mock_rx(void *p, size_t size, int prot) { + (void)p; (void)size; + assert(prot == (PROT_READ | PROT_EXEC)); + if (fail_rx) return -1; + rx = 1; + return 0; +} +static int mock_unmap(void *p, size_t size) { (void)size; free(p); live--; return 0; } +static long sc(long nr, long fd, long buf, long len, long at, long unused) { + (void)fd; (void)unused; + assert(nr == SYS_pwrite64 && len == 4 && rx); + if (++writes == fail_write) return -1; + memcpy((void *)at, (void *)buf, len); + return len; +} +#define mprotect mock_rx +#define munmap mock_unmap +#include "HEADER" +int main(void) { + static uint32_t sites[3]; + for (int mode = 0; mode < 5; mode++) { + for (int i=0; i<3; i++) { sites[i]=0xd4000001; svc_plan[i].site=&sites[i]; } + svc_sites=svc_pages=writes=live=rx=0; + fail_alloc=mode==0; fail_emit=mode==1; fail_rx=mode==2; fail_write=mode==3 ? 2 : 0; + svc_publish(7,4096); + if (mode<3) assert(!svc_sites && !svc_pages && !writes && !live); + else assert(svc_sites==(mode==3 ? 2U : 3U) && svc_pages==1 && live==1); + for (int i=0; i<3; i++) assert(sites[i]==(mode<3 || (mode==3 && i==1) ? 0xd4000001 : 0x14000001)); + } + return 0; +} +''', "svc-publish.h") + def test_instruction_decoding_and_branch_limits(self): if shutil.which("cc") is None: self.skipTest("cc is not available")