Add opt-in low32 GPU mapping adapter and integrity probes

tools/hybris/low32/README.md

@@ -0,0 +1,93 @@
+# Opt-in device-backed low mappings for Wine WoW64
+
+On the tested Pixel PowerVR stack, Vulkan maps host-visible GPU allocations
+above 4 GiB. Wine's 32-bit WoW64 Vulkan bridge rejects those pointers. This
+caused American Nightmare (202750) to stop at D3D9 vertex-buffer creation even
+after the separate PowerVR BC texture profile passed device initialization.
+
+`low32.c` is an experimental native Linux/glibc preload adapter. It is off by
+default. It applies only to anonymous-address, non-executable `MAP_SHARED`
+mappings of one explicitly selected character device. It retains the original
+device FD, offset, protection, flags, and actual GPU backing, and searches low
+address space with `MAP_FIXED_NOREPLACE`. It never truncates a pointer, copies
+into shadow storage, replaces existing mappings, or disables Wine's guard.
+Caller-specified/fixed mappings, regular files, private/anonymous memory and
+other devices retain their original behavior. Failure to find a suitable low
+hole fails with ENOMEM. Existing preload hooks are called through RTLD_NEXT;
+only resolver bootstrap uses the raw syscall. A compact atomic cursor avoids
+wasting large address slots on thousands of small driver control allocations.
+If the compact probe collides, a bounded address-range walk of
+/proc/self/maps finds page-sized holes and uses NOREPLACE again to handle races.
+The walk runs before coarse fallback, so small allocations do not fragment
+large available ranges. A game load had 2.3 GB free below 4 GiB but no hole as large as 4 MiB; the
+coarse-only version falsely failed even 4 KiB mappings. Mapping failures are
+reported even after the ordinary 256-event trace limit.
+
+## Build and verify
+
+Use a native 64-bit Linux/glibc compiler, not the Android NDK or macOS compiler:
+
+```sh
+tools/hybris/low32/build.sh /absolute/build-dir /path/to/Vulkan-Headers/include
+```
+
+The native library and unit test are always built. GPU probes are optional when
+headers are provided; the Windows x86 probe also needs i686-w64-mingw32-gcc.
+The unit test uses /dev/zero and proves CPU data integrity, collision handling,
+concurrency, 2000 simultaneous small mappings, invalid offsets, forwarding and
+opt-out. A fragmented-address-space regression reserves the low region, opens
+64 off-grid page holes, and verifies all 64 mappings and their data. The old
+coarse adapter fails that regression; the address-range fallback passes. An
+additional density regression fills 64 existing page holes while preserving a
+separate contiguous 16 MiB interval for a later complete data-readback map; the
+prior gap-after-coarse implementation fails and the gap-first strategy passes. These
+CPU tests do not by themselves prove GPU correctness.
+
+`gpu-copy.c` tests actual Vulkan transfer operations on vendor 0x1010. It maps
+two 32 MiB device allocations, runs four copies of full deterministic patterns,
+including offset/unmap/remap, uses flush/invalidate and barriers/fences, and
+compares every word. It can require each full CPU range to fit below 4 GiB:
+
+```sh
+./gpu-copy-arm64 32 --require-low32
+# Through a separate Proton ARM64/WoW64 test prefix:
+proton run gpu-copy-x86.exe 32 --require-low32 'Z:\path\probe-output.txt'
+```
+
+The Windows output file avoids Wine's detached console hiding probe results.
+Run with Khronos synchronization validation. Keep raw Steam/account logs local;
+export only controlled PROBE results, validation IDs and counts. The tested
+memory type was device-local, host-visible and coherent; noncoherent physical
+memory is not covered by that run.
+
+## Activation scope
+
+Set only the target game's overrides, after identifying the render device's
+`st_rdev` and validating the installed library's hash:
+
+```text
+LD_PRELOAD=/absolute/guest/path/libdroiddeck-low32.so
+DROIDDECK_LOW32_MODE=low32
+DROIDDECK_LOW32_RDEV=<verified decimal st_rdev>
+```
+
+Preserve any existing target-game preload rather than replacing it. The tested
+Pixel's /dev/dri/renderD128 was character device major226/minor128, st_rdev57984.
+Do not hardcode that value for other machines. BC emulation and the repaired
+DXVK presentation component remain separate prerequisites; this adapter does
+not implement them. No global Steam profile, managed graphics pack, Android
+security setting or unrelated game's overrides should change.
+
+Before activating, back up the exact game-environment bytes and only the target
+prefix's graphics DLLs, preserving symlinks. Rollback must check for concurrent
+edits. Keep original backups in the app; do not copy account files or an entire
+Steam prefix to a report. Removing these three overrides disables the adapter.
+
+## Compatibility limits
+
+This initial binary was built on Ubuntu24.04 ARM64 and needs glibc2.38. The
+tested guest supports it. This is not yet a universal runtime component; other
+glibc versions, page sizes, drivers, GPU families, native32-bit processes and
+general game stability require their own tests. Bounded native and Windows GPU
+copies prove data integrity for those workloads, not unrestricted gameplay.
+Never enable globally based on successful testing of one game.

tools/hybris/low32/build.sh

@@ -0,0 +1,18 @@
+#!/bin/sh
+set -eu
+# Run on native 64-bit Linux/glibc. This is not an Android/Bionic library.
+base=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
+output=${1:?usage: build.sh /absolute/build-directory [Vulkan-header-directory]}
+mkdir -p "$output"
+cc=${CC:-cc}
+"$cc" -std=gnu11 -Wall -Wextra -Werror -O2 -fPIC -shared -s \
+  "$base/low32.c" -ldl -o "$output/libdroiddeck-low32.so"
+"$cc" -std=gnu11 -Wall -Wextra -Werror -O2 "$base/test-low32.c" -pthread -o "$output/test-low32"
+LD_PRELOAD="$output/libdroiddeck-low32.so" "$output/test-low32"
+if [ "$#" -ge 2 ]; then
+  "$cc" -std=gnu11 -Wall -Wextra -Werror -O2 -s -I"$2" "$base/gpu-copy.c" -ldl -o "$output/gpu-copy-arm64"
+  if command -v i686-w64-mingw32-gcc >/dev/null 2>&1; then
+    i686-w64-mingw32-gcc -std=gnu11 -Wall -Wextra -Werror -O2 -s -I"$2" \
+      "$base/gpu-copy.c" -o "$output/gpu-copy-x86.exe"
+  fi
+fi

tools/hybris/low32/gpu-copy.c

@@ -0,0 +1,166 @@
+/* Same standalone GPU-copy probe built as Linux AArch64 and Windows x86.
+ * Tests actual Vulkan buffers, complete data comparison, noncoherent cache
+ * operations, remapping with an offset and queue completion. No Steam API.
+ */
+#define VK_NO_PROTOTYPES
+#ifdef _WIN32
+#include <windows.h>
+#else
+#include <dlfcn.h>
+#endif
+#include <inttypes.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <vulkan/vulkan.h>
+
+#define IFNS(X) X(DestroyInstance) X(EnumeratePhysicalDevices) X(GetPhysicalDeviceProperties) \
+ X(GetPhysicalDeviceMemoryProperties) X(GetPhysicalDeviceQueueFamilyProperties) X(CreateDevice) X(GetDeviceProcAddr)
+#define DFNS(X) X(DestroyDevice) X(GetDeviceQueue) X(CreateBuffer) X(DestroyBuffer) \
+ X(GetBufferMemoryRequirements) X(AllocateMemory) X(FreeMemory) X(BindBufferMemory) \
+ X(MapMemory) X(UnmapMemory) X(FlushMappedMemoryRanges) X(InvalidateMappedMemoryRanges) \
+ X(CreateCommandPool) X(DestroyCommandPool) X(AllocateCommandBuffers) X(ResetCommandBuffer) \
+ X(BeginCommandBuffer) X(EndCommandBuffer) X(CmdCopyBuffer) X(CmdPipelineBarrier) \
+ X(CreateFence) X(DestroyFence) X(ResetFences) X(QueueSubmit) X(WaitForFences)
+#define DECL(n) static PFN_vk##n p##n;
+IFNS(DECL)
+DFNS(DECL)
+#undef DECL
+#define CHECK(x) do { VkResult rc=(x); if(rc!=VK_SUCCESS){printf("PROBE_FAIL stage=%s vk_result=%d\n",#x,rc);goto cleanup;} }while(0)
+#define REQUIRE(x,msg) do {if(!(x)){puts("PROBE_FAIL " msg);goto cleanup;}}while(0)
+
+int main(int argc,char **argv)
+{
+#ifdef _WIN32
+    /* Wine/Proton can detach the Windows console from the Linux pipe. */
+    if(argc>3 && !freopen(argv[3],"w",stdout)) return 4;
+#endif
+    setvbuf(stdout,NULL,_IONBF,0);
+    puts("PROBE_STAGE main");
+    unsigned mib=argc>1?(unsigned)strtoul(argv[1],NULL,10):32;
+    int require_low=argc>2&&!strcmp(argv[2],"--require-low32");
+    if(mib<1||mib>256)return 2;
+    const VkDeviceSize bytes=(VkDeviceSize)mib*1024*1024;
+    VkInstance instance=VK_NULL_HANDLE;VkDevice device=VK_NULL_HANDLE;
+    VkBuffer buffers[2]={0};VkDeviceMemory memories[2]={0};
+    VkDeviceSize allocated[2]={0};void *mapped[2]={0};
+    VkCommandPool pool=VK_NULL_HANDLE;VkFence fence=VK_NULL_HANDLE;
+    int success=0,pending=0;uint64_t mismatches=0;unsigned iterations=0;
+    int low_all=1;uint32_t memory_flags[2]={0};
+    PFN_vkGetInstanceProcAddr gipa;
+#ifdef _WIN32
+    HMODULE loader=LoadLibraryA("vulkan-1.dll");
+    REQUIRE(loader,"load_vulkan_failed");
+    gipa=(PFN_vkGetInstanceProcAddr)(void *)GetProcAddress(loader,"vkGetInstanceProcAddr");
+#else
+    void *loader=dlopen("libvulkan.so.1",RTLD_NOW|RTLD_LOCAL);
+    REQUIRE(loader,"load_vulkan_failed");
+    gipa=(PFN_vkGetInstanceProcAddr)dlsym(loader,"vkGetInstanceProcAddr");
+#endif
+    REQUIRE(gipa,"missing_gipa");
+    puts("PROBE_STAGE vulkan_loader");
+    PFN_vkCreateInstance create=(PFN_vkCreateInstance)gipa(NULL,"vkCreateInstance");
+    REQUIRE(create,"missing_create_instance");
+    VkApplicationInfo app={.sType=VK_STRUCTURE_TYPE_APPLICATION_INFO,
+      .pApplicationName="DroidDeck low32 buffer integrity probe",.apiVersion=VK_API_VERSION_1_1};
+    VkInstanceCreateInfo ici={.sType=VK_STRUCTURE_TYPE_INSTANCE_CREATE_INFO,.pApplicationInfo=&app};
+    CHECK(create(&ici,NULL,&instance));
+    puts("PROBE_STAGE instance_created");
+#define LOADI(n) p##n=(PFN_vk##n)gipa(instance,"vk" #n);REQUIRE(p##n,"missing_" #n);
+    IFNS(LOADI)
+#undef LOADI
+    uint32_t count=0;CHECK(pEnumeratePhysicalDevices(instance,&count,NULL));
+    puts("PROBE_STAGE physical_device_enumeration");
+    REQUIRE(count&&count<=16,"physical_device_count");
+    VkPhysicalDevice physicals[16],physical=VK_NULL_HANDLE;
+    CHECK(pEnumeratePhysicalDevices(instance,&count,physicals));
+    VkPhysicalDeviceProperties properties;
+    for(uint32_t i=0;i<count;i++){
+      pGetPhysicalDeviceProperties(physicals[i],&properties);
+      if(properties.vendorID==0x1010&&properties.deviceType!=VK_PHYSICAL_DEVICE_TYPE_CPU){physical=physicals[i];break;}
+    }
+    REQUIRE(physical,"no_real_powervr");
+    printf("PROBE_DEVICE vendor=%x device=%x api=%x pointer_bits=%zu\n",
+           properties.vendorID,properties.deviceID,properties.apiVersion,sizeof(void *)*8);
+    uint32_t qcount=0;pGetPhysicalDeviceQueueFamilyProperties(physical,&qcount,NULL);
+    REQUIRE(qcount&&qcount<=64,"queue_count");
+    VkQueueFamilyProperties qp[64];pGetPhysicalDeviceQueueFamilyProperties(physical,&qcount,qp);
+    uint32_t family=UINT32_MAX;
+    for(uint32_t i=0;i<qcount;i++)if(qp[i].queueCount&&(qp[i].queueFlags&VK_QUEUE_GRAPHICS_BIT)){family=i;break;}
+    REQUIRE(family!=UINT32_MAX,"no_graphics_queue");float priority=1.0f;
+    VkDeviceQueueCreateInfo qci={.sType=VK_STRUCTURE_TYPE_DEVICE_QUEUE_CREATE_INFO,.queueFamilyIndex=family,.queueCount=1,.pQueuePriorities=&priority};
+    VkDeviceCreateInfo dci={.sType=VK_STRUCTURE_TYPE_DEVICE_CREATE_INFO,.queueCreateInfoCount=1,.pQueueCreateInfos=&qci};
+    CHECK(pCreateDevice(physical,&dci,NULL,&device));
+#define LOADD(n) p##n=(PFN_vk##n)pGetDeviceProcAddr(device,"vk" #n);REQUIRE(p##n,"missing_" #n);
+    DFNS(LOADD)
+#undef LOADD
+    VkPhysicalDeviceMemoryProperties mp;pGetPhysicalDeviceMemoryProperties(physical,&mp);
+    for(unsigned j=0;j<2;j++){
+      VkBufferCreateInfo bci={.sType=VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO,.size=bytes,
+        .usage=VK_BUFFER_USAGE_TRANSFER_SRC_BIT|VK_BUFFER_USAGE_TRANSFER_DST_BIT,.sharingMode=VK_SHARING_MODE_EXCLUSIVE};
+      CHECK(pCreateBuffer(device,&bci,NULL,&buffers[j]));VkMemoryRequirements mr;
+      pGetBufferMemoryRequirements(device,buffers[j],&mr);
+      uint32_t type=UINT32_MAX;
+      for(uint32_t i=0;i<mp.memoryTypeCount;i++)if((mr.memoryTypeBits&(1u<<i))&&(mp.memoryTypes[i].propertyFlags&VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT)){type=i;break;}
+      REQUIRE(type!=UINT32_MAX,"no_host_visible_memory");memory_flags[j]=mp.memoryTypes[type].propertyFlags;
+      VkMemoryAllocateInfo mai={.sType=VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO,.allocationSize=mr.size,.memoryTypeIndex=type};
+      allocated[j]=mr.size;CHECK(pAllocateMemory(device,&mai,NULL,&memories[j]));
+      CHECK(pBindBufferMemory(device,buffers[j],memories[j],0));
+    }
+    VkCommandPoolCreateInfo pci={.sType=VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO,
+      .flags=VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT,.queueFamilyIndex=family};
+    CHECK(pCreateCommandPool(device,&pci,NULL,&pool));VkCommandBuffer command;
+    VkCommandBufferAllocateInfo cai={.sType=VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO,
+      .commandPool=pool,.level=VK_COMMAND_BUFFER_LEVEL_PRIMARY,.commandBufferCount=1};
+    CHECK(pAllocateCommandBuffers(device,&cai,&command));
+    VkFenceCreateInfo fci={.sType=VK_STRUCTURE_TYPE_FENCE_CREATE_INFO};
+    CHECK(pCreateFence(device,&fci,NULL,&fence));VkQueue queue;pGetDeviceQueue(device,family,0,&queue);
+    for(unsigned k=0;k<4;k++){
+      /* Alternate full mapping and a 64KiB aligned offset, preserving every word. */
+      VkDeviceSize offset=(k&1)?65536:0,span=bytes-offset;
+      for(unsigned j=0;j<2;j++){
+        CHECK(pMapMemory(device,memories[j],offset,VK_WHOLE_SIZE,0,&mapped[j]));
+        int low=(uint64_t)(uintptr_t)mapped[j]+(allocated[j]-offset)<=UINT64_C(0x100000000);
+        low_all&=low;
+        printf("PROBE_MAP iteration=%u buffer=%u offset=%" PRIu64 " bytes=%" PRIu64 " ptr=%p low32=%d flags=%x\n",
+          k,j,(uint64_t)offset,(uint64_t)(allocated[j]-offset),mapped[j],low,memory_flags[j]);
+        REQUIRE(!require_low||low,"mapping_not_low32");
+      }
+      uint32_t *src=mapped[0],*dst=mapped[1];size_t words=(size_t)(span/4);
+      for(size_t i=0;i<words;i++){src[i]=((uint32_t)i*2654435761u)^(0xa5a50000u+k);dst[i]=0;}
+      VkMappedMemoryRange ranges[2];memset(ranges,0,sizeof(ranges));
+      for(unsigned j=0;j<2;j++)ranges[j]=(VkMappedMemoryRange){.sType=VK_STRUCTURE_TYPE_MAPPED_MEMORY_RANGE,.memory=memories[j],.offset=offset,.size=VK_WHOLE_SIZE};
+      CHECK(pFlushMappedMemoryRanges(device,2,ranges));
+      if(k){CHECK(pResetFences(device,1,&fence));CHECK(pResetCommandBuffer(command,0));}
+      VkCommandBufferBeginInfo cbi={.sType=VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO,.flags=VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT};
+      CHECK(pBeginCommandBuffer(command,&cbi));
+      VkMemoryBarrier before={.sType=VK_STRUCTURE_TYPE_MEMORY_BARRIER,.srcAccessMask=VK_ACCESS_HOST_WRITE_BIT,.dstAccessMask=VK_ACCESS_TRANSFER_READ_BIT|VK_ACCESS_TRANSFER_WRITE_BIT};
+      pCmdPipelineBarrier(command,VK_PIPELINE_STAGE_HOST_BIT,VK_PIPELINE_STAGE_TRANSFER_BIT,0,1,&before,0,NULL,0,NULL);
+      VkBufferCopy copy={.srcOffset=offset,.dstOffset=offset,.size=span};pCmdCopyBuffer(command,buffers[0],buffers[1],1,&copy);
+      VkMemoryBarrier after={.sType=VK_STRUCTURE_TYPE_MEMORY_BARRIER,.srcAccessMask=VK_ACCESS_TRANSFER_WRITE_BIT,.dstAccessMask=VK_ACCESS_HOST_READ_BIT};
+      pCmdPipelineBarrier(command,VK_PIPELINE_STAGE_TRANSFER_BIT,VK_PIPELINE_STAGE_HOST_BIT,0,1,&after,0,NULL,0,NULL);
+      CHECK(pEndCommandBuffer(command));VkSubmitInfo si={.sType=VK_STRUCTURE_TYPE_SUBMIT_INFO,.commandBufferCount=1,.pCommandBuffers=&command};
+      CHECK(pQueueSubmit(queue,1,&si,fence));pending=1;
+      VkResult waited=pWaitForFences(device,1,&fence,VK_TRUE,UINT64_C(10000000000));
+      if(waited!=VK_SUCCESS){printf("PROBE_FAIL GPU_COMPLETION_UNCONFIRMED result=%d\n",waited);goto cleanup;}
+      pending=0;CHECK(pInvalidateMappedMemoryRanges(device,1,&ranges[1]));
+      for(size_t i=0;i<words;i++)if(dst[i]!=(((uint32_t)i*2654435761u)^(0xa5a50000u+k)))mismatches++;
+      for(unsigned j=0;j<2;j++){pUnmapMemory(device,memories[j]);mapped[j]=NULL;}
+      iterations++;REQUIRE(!mismatches,"data_mismatch");
+    }
+    success=1;
+cleanup:
+    printf("PROBE_RESULT success=%d mib=%u iterations=%u mismatches=%" PRIu64 " all_mappings_low32=%d\n",success,mib,iterations,mismatches,low_all);
+    if(pending)return 3; /* Do not explicitly destroy objects with unknown GPU completion. */
+    if(device){
+      if(pUnmapMemory)for(unsigned j=0;j<2;j++)if(mapped[j])pUnmapMemory(device,memories[j]);
+      if(fence&&pDestroyFence)pDestroyFence(device,fence,NULL);
+      if(pool&&pDestroyCommandPool)pDestroyCommandPool(device,pool,NULL);
+      for(unsigned j=0;j<2;j++)if(buffers[j]&&pDestroyBuffer)pDestroyBuffer(device,buffers[j],NULL);
+      for(unsigned j=0;j<2;j++)if(memories[j]&&pFreeMemory)pFreeMemory(device,memories[j],NULL);
+      if(pDestroyDevice)pDestroyDevice(device,NULL);
+    }
+    if(instance&&pDestroyInstance)pDestroyInstance(instance,NULL);
+    return success?0:1;
+}

tools/hybris/low32/low32.c

@@ -0,0 +1,233 @@
+/* Experimental per-process PowerVR device-backed low-address mmap adapter.
+ * No copied/shadow memory, changed device offset or MAP_FIXED replacement.
+ * Default is off. Enable only for the identified render-device rdev and the
+ * target x86/WoW64 game. Native 64-bit Linux/glibc preload, not an Android ABI DLL.
+ */
+#define _GNU_SOURCE
+#define _LARGEFILE64_SOURCE
+#include <dlfcn.h>
+#include <errno.h>
+#include <inttypes.h>
+#include <stdatomic.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/mman.h>
+#include <sys/stat.h>
+#include <sys/syscall.h>
+#include <unistd.h>
+
+#ifndef MAP_FIXED_NOREPLACE
+#define MAP_FIXED_NOREPLACE 0x100000
+#endif
+
+static _Atomic uint64_t cursor;
+static _Atomic unsigned logged;
+typedef void *(*map_function)(void *,size_t,int,int,int,off_t);
+static _Atomic(map_function) next_map;
+static __thread int resolving __attribute__((tls_model("initial-exec")));
+static __thread int active __attribute__((tls_model("initial-exec")));
+static void *native_map(void *addr, size_t length, int prot, int flags, int fd, off_t offset)
+{
+    map_function next=atomic_load(&next_map);
+    if (!next && !resolving) {
+        resolving=1;
+        next=(map_function)dlsym(RTLD_NEXT,"mmap");
+        atomic_store(&next_map,next);
+        resolving=0;
+    }
+    /* Bootstrap only while resolving; otherwise preserve existing preload hooks. */
+    if (next) return next(addr,length,prot,flags,fd,offset);
+    return (void *)syscall(SYS_mmap, addr, length, prot, flags, fd, offset);
+}
+
+static void trace_map(const char *kind, size_t n, int prot, int flags, int fd,
+                      off_t off, const struct stat *st, void *p, int error)
+{
+    int saved = errno;
+    unsigned count = atomic_fetch_add(&logged, 1);
+    if (count < 256 || error != 0) {
+        char text[320];
+        int nbytes = snprintf(text, sizeof(text),
+            "DD_LOW32 %s pid=%ld len=%zu prot=%x flags=%x fd=%d off=%jd mode=%o rdev=%ju ptr=%p errno=%d\n",
+            kind, (long)getpid(), n, prot, flags, fd, (intmax_t)off, st ? st->st_mode : 0,
+            st ? (uintmax_t)st->st_rdev : 0, p, error);
+        if (nbytes > 0) {
+            ssize_t written = write(STDERR_FILENO, text,
+                (size_t)nbytes < sizeof(text) ? (size_t)nbytes : sizeof(text)-1);
+            (void)written;
+        }
+    }
+    errno = saved;
+}
+
+static int target_device(int fd, struct stat *st)
+{
+    const char *value = getenv("DROIDDECK_LOW32_RDEV");
+    char *end = NULL;
+    if (!value || !*value || fd < 0) return 0;
+    int saved = errno;
+    errno = 0;
+    uintmax_t expected = strtoumax(value, &end, 10);
+    int valid = !errno && end && !*end && expected > 0 &&
+                fstat(fd, st) == 0 && S_ISCHR(st->st_mode) &&
+                (uintmax_t)st->st_rdev == expected;
+    errno = saved;
+    return valid;
+}
+
+static void *adapted_map(void *addr, size_t length, int prot, int flags, int fd, off_t offset)
+{
+    const char *mode = getenv("DROIDDECK_LOW32_MODE");
+    struct stat st = {0};
+    if (mode && !strcmp(mode, "trace")) {
+        void *p = native_map(addr, length, prot, flags, fd, offset);
+        if (fd >= 0 && (prot & PROT_WRITE) && !(prot & PROT_EXEC) &&
+            (flags & MAP_TYPE) == MAP_SHARED) {
+            int saved=errno;
+            (void)fstat(fd,&st);
+            errno=saved;
+            trace_map("trace", length, prot, flags, fd, offset, &st, p,
+                      p == MAP_FAILED ? errno : 0);
+        }
+        return p;
+    }
+    if (!mode || strcmp(mode, "low32") || addr || !length ||
+        (flags & (MAP_ANONYMOUS | MAP_FIXED | MAP_FIXED_NOREPLACE)) ||
+        (prot & PROT_EXEC) || (flags & MAP_TYPE) != MAP_SHARED ||
+        !target_device(fd, &st))
+        return native_map(addr, length, prot, flags, fd, offset);
+
+    const uintptr_t lower = 0x10000000u, upper = 0xffff0000u;
+    const uintptr_t step = 0x00400000u;
+    if (length > upper-lower) {
+        errno = ENOMEM;
+        trace_map("too_large", length, prot, flags, fd, offset, &st, MAP_FAILED, errno);
+        return MAP_FAILED;
+    }
+    long page_value=sysconf(_SC_PAGESIZE);
+    uintptr_t page=page_value>0?(uintptr_t)page_value:4096;
+    uintptr_t rounded=(length+page-1)&~(page-1);
+    uintptr_t span=upper-lower;
+    uintptr_t compact=(uintptr_t)(atomic_fetch_add(&cursor,rounded)%span);
+    if (rounded>span-compact) compact=0;
+    unsigned slots = (unsigned)((span-length)/step)+1;
+    unsigned start = (unsigned)(compact/step)%slots;
+    int original_errno = errno;
+    for (unsigned i = 0; i < 1; i++) {
+        /* Compact first: hundreds of small driver mappings must not waste a
+         * 4MiB address slot each. Coarse search is only for existing Wine holes. */
+        uintptr_t hint = lower + (i==0?compact:(uintptr_t)((start+i-1)%slots)*step);
+        void *p = native_map((void *)hint, length, prot,
+                             flags|MAP_FIXED_NOREPLACE, fd, offset);
+        if (p != MAP_FAILED) {
+            uintptr_t first = (uintptr_t)p;
+            if (first >= lower && first < upper && length <= upper-first) {
+                errno = original_errno;
+                trace_map("mapped", length, prot, flags, fd, offset, &st, p, 0);
+                return p;
+            }
+            /* Older kernels can ignore NOREPLACE; never return a high mapping. */
+            int saved = errno;
+            (void)syscall(SYS_munmap, p, length);
+            errno = saved;
+        } else if (errno != EEXIST && errno != ENOMEM && errno != EAGAIN) {
+            trace_map("rejected", length, prot, flags, fd, offset, &st, p, errno);
+            return p;
+        }
+    }
+    /* Pack small allocations into address holes before coarse probing can
+     * fragment large holes. Walk
+     * this process's current mappings, then use NOREPLACE to tolerate races.
+     * No pointer paths or contents are read; only address ranges are used. */
+    for (unsigned pass = 0; pass < 2; ++pass) {
+        FILE *maps = fopen("/proc/self/maps", "r");
+        if (!maps) break;
+        char line[512];
+        uintptr_t gap_start = lower;
+        unsigned lines = 0;
+        while (fgets(line, sizeof(line), maps) && ++lines < 65536) {
+            uintptr_t first, last;
+            if (sscanf(line, "%" SCNxPTR "-%" SCNxPTR, &first, &last) != 2) continue;
+            if (last <= gap_start) continue;
+            uintptr_t gap_end = first < upper ? first : upper;
+            if (gap_end > gap_start && rounded <= gap_end - gap_start) {
+                void *p = native_map((void *)gap_start, length, prot,
+                    flags | MAP_FIXED_NOREPLACE, fd, offset);
+                if (p != MAP_FAILED) {
+                    uintptr_t address = (uintptr_t)p;
+                    if (address >= lower && address < upper && length <= upper - address) {
+                        fclose(maps);
+                        errno = original_errno;
+                        trace_map("mapped_gap", length, prot, flags, fd, offset, &st, p, 0);
+                        return p;
+                    }
+                    (void)syscall(SYS_munmap, p, length);
+                } else if (errno != EEXIST && errno != ENOMEM && errno != EAGAIN) {
+                    int saved = errno;
+                    fclose(maps);
+                    errno = saved;
+                    trace_map("rejected", length, prot, flags, fd, offset, &st, MAP_FAILED, errno);
+                    return MAP_FAILED;
+                }
+            }
+            if (last > gap_start) gap_start = last;
+            if (gap_start >= upper) break;
+        }
+        if (gap_start < upper && rounded <= upper - gap_start) {
+            void *p = native_map((void *)gap_start, length, prot,
+                flags | MAP_FIXED_NOREPLACE, fd, offset);
+            if (p != MAP_FAILED) {
+                uintptr_t address = (uintptr_t)p;
+                if (address >= lower && address < upper && length <= upper - address) {
+                    fclose(maps);
+                    errno = original_errno;
+                    trace_map("mapped_gap", length, prot, flags, fd, offset, &st, p, 0);
+                    return p;
+                }
+                (void)syscall(SYS_munmap, p, length);
+            }
+        }
+        fclose(maps);
+    }
+    for (unsigned i = 1; i <= slots; i++) {
+        /* Last resort if address walking was unavailable or raced. */
+        uintptr_t hint = lower + (i==0?compact:(uintptr_t)((start+i-1)%slots)*step);
+        void *p = native_map((void *)hint, length, prot,
+                             flags|MAP_FIXED_NOREPLACE, fd, offset);
+        if (p != MAP_FAILED) {
+            uintptr_t first = (uintptr_t)p;
+            if (first >= lower && first < upper && length <= upper-first) {
+                errno = original_errno;
+                trace_map("mapped", length, prot, flags, fd, offset, &st, p, 0);
+                return p;
+            }
+            /* Older kernels can ignore NOREPLACE; never return a high mapping. */
+            int saved = errno;
+            (void)syscall(SYS_munmap, p, length);
+            errno = saved;
+        } else if (errno != EEXIST && errno != ENOMEM && errno != EAGAIN) {
+            trace_map("rejected", length, prot, flags, fd, offset, &st, p, errno);
+            return p;
+        }
+    }
+    errno = ENOMEM;
+    trace_map("no_low_hole", length, prot, flags, fd, offset, &st, MAP_FAILED, errno);
+    return MAP_FAILED;
+}
+
+void *mmap(void *addr, size_t length, int prot, int flags, int fd, off_t offset)
+{
+    if (active) return native_map(addr,length,prot,flags,fd,offset);
+    active=1;
+    void *p=adapted_map(addr,length,prot,flags,fd,offset);
+    active=0;
+    return p;
+}
+
+void *mmap64(void *addr, size_t length, int prot, int flags, int fd, off64_t offset)
+{
+    _Static_assert(sizeof(off_t) == sizeof(off64_t), "64-bit host required");
+    return mmap(addr, length, prot, flags, fd, (off_t)offset);
+}

tools/hybris/low32/test-low32.c

@@ -0,0 +1,119 @@
+#define _GNU_SOURCE
+#include <assert.h>
+#include <errno.h>
+#include <fcntl.h>
+#include <pthread.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/mman.h>
+#include <sys/stat.h>
+#include <unistd.h>
+
+#define N (32u*1024u*1024u)
+static int zero_fd;
+static void *thread_test(void *arg)
+{
+    unsigned seed = (unsigned)(uintptr_t)arg;
+    uint32_t *p = mmap(NULL,N,PROT_READ|PROT_WRITE,MAP_SHARED,zero_fd,0);
+    assert(p != MAP_FAILED && (uintptr_t)p+N <= 0x100000000ull);
+    for (size_t i=0;i<N/4;i++) p[i]=(uint32_t)i^seed;
+    for (size_t i=0;i<N/4;i++) assert(p[i]==((uint32_t)i^seed));
+    assert(!munmap(p,N));
+    return NULL;
+}
+int main(void)
+{
+    zero_fd=open("/dev/zero",O_RDWR|O_CLOEXEC); assert(zero_fd>=0);
+    struct stat st; assert(!fstat(zero_fd,&st)); char value[48];
+    snprintf(value,sizeof(value),"%ju",(uintmax_t)st.st_rdev);
+    assert(!setenv("DROIDDECK_LOW32_RDEV",value,1));
+    assert(!setenv("DROIDDECK_LOW32_MODE","low32",1));
+    /* Existing low-address memory must survive device mappings. */
+    uint32_t *guard=mmap((void *)0x10000000,128u*1024u*1024u,PROT_READ|PROT_WRITE,
+                         MAP_PRIVATE|MAP_ANONYMOUS|MAP_FIXED_NOREPLACE,-1,0);
+    assert(guard==(void *)0x10000000);guard[0]=0xdeadbeef;guard[32767]=0xcafef00d;
+    pthread_t threads[8];
+    for (uintptr_t i=0;i<8;i++) assert(!pthread_create(&threads[i],NULL,thread_test,(void *)i));
+    for (unsigned i=0;i<8;i++) assert(!pthread_join(threads[i],NULL));
+    assert(guard[0]==0xdeadbeef && guard[32767]==0xcafef00d);
+    puts("PASS concurrent 32MiB mappings, full pattern readback and occupied-memory preservation");
+    assert(!munmap(guard,128u*1024u*1024u));
+    /* Caller-specified address and MAP_FIXED semantics are preserved. */
+    void *explicit=mmap((void *)0x30000000,4096,PROT_READ|PROT_WRITE,
+                         MAP_SHARED|MAP_FIXED_NOREPLACE,zero_fd,0);
+    assert(explicit==(void *)0x30000000); assert(!munmap(explicit,4096));
+    int mem=memfd_create("low32-nontarget",0);assert(mem>=0);assert(!ftruncate(mem,N));
+    uint32_t *q=mmap((void *)0x500000000ull,N,PROT_READ|PROT_WRITE,MAP_SHARED,mem,0);
+    assert(q==(void *)0x500000000ull);q[0]=123;assert(q[0]==123);assert(!munmap(q,N));close(mem);
+    puts("PASS unrelated regular-file mapping and explicit-address forwarding");
+    errno=0;
+    void *bad=mmap(NULL,4096,PROT_READ|PROT_WRITE,MAP_SHARED,zero_fd,1);
+    assert(bad==MAP_FAILED && errno==EINVAL);
+    puts("PASS invalid device offset retains EINVAL");
+    /* Real game initialization creates many small control/buffer mappings. */
+    void *small[2000];
+    for(unsigned i=0;i<2000;i++) {
+      small[i]=mmap(NULL,4096,PROT_READ|PROT_WRITE,MAP_SHARED,zero_fd,0);
+      assert(small[i]!=MAP_FAILED && (uintptr_t)small[i]+4096<=0x100000000ull);
+      *(uint32_t *)small[i]=i;
+    }
+    for(unsigned i=0;i<2000;i++){
+      assert(*(uint32_t *)small[i]==i);assert(!munmap(small[i],4096));
+    }
+    puts("PASS 2000 simultaneous small mappings without address-slot exhaustion");
+    /* A fragmented Wine address space has usable holes away from every
+     * 4MiB probe location. The mapping walk must find them without clobbering. */
+    const uintptr_t lo=0x10000000u, hi=0xffff0000u;
+    void *reservation=mmap((void *)lo,hi-lo,PROT_NONE,
+        MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE|MAP_FIXED_NOREPLACE,-1,0);
+    assert(reservation==(void *)lo);
+    void *holes[64], *fragmented[64];
+    for (unsigned i=0;i<64;i++) {
+      holes[i]=(void *)(lo+0x12345000u+(uintptr_t)i*0x10000u);
+      assert(!munmap(holes[i],4096));
+    }
+    for (unsigned i=0;i<64;i++) {
+      fragmented[i]=mmap(NULL,4096,PROT_READ|PROT_WRITE,MAP_SHARED,zero_fd,0);
+      assert(fragmented[i]!=MAP_FAILED);
+      int found=0;for(unsigned j=0;j<64;j++)found|=fragmented[i]==holes[j];assert(found);
+      *(uint32_t *)fragmented[i]=i+77;
+    }
+    for (unsigned i=0;i<64;i++) assert(*(uint32_t *)fragmented[i]==i+77);
+    for (unsigned i=0;i<64;i++) assert(!munmap(fragmented[i],4096));
+    assert(!munmap(reservation,hi-lo));
+    puts("PASS fragmented low32 mappings find 64 page holes beyond coarse probes");
+    /* Small allocations must consume existing page holes rather than punch
+     * coarse-grid holes into the only large interval needed by a later map. */
+    reservation=mmap((void *)lo,hi-lo,PROT_NONE,
+        MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE|MAP_FIXED_NOREPLACE,-1,0);
+    assert(reservation==(void *)lo);
+    for (unsigned i=0;i<64;i++) assert(!munmap(holes[i],4096));
+    const uintptr_t large_address=0x70000000u;
+    const size_t large_size=16u*1024u*1024u;
+    assert(!munmap((void *)large_address,large_size));
+    for (unsigned i=0;i<64;i++) {
+      fragmented[i]=mmap(NULL,4096,PROT_READ|PROT_WRITE,MAP_SHARED,zero_fd,0);
+      assert(fragmented[i]!=MAP_FAILED);
+      int found=0;for(unsigned j=0;j<64;j++)found|=fragmented[i]==holes[j];assert(found);
+      *(uint32_t *)fragmented[i]=i+99;
+    }
+    uint32_t *large=mmap(NULL,large_size,PROT_READ|PROT_WRITE,MAP_SHARED,zero_fd,0);
+    assert(large==(void *)large_address);
+    for(size_t i=0;i<large_size/4;i++)large[i]=(uint32_t)i^0xa5a5a5a5u;
+    for(size_t i=0;i<large_size/4;i++)assert(large[i]==((uint32_t)i^0xa5a5a5a5u));
+    for(unsigned i=0;i<64;i++){
+      assert(*(uint32_t *)fragmented[i]==i+99);assert(!munmap(fragmented[i],4096));
+    }
+    assert(!munmap(large,large_size));assert(!munmap(reservation,hi-lo));
+    puts("PASS page-sized mapping density preserves subsequent contiguous 16MiB allocation");
+    assert(!setenv("DROIDDECK_LOW32_RDEV","999999999",1));
+    q=mmap((void *)0x510000000ull,4096,PROT_READ|PROT_WRITE,MAP_SHARED,zero_fd,0);
+    assert(q==(void *)0x510000000ull);assert(!munmap(q,4096));
+    puts("PASS mismatched device remains untouched");
+    assert(!unsetenv("DROIDDECK_LOW32_MODE"));
+    q=mmap((void *)0x520000000ull,4096,PROT_READ|PROT_WRITE,MAP_SHARED,zero_fd,0);
+    assert(q==(void *)0x520000000ull);assert(!munmap(q,4096));
+    puts("PASS opt-in disabled by default");
+    close(zero_fd);return 0;
+}