mirror of
https://github.com/valentderah/totk-pc-recomp.git
synced 2026-11-19 20:25:57 +00:00
Run agent tasks in git worktrees
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
19c9b648d1
commit
2d714537e7
7 files changed
+385
-56
No files matched your search
@@ -1,55 +1,77 @@
|
||||
---
|
||||
name: private-build
|
||||
description: Fork a private build of the runtime to change and test it without touching runtime/ or other agents' builds, then hand back a diff. Use before editing runtime/ or tools/ code, building totk.exe, or cleaning up a finished build dir.
|
||||
description: Run a task in its own git worktree and build dir, commit on task/<tag>, hand back a branch. Use before editing runtime/ or tools/ code, building totk.exe, or cleaning up a finished task.
|
||||
---
|
||||
|
||||
# Private build: fork, build, hand back, clean up
|
||||
# Task worktree: start, build, commit, hand back, clean up
|
||||
|
||||
Several agents change `runtime/` at once and the coordinator merges their diffs, so every change starts in a fork:
|
||||
`build/v121-<tag>/`, where `<tag>` is a short name for your task.
|
||||
Several agents change `runtime/` and `tools/` at once and the coordinator merges their branches, so every task gets a
|
||||
git worktree `.worktrees/<tag>` on branch `task/<tag>` and a build dir `build/v121-<tag>`. `<tag>` is a short name for
|
||||
your task.
|
||||
|
||||
## 1. Fork
|
||||
## 1. Start
|
||||
|
||||
Copy `runtime/` to `build/v121-<tag>/runtime-src` once and edit only there. For a `tools/` change, copy the file into
|
||||
`build/v121-<tag>/` and run the copy. `generated/v121` and `recomp/` are shared: report a needed lifter change and leave
|
||||
it unmade.
|
||||
```
|
||||
.venv/Scripts/python tools/worktree.py new <tag>
|
||||
```
|
||||
|
||||
Work and build only inside `.worktrees/<tag>`. The worktree holds tracked files. `build/`, `generated/`, `refs/`,
|
||||
`toolchain/`, `.venv/`, `.venv-py314/` and `config.local.toml` are links to the main checkout, so everything written
|
||||
there is shared: `generated/` and `recomp/` belong to the coordinator. A needed lifter or generator change is reported
|
||||
in the hand-back, never made. `tools/` changes are made in the worktree like any other file.
|
||||
|
||||
## 2. Build
|
||||
|
||||
```
|
||||
export PATH=/c/msys64/ucrt64/bin:$PATH && .venv/Scripts/python tools/build_game.py --runtime build/v121-<tag>/runtime-src --out build/v121-<tag>
|
||||
export PATH=/c/msys64/ucrt64/bin:$PATH && .venv/Scripts/python tools/build_game.py --runtime runtime --out build/v121-<tag>
|
||||
```
|
||||
|
||||
The fork reuses `build/v121`'s generated objects: it hardlinks them into its `obj/` and compiles only `runtime/*.c`, so
|
||||
a first build takes about 10 s. It does so when `build/v121` is up to date and every header the generated code includes
|
||||
is byte-identical in your `runtime-src`. If you changed such a header (`cpu.h`, ...) or the main build is stale, it
|
||||
prints `compiling all generated C: <why>` and rebuilds everything (about 200 s on every core). `--full-build` forces
|
||||
that. A `.c` change is incremental either way. After every build check the exe's mtime: a failed build leaves the old
|
||||
The build reuses `build/v121`'s generated objects: it hardlinks them into `obj/` and compiles only `runtime/*.c`, so a
|
||||
first build takes about 10 s. It does so when `build/v121` is up to date and every header the generated code includes
|
||||
is byte-identical in your `runtime/`. If you changed such a header (`cpu.h`, ...) or the main build is stale, it prints
|
||||
`compiling all generated C: <why>` and rebuilds everything (about 200 s on every core). `--full-build` forces that. A
|
||||
`.c` change is incremental either way. After every build check the exe's mtime: a failed build leaves the old
|
||||
`totk.exe` in place, and you would test old code.
|
||||
|
||||
Run the game with the `run-game` skill and the probes from the build dir: `totk.exe --probe-kernel`, `--probe-vkexec`,
|
||||
`--probe-nvn-tex`, `--probe-shader` must pass before you hand back.
|
||||
Run the game with the `run-game` skill. `totk.exe --probe-kernel`, `--probe-vkexec`, `--probe-nvn-tex` and
|
||||
`--probe-shader` pass before you hand back.
|
||||
|
||||
## 3. Hand back
|
||||
## 3. Commit
|
||||
|
||||
A fix that costs frame time is not done: keep new work off per-draw and per-pixel paths unless the semantics need it,
|
||||
and say in the report what the change costs per draw, shader or frame. The coordinator measures `fight.txt` with
|
||||
`--quiet` before merging.
|
||||
Commit on `task/<tag>` as often as you like.
|
||||
|
||||
- Run `python tools/check_content.py --staged` before each commit.
|
||||
- Messages are plain English sentences, no type prefix, no `Co-Authored-By` trailer.
|
||||
- `main` stays untouched: you never commit to it, merge into it or push anything.
|
||||
|
||||
Keep notes in `build/v121-<tag>/NOTES.md` as you go. When you stop unfinished, write `build/v121-<tag>/HANDOFF.md`
|
||||
(under 80 lines): findings with paths, what is ruled out, the leading hypothesis, the exact next step, temporary debug
|
||||
edits (file:line). Both stay uncommitted.
|
||||
|
||||
## 4. Hand back
|
||||
|
||||
A fix that costs frame time is not done: keep new work off per-draw and per-pixel paths unless the semantics need it.
|
||||
The coordinator measures `fight.txt` with `--quiet` before merging.
|
||||
|
||||
A change on the render path also gets one look: a field `--shot` of your build next to one of `build/v121` at the same
|
||||
frame. Runs diverge, so small differences prove nothing, but gross regressions show (grass gone, colours off, black
|
||||
areas) and block the hand-back.
|
||||
|
||||
Write `build/v121-<tag>/<tag>.diff` with `diff -u --strip-trailing-cr runtime/<f> build/v121-<tag>/runtime-src/<f>` for
|
||||
every changed file, against `runtime/` as it is just before you write it (others merge meanwhile). Keep LF line
|
||||
endings. Read the diff's `-` lines: each must be a line you meant to change, not a merge you are reverting.
|
||||
Then:
|
||||
|
||||
Keep notes in `build/v121-<tag>/NOTES.md` as you go. When you stop unfinished, write `build/v121-<tag>/HANDOFF.md`
|
||||
(under 80 lines): findings with paths, what is ruled out, the leading hypothesis, the exact next step, temporary debug
|
||||
edits (file:line).
|
||||
1. Merge or rebase `main` into `task/<tag>` until your branch merges into `main` cleanly. Rebuild and rerun the four
|
||||
probes on the result.
|
||||
2. Report: the branch name, the commit list (`git log main..task/<tag> --oneline`), what each change costs per draw,
|
||||
shader or frame, the probe results, and any lifter or generator change you need.
|
||||
|
||||
## 4. Clean up
|
||||
## 5. Clean up (coordinator)
|
||||
|
||||
After the coordinator merges the diff, or drops the task, the fork goes: copy `*.md`, `*.diff` and any helper scripts
|
||||
to `build/archive/v121-<tag>/`, then delete `build/v121-<tag>/`. Forks of tasks waiting to be continued stay until
|
||||
then. `build/v121` (the main build) and `build/v121-pf` (shared shader and pipeline caches) are never deleted.
|
||||
After the merge, or when the task is dropped:
|
||||
|
||||
1. Copy `*.md` and any helper scripts from `build/v121-<tag>/` to `build/archive/v121-<tag>/`.
|
||||
2. `.venv/Scripts/python tools/worktree.py remove <tag> --delete-branch` (it unlinks the links before git removes the
|
||||
worktree, refuses a dirty worktree or an unmerged branch; `--force` overrides).
|
||||
3. Delete `build/v121-<tag>/`.
|
||||
|
||||
Tasks waiting to be continued keep their worktree and build dir. `build/v121` (the main build) and `build/v121-pf`
|
||||
(shared shader and pipeline caches) are never deleted.
|
||||
@@ -67,3 +67,6 @@ desktop.ini
|
||||
|
||||
# Milestone plans and task splits: working notes for local agents, not published
|
||||
/docs/milestones/
|
||||
|
||||
# Agent task worktrees (tools/worktree.py)
|
||||
/.worktrees/
|
||||
@@ -47,8 +47,8 @@ Setup and verify commands: [docs/toolchain-windows.md](docs/toolchain-windows.md
|
||||
- Shader census (`.venv-py314`): `tools\shader_census.py --version v121 [--translate]` extracts every Maxwell shader to `build/v121/shaders/` and runs it through `build\xlate\totk_xlate.exe` and `spirv-val`; failure groups go to `failures.txt`. `tools\xlate_smoke.py` checks the translator alone, without game files.
|
||||
- Shader preparation (`.venv-py314`, once per build dir and translator): `tools\prepare_shaders.py build\v121` translates the variants of the MacNativeGames shared shader list (read from `refs/` at run time) into the build's shader cache and builds their stage libraries into the driver cache (`totk.exe --prepare-shaders`, about 5 min, 1.3 GB); play then translates and compiles only variants the list lacks.
|
||||
- The plan by stage and the proposed fixes with their evidence: [docs/backlog.md](docs/backlog.md).
|
||||
- Running, measuring or frame-dumping the game, save backups, private runtime copies: [docs/agent-playbook.md](docs/agent-playbook.md).
|
||||
- Project skills (`.claude/skills/`): `private-build` before changing runtime/ or tools/ code and building totk.exe; `run-game` to run a build into gameplay from a save.
|
||||
- Running, measuring or frame-dumping the game, save backups, task worktrees and their builds: [docs/agent-playbook.md](docs/agent-playbook.md).
|
||||
- Project skills (`.claude/skills/`): `private-build` before changing runtime/ or tools/ code and building totk.exe (one git worktree per task); `run-game` to run a build into gameplay from a save.
|
||||
- hactool: `toolchain/hactool/hactool.exe`. From an MSYS2 shell pass `-k <keys>` explicitly; the shell's `HOME` hides the default key location.
|
||||
- Ghidra 12.1.2 with SwitchLoader: `toolchain/ghidra/ghidra_12.1.2_PUBLIC`. Game modules (NSO) open here, headless or GUI.
|
||||
- REA (`reverse-engineer-anything` skill + MCP server) analyses our own x86-64 builds. On Windows its Ghidra provider admits PE files only, so NSO analysis goes through Ghidra directly. Register the MCP server once per clone, at local scope:
|
||||
|
||||
@@ -3,13 +3,13 @@
|
||||
Read this before you run the game, take a frame dump or measure performance. This page holds the conventions; the
|
||||
`--help` of each tool holds its options.
|
||||
|
||||
## Work in a private runtime copy
|
||||
## Work in a task worktree
|
||||
|
||||
Fork, build, hand back and clean up with the `private-build` skill (`.claude/skills/private-build/SKILL.md`).
|
||||
Start, build, hand back and clean up a task with the `private-build` skill (`.claude/skills/private-build/SKILL.md`): a git worktree on `task/<tag>` and a build dir `build/v121-<tag>`.
|
||||
The pipeline cache, `pipeline-keys.bin`, `shader-cache`, `shader-programs.bin` and `shader-answers.bin` live next to
|
||||
each `totk.exe`, so a fork starts cold: seed it by copying `build/v121`'s five before timing anything. A fork seeded
|
||||
from an older build stalls on pipeline and shader builds, and its slow frames say nothing about the change. (Until 2026-10-22 every fork wrote into
|
||||
`build/v121`'s caches, and forks broke each other's runs.)
|
||||
each `totk.exe`, so a task build starts cold: seed it by copying `build/v121`'s five before timing anything. A build seeded
|
||||
from an older one stalls on pipeline and shader builds, and its slow frames say nothing about the change. (Until 2026-10-22 every task build wrote into
|
||||
`build/v121`'s caches, and task builds broke each other's runs.)
|
||||
|
||||
## Running
|
||||
|
||||
@@ -66,7 +66,7 @@ reports: alignments, subgroup size, depth formats, `maxPushDescriptors`.
|
||||
must pass. CPU-header changes also need `tests\diff\simd.py`, `lift.py` and `func.py` (AGENTS.md).
|
||||
`tools\build_game.py` hashes every object against `obj/objects.sha256` before it links and recompiles a mismatch (the
|
||||
faulty RAM flipped bits in stored objects once, `build/BOOTCRASH.md`). The line `objects verified: N, recompiled after
|
||||
a hash mismatch: M` should show M of 0; a nonzero M means rerun the probes. A fork checks the shared objects against
|
||||
a hash mismatch: M` should show M of 0; a nonzero M means rerun the probes. A task build checks the shared objects against
|
||||
the main build's manifest, so build `build/v121` first.
|
||||
|
||||
## Bug classes already fixed
|
||||
|
||||
@@ -0,0 +1,130 @@
|
||||
"""tools/worktree.py against a scratch repo whose "main checkout" holds dummy ignored data. No game files needed.
|
||||
|
||||
Proves that marker files in the linked directories and in the linked file survive:
|
||||
1. tools/worktree.py remove,
|
||||
2. git worktree remove --force with the junctions still in place,
|
||||
3. git worktree prune after the worktree directory is deleted with its junctions in place.
|
||||
Also checks that git status in a worktree shows no untracked or typechanged entry for the links.
|
||||
Run with the repo's Python: tests/worktree_test.py
|
||||
"""
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT / 'tools'))
|
||||
import worktree # noqa: E402
|
||||
|
||||
IGNORED = '\n'.join('/' + d + '/' for d in worktree.LINK_DIRS) + '\n/' + worktree.LINK_FILES[0] + '\n/.worktrees/\n'
|
||||
|
||||
|
||||
def git(cwd, *a):
|
||||
r = subprocess.run(['git', '-C', str(cwd), *a], capture_output=True, text=True)
|
||||
assert r.returncode == 0, f'git {a}: {r.stderr}'
|
||||
return r.stdout
|
||||
|
||||
|
||||
def tool(scratch, *a):
|
||||
r = subprocess.run([sys.executable, str(scratch / 'tools' / 'worktree.py'), *a], capture_output=True, text=True)
|
||||
assert r.returncode == 0, f'worktree.py {a}: {r.stdout}{r.stderr}'
|
||||
return r.stdout
|
||||
|
||||
|
||||
def markers(main):
|
||||
return [main / d / 'deep' / 'marker.txt' for d in worktree.LINK_DIRS if (main / d).is_dir()] + \
|
||||
[main / f for f in worktree.LINK_FILES]
|
||||
|
||||
|
||||
def check(main, what):
|
||||
for m in markers(main):
|
||||
assert m.is_file() and m.read_text() == 'keep', f'{what}: {m} lost'
|
||||
print(f'ok: {what}: {len(markers(main))} markers intact')
|
||||
|
||||
|
||||
def make_repo(base):
|
||||
main = base / 'main'
|
||||
(main / 'tools').mkdir(parents=True)
|
||||
shutil.copy(ROOT / 'tools' / 'worktree.py', main / 'tools' / 'worktree.py')
|
||||
(main / '.gitignore').write_text(IGNORED)
|
||||
(main / 'README.md').write_text('x\n')
|
||||
git(main, 'init', '-b', 'main')
|
||||
git(main, 'config', 'user.email', 't@t')
|
||||
git(main, 'config', 'user.name', 't')
|
||||
git(main, 'add', '.')
|
||||
git(main, 'commit', '-m', 'init')
|
||||
for d in worktree.LINK_DIRS:
|
||||
if d == 'game':
|
||||
continue # absent in a real checkout too: link only what exists
|
||||
(main / d / 'deep').mkdir(parents=True)
|
||||
(main / d / 'deep' / 'marker.txt').write_text('keep')
|
||||
for f in worktree.LINK_FILES:
|
||||
(main / f).write_text('keep')
|
||||
return main
|
||||
|
||||
|
||||
def main():
|
||||
base = Path(tempfile.mkdtemp(prefix='wt-test-')).resolve()
|
||||
try:
|
||||
main_dir = make_repo(base)
|
||||
check(main_dir, 'setup')
|
||||
|
||||
# tools/worktree.py new / status / remove
|
||||
wt = Path(tool(main_dir, 'new', 'a').strip())
|
||||
assert wt.resolve() == (main_dir / '.worktrees' / 'a').resolve(), wt
|
||||
for d in worktree.LINK_DIRS:
|
||||
if (main_dir / d).is_dir():
|
||||
assert os.path.isjunction(wt / d), d
|
||||
assert (wt / 'build' / 'deep' / 'marker.txt').read_text() == 'keep'
|
||||
assert os.path.samefile(wt / 'config.local.toml', main_dir / 'config.local.toml')
|
||||
status = git(wt, 'status', '--porcelain', '--ignored=no')
|
||||
assert status == '', f'links show in git status: {status!r}'
|
||||
assert git(wt, 'ls-files', '--others', '--exclude-standard') == ''
|
||||
print('ok: git status in the worktree is clean, links are ignored')
|
||||
assert '.worktrees/a' in tool(main_dir, 'list').replace(os.sep, '/')
|
||||
(wt / 'dirty.txt').write_text('x')
|
||||
r = subprocess.run([sys.executable, str(main_dir / 'tools' / 'worktree.py'), 'remove', 'a'],
|
||||
capture_output=True, text=True)
|
||||
assert r.returncode != 0 and (wt / 'build').is_dir(), 'remove must refuse a dirty worktree'
|
||||
check(main_dir, 'dirty remove refused')
|
||||
(wt / 'dirty.txt').unlink()
|
||||
tool(main_dir, 'remove', 'a')
|
||||
assert not wt.exists()
|
||||
check(main_dir, 'tools/worktree.py remove')
|
||||
|
||||
# unmerged branch: --delete-branch refuses, --force deletes
|
||||
wt = Path(tool(main_dir, 'new', 'b').strip())
|
||||
(wt / 'f.txt').write_text('x')
|
||||
git(wt, 'add', 'f.txt')
|
||||
git(wt, 'commit', '-m', 'work')
|
||||
r = subprocess.run([sys.executable, str(main_dir / 'tools' / 'worktree.py'), 'remove', 'b', '--delete-branch'],
|
||||
capture_output=True, text=True)
|
||||
assert r.returncode != 0 and wt.is_dir(), 'unmerged branch must block --delete-branch'
|
||||
git(main_dir, 'merge', '--ff-only', 'task/b')
|
||||
tool(main_dir, 'remove', 'b', '--delete-branch')
|
||||
assert 'task/b' not in git(main_dir, 'branch', '--list')
|
||||
check(main_dir, 'remove --delete-branch after merge')
|
||||
|
||||
# raw git: worktree remove --force with the links in place
|
||||
wt = Path(tool(main_dir, 'new', 'c').strip())
|
||||
git(main_dir, 'worktree', 'remove', '--force', str(wt))
|
||||
check(main_dir, 'raw git worktree remove --force')
|
||||
|
||||
# raw: delete the worktree directory with the junctions in place, then prune
|
||||
wt = Path(tool(main_dir, 'new', 'e').strip())
|
||||
shutil.rmtree(wt)
|
||||
git(main_dir, 'worktree', 'prune')
|
||||
check(main_dir, 'rmtree of a linked worktree, then git worktree prune')
|
||||
print('PASS')
|
||||
finally:
|
||||
# never follow a junction while cleaning up
|
||||
for w in (base / 'main' / '.worktrees').glob('*'):
|
||||
for d in worktree.LINK_DIRS:
|
||||
worktree.drop_link(w / d)
|
||||
shutil.rmtree(base, ignore_errors=True)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
+24
-18
@@ -52,6 +52,9 @@ import romfs_index
|
||||
from check_content import GENERATED_MARKER
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
# In a git worktree (tools/worktree.py) build/ and generated/ are junctions into the main checkout, and resolve() follows
|
||||
# them. MAIN_ROOT is where they land, so resolved paths still print and compare relative to a root.
|
||||
MAIN_ROOT = (ROOT / 'build').resolve().parent
|
||||
UCRT = Path(r'C:\msys64\ucrt64\bin')
|
||||
CLANG = UCRT / 'clang.exe'
|
||||
NINJA = UCRT / 'ninja.exe'
|
||||
@@ -74,22 +77,23 @@ JOB_MEM = 768 << 20
|
||||
MEM_BUDGET = 50 * GiB # total for all compile jobs
|
||||
|
||||
|
||||
def below_root(p):
|
||||
"""Resolved p relative to ROOT, else to MAIN_ROOT (the same relative path through the worktree's junctions), else
|
||||
absolute."""
|
||||
p = Path(p).resolve()
|
||||
for root in (ROOT, MAIN_ROOT):
|
||||
if p.is_relative_to(root):
|
||||
return p.relative_to(root)
|
||||
return p
|
||||
|
||||
|
||||
def ninja_path(p):
|
||||
"""A path for a build line: relative to the repo root where possible, forward slashes, Ninja-escaped."""
|
||||
p = Path(p)
|
||||
try:
|
||||
p = p.resolve().relative_to(ROOT)
|
||||
except ValueError:
|
||||
p = p.resolve()
|
||||
return p.as_posix().replace('$', '$$').replace(' ', '$ ').replace(':', '$:')
|
||||
return below_root(p).as_posix().replace('$', '$$').replace(' ', '$ ').replace(':', '$:')
|
||||
|
||||
|
||||
def flag_path(p):
|
||||
p = Path(p).resolve()
|
||||
try:
|
||||
return p.relative_to(ROOT).as_posix()
|
||||
except ValueError:
|
||||
return p.as_posix()
|
||||
return below_root(p).as_posix()
|
||||
|
||||
|
||||
def read_layout(gen):
|
||||
@@ -244,7 +248,7 @@ def main_objects(gen, main_out, rt_dir, mods):
|
||||
main_ninja = main_out / 'build.ninja'
|
||||
if not main_ninja.is_file():
|
||||
return None, f'{flag_path(main_ninja)} not found'
|
||||
if gen != ROOT / 'generated' / main_out.name:
|
||||
if gen != (ROOT / 'generated' / main_out.name).resolve():
|
||||
return None, 'generated C directory is not the main one'
|
||||
lines = main_ninja.read_text().splitlines()
|
||||
sources = dict(l[6:].split(': cc_gen ') for l in lines if l.startswith('build ') and ': cc_gen ' in l)
|
||||
@@ -261,14 +265,16 @@ def main_objects(gen, main_out, rt_dir, mods):
|
||||
if not theirs.is_file():
|
||||
return None, f'main object {flag_path(theirs)} is missing'
|
||||
pairs[o] = theirs
|
||||
# Ninja runs in the main checkout: the main build.ninja is relative to it, and the worktree's runtime/ files carry newer
|
||||
# mtimes that would make every object look stale. Header equality is checked byte for byte below.
|
||||
env = dict(os.environ, PATH=str(UCRT) + os.pathsep + os.environ.get('PATH', ''))
|
||||
targets = [ninja_path(t) for t in pairs.values()]
|
||||
for i in range(0, len(targets), 200):
|
||||
r = subprocess.run([str(NINJA), '-f', str(main_ninja), '-n', '-d', 'explain'] + targets[i:i + 200], cwd=ROOT,
|
||||
r = subprocess.run([str(NINJA), '-f', str(main_ninja), '-n', '-d', 'explain'] + targets[i:i + 200], cwd=MAIN_ROOT,
|
||||
env=env, capture_output=True, text=True)
|
||||
if r.returncode or 'no work to do' not in r.stdout:
|
||||
return None, f'main generated objects are stale ({flag_path(main_out)} needs a build)'
|
||||
r = subprocess.run([str(NINJA), '-f', str(main_ninja), '-t', 'deps'], cwd=ROOT, env=env, capture_output=True,
|
||||
r = subprocess.run([str(NINJA), '-f', str(main_ninja), '-t', 'deps'], cwd=MAIN_ROOT, env=env, capture_output=True,
|
||||
text=True)
|
||||
ours = {flag_path(t) for t in pairs.values()}
|
||||
headers, cur = set(), False
|
||||
@@ -278,9 +284,9 @@ def main_objects(gen, main_out, rt_dir, mods):
|
||||
headers.add(l.strip())
|
||||
else:
|
||||
cur = l.split(': #deps')[0] in ours
|
||||
main_rt = ROOT / 'runtime'
|
||||
main_rt = MAIN_ROOT / 'runtime'
|
||||
for h in sorted(headers):
|
||||
p = Path(h) if Path(h).is_absolute() else ROOT / h
|
||||
p = Path(h) if Path(h).is_absolute() else MAIN_ROOT / h
|
||||
try:
|
||||
rel = p.resolve().relative_to(main_rt)
|
||||
except ValueError:
|
||||
@@ -558,7 +564,7 @@ def main():
|
||||
global RUNTIME
|
||||
RUNTIME = (args.runtime or RUNTIME).resolve()
|
||||
skip = {s for s in args.skip_runtime.split(',') if s}
|
||||
main_out = ROOT / 'build' / args.version
|
||||
main_out = (ROOT / 'build' / args.version).resolve()
|
||||
pairs = None
|
||||
if args.full_build:
|
||||
why = '--full-build'
|
||||
@@ -601,7 +607,7 @@ def main():
|
||||
for dll in SDL3_DLLS: # dynamic SDL3 (zlib) and its libiconv (LGPL, unmodified DLL), found next to totk.exe
|
||||
shutil.copy2(dll, exe.parent / dll.name)
|
||||
sections, image = pe_sections(exe)
|
||||
print(f'{exe.relative_to(ROOT) if exe.is_relative_to(ROOT) else exe}: file {mib(exe.stat().st_size)}, '
|
||||
print(f'{below_root(exe)}: file {mib(exe.stat().st_size)}, '
|
||||
f'image {mib(image)}; ' + ', '.join(f'{k} {mib(v)}' for k, v in sections.items() if k.startswith('.') and v))
|
||||
if image > 3 * GiB // 2:
|
||||
print('build_game: warning: image over 1.5 GiB; RIP-relative code reaches only 2 GiB', file=sys.stderr)
|
||||
|
||||
@@ -0,0 +1,168 @@
|
||||
"""Git worktrees for agent tasks: .worktrees/<tag> on branch task/<tag>, linked to the main checkout's ignored data.
|
||||
|
||||
worktree.py new <tag> [--base REF] create the worktree and the links, print its path
|
||||
worktree.py list worktrees with branch, ahead/behind main, dirty or not
|
||||
worktree.py remove <tag> [--delete-branch] [--force]
|
||||
|
||||
A worktree holds tracked files only. build/, generated/, refs/, toolchain/, .venv/, .venv-py314/ and game/ of the main
|
||||
checkout appear in it as directory junctions, config.local.toml as a hard link. Nothing is copied.
|
||||
`remove` deletes the links first (os.rmdir / os.unlink never follow a junction), checks that none is left, and only
|
||||
then lets git delete the worktree. Stdlib only.
|
||||
"""
|
||||
import argparse
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
LINK_DIRS = ['build', 'generated', 'refs', 'toolchain', '.venv', '.venv-py314', 'game']
|
||||
LINK_FILES = ['config.local.toml']
|
||||
MAIN_BRANCH = 'main'
|
||||
TAG_RE = re.compile(r'^[A-Za-z0-9][A-Za-z0-9._-]*$')
|
||||
|
||||
|
||||
def git(cwd, *args, check=True):
|
||||
r = subprocess.run(['git', '-C', str(cwd), *args], capture_output=True, text=True)
|
||||
if check and r.returncode:
|
||||
sys.exit(f'git {" ".join(args)} failed: {(r.stderr or r.stdout).strip()}')
|
||||
return r
|
||||
|
||||
|
||||
def main_checkout(start):
|
||||
"""The main checkout: the parent of the shared .git directory, whichever worktree `start` is in."""
|
||||
common = git(start, 'rev-parse', '--path-format=absolute', '--git-common-dir').stdout.strip()
|
||||
return Path(common).resolve().parent
|
||||
|
||||
|
||||
def is_link(p):
|
||||
"""True for a junction or symlink."""
|
||||
return os.path.islink(p) or os.path.isjunction(p)
|
||||
|
||||
|
||||
def make_junction(target, link):
|
||||
if os.name == 'nt':
|
||||
import _winapi
|
||||
_winapi.CreateJunction(str(target), str(link))
|
||||
else:
|
||||
os.symlink(target, link, target_is_directory=True)
|
||||
|
||||
|
||||
def drop_link(p):
|
||||
"""Remove one link without touching what it points at. A real directory or file is left alone."""
|
||||
if os.path.isjunction(p) or os.path.islink(p):
|
||||
os.rmdir(p) if os.path.isjunction(p) else os.unlink(p)
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def link_data(main, wt):
|
||||
made = []
|
||||
for name in LINK_DIRS:
|
||||
src, dst = main / name, wt / name
|
||||
if not src.is_dir() or os.path.lexists(dst):
|
||||
continue
|
||||
make_junction(src, dst)
|
||||
made.append(name)
|
||||
for name in LINK_FILES:
|
||||
src, dst = main / name, wt / name
|
||||
if not src.is_file() or os.path.lexists(dst):
|
||||
continue
|
||||
os.link(src, dst)
|
||||
made.append(name)
|
||||
return made
|
||||
|
||||
|
||||
def unlink_data(wt):
|
||||
"""Remove the links of a worktree. A file link goes by os.unlink: it only drops one name of the data."""
|
||||
for name in LINK_DIRS:
|
||||
drop_link(wt / name)
|
||||
for name in LINK_FILES:
|
||||
p = wt / name
|
||||
if os.path.lexists(p) and not is_link(p):
|
||||
os.unlink(p) # a hard link looks like a plain file; its data survives under the main checkout's name
|
||||
left = [n for n in LINK_DIRS if is_link(wt / n)]
|
||||
if left:
|
||||
sys.exit(f'links still present in {wt}: {left}; not removing the worktree')
|
||||
|
||||
|
||||
def wt_path(main, tag):
|
||||
return main / '.worktrees' / tag
|
||||
|
||||
|
||||
def cmd_new(a):
|
||||
if not TAG_RE.match(a.tag):
|
||||
sys.exit(f'bad tag {a.tag!r}')
|
||||
main = main_checkout(Path(__file__).resolve().parent)
|
||||
wt = wt_path(main, a.tag)
|
||||
if wt.exists():
|
||||
sys.exit(f'{wt} exists')
|
||||
git(main, 'worktree', 'add', str(wt), '-b', f'task/{a.tag}', a.base)
|
||||
made = link_data(main, wt)
|
||||
print(wt)
|
||||
print('linked: ' + ', '.join(made), file=sys.stderr)
|
||||
|
||||
|
||||
def worktrees(main):
|
||||
out, cur = [], {}
|
||||
for line in git(main, 'worktree', 'list', '--porcelain').stdout.splitlines() + ['']:
|
||||
if not line:
|
||||
if cur:
|
||||
out.append(cur)
|
||||
cur = {}
|
||||
else:
|
||||
k, _, v = line.partition(' ')
|
||||
cur[k] = v
|
||||
return out
|
||||
|
||||
|
||||
def cmd_list(a):
|
||||
main = main_checkout(Path(__file__).resolve().parent)
|
||||
for w in worktrees(main):
|
||||
path = Path(w['worktree'])
|
||||
branch = w.get('branch', '(detached)').removeprefix('refs/heads/')
|
||||
r = git(path, 'rev-list', '--left-right', '--count', f'{MAIN_BRANCH}...HEAD', check=False)
|
||||
behind, ahead = r.stdout.split() if r.returncode == 0 else ('?', '?')
|
||||
dirty = bool(git(path, 'status', '--porcelain', check=False).stdout.strip())
|
||||
print(f'{path} {branch} ahead {ahead} behind {behind} {"dirty" if dirty else "clean"}')
|
||||
|
||||
|
||||
def cmd_remove(a):
|
||||
main = main_checkout(Path(__file__).resolve().parent)
|
||||
wt = wt_path(main, a.tag)
|
||||
branch = f'task/{a.tag}'
|
||||
if not wt.is_dir():
|
||||
sys.exit(f'{wt} not found')
|
||||
if git(wt, 'status', '--porcelain').stdout.strip() and not a.force:
|
||||
sys.exit(f'{wt} has uncommitted changes; commit them or pass --force')
|
||||
if a.delete_branch and not a.force:
|
||||
merged = git(main, 'branch', '--merged', MAIN_BRANCH, '--format=%(refname:short)').stdout.split()
|
||||
if branch not in merged:
|
||||
sys.exit(f'{branch} is not merged into {MAIN_BRANCH}; pass --force to delete it anyway')
|
||||
unlink_data(wt)
|
||||
git(main, 'worktree', 'remove', *(['--force'] if a.force else []), str(wt))
|
||||
print(f'removed {wt}')
|
||||
if a.delete_branch:
|
||||
git(main, 'branch', '-D', branch)
|
||||
print(f'deleted branch {branch}')
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
sub = ap.add_subparsers(dest='cmd', required=True)
|
||||
p = sub.add_parser('new')
|
||||
p.add_argument('tag')
|
||||
p.add_argument('--base', default=MAIN_BRANCH)
|
||||
p.set_defaults(fn=cmd_new)
|
||||
sub.add_parser('list').set_defaults(fn=cmd_list)
|
||||
p = sub.add_parser('remove')
|
||||
p.add_argument('tag')
|
||||
p.add_argument('--delete-branch', action='store_true')
|
||||
p.add_argument('--force', action='store_true')
|
||||
p.set_defaults(fn=cmd_remove)
|
||||
a = ap.parse_args()
|
||||
a.fn(a)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
Reference in new issue
Block a user