Merge task/prwave8: community pull requests #527 and the setup part of #489

#527: the setup takes NSZ and XCZ game files and unpacks them itself (tools/ncz.py), every NCA checked against its
hash. #489: the setup downloads NVIDIA's nvngx_dlss.dll for RTX GPUs (pinned tag, size and SHA-256); its runtime part
came in with prwave6. On top: a cut download warns instead of failing the setup, no warning without an RTX GPU, and
the file dialog keeps "All files".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Valiantsin DzerakhandClaude Opus 5.5 committed 2026-11-03 15:37:54 +03:00
commit e3111ac012
16 files changed
+762 -61

No files matched your search

+6 -1
View File
@@ -17,8 +17,11 @@ Report problems as issues, following [CONTRIBUTING.md](CONTRIBUTING.md).
- Native code, not emulation
- Game updates 1.1.0, 1.2.1 and 1.4.3; 1.2.1 and 1.4.3 saves carry over between them
- Takes NSP and XCI game files, and compressed NSZ and XCZ as they are: the setup unpacks them itself
- 60 fps by default; up to 144 fps or the monitor's refresh rate (experimental)
- Custom resolutions (1080p, 2k...), supersampling and AMD FSR 1 upscaling
- NVIDIA DLSS / DLAA on RTX GPUs: the setup downloads `nvngx_dlss.dll` from NVIDIA's DLSS SDK repository
(github.com/NVIDIA/DLSS, NVIDIA RTX SDK license) when it finds an RTX GPU; untick it in the setup to skip it
- HDR10 output with peak, game and HUD brightness settings
- 16:10 (Steam Deck, handhelds), 3:2 (Surface, Framework), 21:9 and 32:9 (ultrawide) game resolutions, with the camera and the interface in that shape
- Anisotropic texture filtering up to 16x, for sharp ground into the distance
@@ -44,7 +47,9 @@ Requirements: Windows 10 or later, an x86-64 CPU with SSE4.2 (AVX2/FMA machines
1. Download `TotK-Setup-<version>-windows.zip` from [Releases](../../releases/latest) and unpack it.
2. Start `TotK-Setup.exe`. If SmartScreen warns about an unsigned app, choose *More info*, then *Run anyway*.
3. Pick your game files, the language and an install folder, and press *Start*. The setup builds the game once.
3. Pick your game files, the language and an install folder, and press *Start*. The setup builds the game once. With
an NVIDIA RTX GPU it also downloads NVIDIA DLSS, unless you untick *Download NVIDIA DLSS*; turn DLSS on in the
menu (F1).
4. Play `totk.exe` from the install folder.
### Linux
+20 -5
View File
@@ -9,19 +9,21 @@ holds only the game; the installer and its tools stay outside it.
1. Downloads `TotK-Setup-<version>-windows.zip` from GitHub Releases (none is published yet), unpacks it anywhere and
starts `TotK-Setup.exe`.
2. Picks:
- the base game (XCI or NSP) and the 1.1.0, 1.2.1 or 1.4.3 update (NSP);
- the base game (XCI or NSP) and the 1.1.0, 1.2.1 or 1.4.3 update (NSP); an NSZ or XCZ (compressed by the nsz
tool) works as it is, the setup unpacks it itself;
- their `prod.keys`;
- optionally the firmware folder (`system/Contents/registered` of an unpacked firmware or an emulator's install),
needed for Chinese, Korean and Traditional Chinese text;
- the game language and the install folder.
3. Waits for the setup, one time. The window shows each step and its progress.
4. Plays `totk.exe` from the install folder. The installer and the source NSP or XCI can be deleted.
4. Plays `totk.exe` from the install folder. The installer and the source NSP or XCI (or NSZ, XCZ) can be deleted.
The install folder (`main.c` `portable_paths()` reads this layout; it stays portable):
```
<install folder>/
totk.exe, SDL3.dll, libopus-0.dll, totk_xlate.exe
nvngx_dlss.dll NVIDIA DLSS, downloaded by the setup's DLSS step when the player keeps it ticked
totk.ini language; the other keys as commented defaults (totk.ini.example)
exefs/, romfs/ extracted from the player's files
shared-fonts/ the Switch system fonts, with Chinese, Korean or Traditional Chinese text
@@ -44,7 +46,7 @@ loads the saves 1.4.3 wrote.
## Setup steps
1. **Check** before writing anything: the files open with the keys, the update is 1.1.0, 1.2.1 or 1.4.3 (by hash, as
1. **Check** before writing anything: an NSZ or XCZ input is unpacked first (below), the files open with the keys, the update is 1.1.0, 1.2.1 or 1.4.3 (by hash, as
`tools/extract.py`), the keys are new enough for the update, enough space is free on the target disk (about 36 GiB at the peak of a
first install, 20 GiB after it), the CPU is at least x86-64-v2 (SSE4.2; with AVX2/FMA the faster `v3` build is used,
otherwise the `v2` one), a Vulkan 1.3 GPU exists. Each failure is one clear message.
@@ -71,7 +73,13 @@ loads the saves 1.4.3 wrote.
When the graphics driver crashes while building the shaders, `prepare_shaders.py` runs that part again without the
variants it was building ([building.md](building.md), step 6), and the setup finishes with a warning that names how
many were left out.
7. **Clean up** the temp folder, except the generated C and the objects (about 1.3 GB), and write
7. **DLSS** (the window's tick box, on by default, or `--dlss`; Windows with an NVIDIA RTX GPU): download
`nvngx_dlss.dll` from NVIDIA's own DLSS SDK repository (github.com/NVIDIA/DLSS, a pinned tag, NVIDIA RTX SDK
license) into the install folder and check its size and SHA-256 (`steps.py`, `Dlss`). The release does not ship
the library; NGX loads it from the folder of totk.exe for `dlaa` and the DLSS modes. A DLSS library already in the
folder (a newer one the player put there) is kept. Without an RTX GPU the step only logs and does nothing; a failed
download warns, the game runs without DLSS, and the next run of the setup tries again.
8. **Clean up** the temp folder, except the generated C and the objects (about 1.3 GB), and write
`build-cache/rebuild.txt` in the install folder.
A cancelled or failed setup resumes from the last finished step (`totk-install.json`).
@@ -213,8 +221,15 @@ bin/ totk_xlate, hactool, ninja, libSDL3.so.0
the file and asks to copy it again (`extract.DamagedFile`). A failed merge checks the base Program NCA alone to tell
which file is damaged.
- Unsigned exe: SmartScreen warns on first start.
- NSZ and XCZ input (game files compressed by the nsz tool: NCAs stored as NCZ, their sections decrypted and
compressed with zstd): the check step tells them by their content, not their name (an NSP or XCI holding NCZ files),
so the file dialog has one filter for all four. It unpacks each into `<temp>/unpacked/<base|update>.nsp`
(`tools/ncz.py`), an NSP of the NSZ's files or of the XCZ's secure partition, and the extraction reads that.
Unpacking encrypts the sections again, so every NCA comes back byte for byte, and checks each against the hash its
name carries; a mismatch is `check.input_damaged`. It needs the unpacked size free in the temp folder (checked
first), runs at about 500 MB/s, and an interrupted setup reuses a finished NSP of the same file. The extract step
deletes the folder when it is done. An XCZ update is refused like an XCI one.
## Later
- Re-warm the driver cache on start after a move, GPU or driver change.
- NSZ and XCZ input.
+1 -1
View File
@@ -4,7 +4,7 @@
[--temp DIR]
Modules: layout (where the bundled tools are, release or repository), events (JSON progress on stdout and the log
file), state (totk-install.json), runner (which steps run: resume and update), steps (the seven steps), ini (totk.ini),
file), state (totk-install.json), runner (which steps run: resume and update), steps (the eight steps), ini (totk.ini),
cli (the command line). text/en.json holds the English text of every message id the events carry.
Standard library only, apart from what the extraction tools import (cryptography). Runs on Python 3.14.
+8 -3
View File
@@ -1,5 +1,5 @@
"""Command line: python -m installer --base FILE --update FILE --keys FILE --lang CODE --dest DIR [--firmware DIR]
[--root DIR] [--temp DIR] [--verify]
[--root DIR] [--temp DIR] [--verify] [--dlss]
A first install needs every option. --firmware names the Switch firmware folder (an unpacked firmware or an emulator's
Contents/registered); the setup extracts the system shared fonts from it when the language is Chinese, Korean or
@@ -7,6 +7,9 @@ Traditional Chinese. On a folder with totk-install.json (resume after a cancel o
release) only --dest is needed: --lang defaults to the recorded language, and the game files are needed only when the
extraction has to run again. Prints JSON events on stdout (events.py). Exit code: 0 done, 1 failed, 2 bad command
line, 3 cancelled (Ctrl+C or Ctrl+Break).
--dlss downloads NVIDIA's DLSS library (nvngx_dlss.dll) from NVIDIA's DLSS SDK repository into the install folder on
Windows with an NVIDIA RTX GPU (steps.py, Dlss).
"""
import argparse
import hashlib
@@ -47,8 +50,8 @@ def cancel(signum, frame):
def main(argv=None):
ap = Parser(prog='python -m installer', description=__doc__.splitlines()[0])
ap.add_argument('--base', type=Path, help='base game, XCI or NSP')
ap.add_argument('--update', type=Path, help='the update NSP: 1.1.0, 1.2.1 or 1.4.3')
ap.add_argument('--base', type=Path, help='base game, XCI or NSP (or NSZ, XCZ: unpacked first)')
ap.add_argument('--update', type=Path, help='the update NSP or NSZ: 1.1.0, 1.2.1 or 1.4.3')
ap.add_argument('--keys', type=Path, help='prod.keys')
ap.add_argument('--firmware', type=Path, help='Switch firmware folder (Contents/registered) for the shared fonts '
'of Chinese, Korean and Traditional Chinese text')
@@ -58,6 +61,8 @@ def main(argv=None):
ap.add_argument('--temp', type=Path, help='temp folder (default: temp/ of the release on Windows, .setup-temp in '
'the install folder on Linux, build/install-temp in a repository)')
ap.add_argument('--verify', action='store_true', help='hash every kept file, also those this release made')
ap.add_argument('--dlss', action='store_true', help="download NVIDIA's nvngx_dlss.dll (NVIDIA RTX SDK license) next "
'to the game, for dlaa and the DLSS modes')
args = ap.parse_args(argv)
# The window cancels with Ctrl+Break to the installer's process group (CREATE_NEW_PROCESS_GROUP), which also reaches
+12 -5
View File
@@ -225,7 +225,14 @@ class App:
self.label(grid, 'ui.dest').grid(row=i + 1, column=0, sticky='w', pady=4)
ttk.Entry(grid, textvariable=self.vars['dest']).grid(row=i + 1, column=1, sticky='ew', pady=4)
self.button(grid, 'ui.browse', self.pick_dest).grid(row=i + 1, column=2, padx=(8, 0))
self.label(f, 'ui.compressed_help', wraplength=620, justify='left').pack(anchor='w', pady=(8, 0))
self.label(f, 'ui.firmware_help', wraplength=620, justify='left').pack(anchor='w', pady=(8, 0))
# On by default and named with its source and licence: the player sees what is downloaded and can untick it
self.dlss_var = tk.BooleanVar(value=True)
dlss = ttk.Checkbutton(f, variable=self.dlss_var)
self.bind_text(dlss, 'text', 'ui.dlss')
dlss.pack(anchor='w', pady=(8, 0))
self.label(f, 'ui.dlss_help', wraplength=620, justify='left').pack(anchor='w', padx=(22, 0))
self.update_hint = ttk.Label(f, wraplength=620, justify='left')
self.update_hint.pack(anchor='w', pady=(8, 0))
self.form_error = ttk.Label(f, style='Err.TLabel', wraplength=620, justify='left')
@@ -288,7 +295,8 @@ class App:
def form(self):
return Form(base=self.vars['base'].get(), update=self.vars['update'].get(), keys=self.vars['keys'].get(),
firmware=self.vars['firmware'].get(), lang=self.selected_lang(), dest=self.vars['dest'].get())
firmware=self.vars['firmware'].get(), lang=self.selected_lang(), dest=self.vars['dest'].get(),
dlss=self.dlss_var.get())
def refresh_form(self):
form = self.form()
@@ -303,9 +311,8 @@ class App:
self.fit()
def pick_file(self, name, pick_id):
types = [(self.cat('ui.type_game'), '*.nsp *.xci'), (self.cat('ui.type_all'), '*.*')]
if name == 'update':
types = [(self.cat('ui.type_nsp'), '*.nsp'), (self.cat('ui.type_all'), '*.*')]
# One filter for every game file: the setup tells an NSZ or XCZ from a plain file by its content (Check.unpack)
types = [(self.cat('ui.type_game'), '*.nsp *.xci *.nsz *.xcz'), (self.cat('ui.type_all'), '*.*')]
if name == 'keys':
types = [(self.cat('ui.type_keys'), '*.keys'), (self.cat('ui.type_all'), '*.*')]
path = filedialog.askopenfilename(title=self.cat(pick_id), filetypes=types)
@@ -537,7 +544,7 @@ class App:
# ---- self test ---------------------------------------------------------------------------------------------------
SAMPLE_START = {'type': 'start', 'release': 'selftest', 'mode': 'install', 'dest': 'Z:/selftest/TotK', 'log': 'D:\\log.txt',
'steps': ['check', 'fonts', 'extract', 'lift', 'compile', 'prepare', 'cleanup']}
'steps': ['check', 'fonts', 'extract', 'lift', 'compile', 'prepare', 'dlss', 'cleanup']}
SAMPLE_RUN = [
SAMPLE_START,
{'type': 'plan', 'run': ['check', 'fonts', 'extract'], 'skip': {}},
+4 -1
View File
@@ -27,7 +27,7 @@ TEXT = Path(__file__).resolve().parent / 'text'
UI_LANGUAGES = ('en', 'ru', 'pt-BR', 'zh-Hans')
# what the language switch in the window's corner shows for each
UI_LANGUAGE_LABELS = {'en': 'English', 'ru': 'Русский', 'pt-BR': 'Português (Brasil)', 'zh-Hans': '简体中文'}
STEP_ORDER = ('check', 'fonts', 'extract', 'lift', 'compile', 'prepare', 'cleanup')
STEP_ORDER = ('check', 'fonts', 'extract', 'lift', 'compile', 'prepare', 'dlss', 'cleanup')
# message parameter -> catalog id prefix of the names that replace its raw values
PARAM_NAMES = {'lang': 'lang.', 'known': 'lang.', 'step': 'step.', 'input': 'param.input.'}
LANG_RU = 0x19 # primary language id of Russian in a Windows LANGID
@@ -137,6 +137,7 @@ class Form:
firmware: str = ''
lang: str = ''
dest: str = ''
dlss: bool = False
def installed_state(self):
"""The recorded state of the install folder (totk-install.json), or None. Never raises."""
@@ -187,6 +188,8 @@ class Form:
cmd += ['--temp', str(temp)]
if verify_hashes:
cmd.append('--verify')
if self.dlss:
cmd.append('--dlss')
return cmd
+2 -1
View File
@@ -90,7 +90,8 @@ class Layout:
def missing(self, cpu='v3'):
"""Paths the steps need that do not exist. cpu picks the runtime library the host will use."""
lib = self.runtime_lib_for(cpu)
need = [self.python, self.tools / 'extract.py', self.tools / 'build_game.py', self.tools / 'prepare_shaders.py',
need = [self.python, self.tools / 'extract.py', self.tools / 'ncz.py', self.tools / 'build_game.py',
self.tools / 'prepare_shaders.py',
self.tools / 'pack_shader_cache.py', self.tools / 'make_romfs_patches.py', self.tools / 'ultrawide_layout.py',
self.tools / 'button_icons.py',
self.tools / 'vendor' / 'macnativegames' / 'extract_shared_fonts.py',
+164 -16
View File
@@ -1,4 +1,4 @@
"""The seven setup steps of docs/installer.md: check, fonts, extract, lift, compile, prepare, cleanup.
"""The eight setup steps of docs/installer.md: check, fonts, extract, lift, compile, prepare, dlss, cleanup.
Extraction runs in this process through tools/extract.py (the package's sibling tools/, which ships with it); fonts
runs the vendored extract_shared_fonts.py on the firmware folder; lift, compile and prepare run the bundled Python on
@@ -7,18 +7,20 @@ recomp/gen.py, tools/build_game.py, tools/make_romfs_patches.py and tools/prepar
with the bundled clang (hostcheck.c).
Install folder: exefs/, romfs/ (romfs.bin, index.json), profile/, shared-fonts/ (Chinese and Korean text), totk.exe
and its DLLs, romfs-index.bin, romfs-patches/, romfs-patches-xbox/, romfs-patches-xbox-label/, totk_xlate.exe, the
and its DLLs, nvngx_dlss.dll (with --dlss), romfs-index.bin, romfs-patches/, romfs-patches-xbox/, romfs-patches-xbox-label/, totk_xlate.exe, the
shader state of totk.exe --prepare-shaders, totk.ini, totk-install.json.
Temp folder (<temp>/<dest name>-<hash>): check/ (ExeFS of both inputs, the host check), gen/ (C), build/ (objects),
logs/. The cleanup keeps gen/, build/ and logs/: the next run lifts into the same gen/, where recomp/gen.py rewrites
only the chunks whose C changed, and Ninja in build/ recompiles only those. A change of the mods' code cheats then
costs the lift (about a minute) and the chunks it touches, not a full compile.
Temp folder (<temp>/<dest name>-<hash>): check/ (ExeFS of both inputs, the host check), unpacked/ (the NSP of an NSZ
or XCZ input until the extraction is done), gen/ (C), build/ (objects), logs/. The cleanup keeps gen/, build/ and
logs/: the next run lifts into the same gen/, where recomp/gen.py rewrites only the chunks whose C changed, and Ninja
in build/ recompiles only those. A change of the mods' code cheats then costs the lift (about a minute) and the chunks
it touches, not a full compile.
Install folder build-cache/: cheats.status (tools/cheats.py), exefs.status (tools/exefs_patches.py) and rebuild.txt, the command that starts this setup in
its rebuild mode on the install folder (installer/gui.py --rebuild), one argument per line; totk.exe runs it at start
when an enabled mod has code cheats the build lacks (runtime/cheats.c).
"""
import hashlib
import http.client
import json
import os
import re
@@ -28,6 +30,7 @@ import subprocess
import sys
import threading
import time
import urllib.request
from pathlib import Path
from . import ini
@@ -36,6 +39,7 @@ from .runner import InstallError, Step
sys.path.insert(0, str(Path(__file__).resolve().parent.parent / 'tools'))
import extract # noqa: E402
import ncz # noqa: E402
import shader_list # noqa: E402
from extract_exefs import (common_tickets, container_entries, load_keys, nca_ctr_key, nca_inventory, # noqa: E402
program_nca)
@@ -174,6 +178,12 @@ def raise_if_damaged(error, name, path):
raise InstallError('check.input_damaged', input=name, path=str(path), detail=str(error)) from None
def game_file(ctx, name):
"""The file the extraction reads for the base or the update: the NSP that Check.unpack made from an NSZ or XCZ,
else the chosen file itself."""
return getattr(ctx, 'unpacked', {}).get(name) or Path(getattr(ctx.options, name))
def game_version(ctx):
"""The update this install is built from: what the check identified, else the recorded one. An install made before
1.4.3 support records no version, and that install is 1.2.1: name it here rather than leaning on SUPPORTED's order."""
@@ -451,14 +461,73 @@ class Check(Step):
raise InstallError('check.keys_unreadable') from None
if 'header_key' not in keys:
raise InstallError('check.keys_no_header_key')
self.unpack(ctx)
total = 0
for name, is_update in (('base', False), ('update', True)):
path = Path(getattr(o, name))
path = game_file(ctx, name)
with path.open('rb') as f:
total += self.open_input(f, path, keys, name, is_update)
del keys
return total
def unpack(self, ctx):
"""Unpack an NSZ or XCZ (a game file compressed by the nsz tool) into <temp>/unpacked/<base|update>.nsp with
tools/ncz.py, which checks every NCA against its hash; the extraction then reads that NSP. An earlier run's NSP
of the same file is reused. The extract step removes the folder once it is done."""
o, rep, folder = ctx.options, ctx.reporter, ctx.temp / 'unpacked'
ctx.unpacked = {}
jobs = []
for name in ('base', 'update'):
path = Path(getattr(o, name))
try:
found = ncz.compressed(path)
if found is None:
continue
if name == 'update' and found[0] != 'nsz':
raise InstallError('check.update_not_nsp')
size = ncz.unpacked_size(path)
except ValueError as e:
raise_if_damaged(e, name, path)
raise InstallError('check.not_game_file', input=name) from None
st = path.stat()
jobs.append((name, path, size, f'{path.resolve()}\n{st.st_size}\n{st.st_mtime_ns}\n{size}\n'))
if not jobs:
remove(folder)
return
done = []
for name, path, size, tag in jobs:
target, stamp = folder / f'{name}.nsp', folder / f'{name}.source'
try:
if target.stat().st_size == size and stamp.read_text(encoding='utf-8') == tag:
done.append(name)
ctx.unpacked[name] = target
except OSError:
pass
need = sum(size for name, _, size, _ in jobs if name not in done)
free, _ = free_bytes(folder)
rep.log(f'unpack: {", ".join(name for name, *_ in jobs)}: {need / GiB:.1f} GiB to write, '
f'{free / GiB:.1f} free')
if need > free:
raise InstallError('check.disk_space', path=str(folder), need_gb=round(need / 1e9, 1),
free_gb=round(free / 1e9, 1))
for name, path, size, tag in jobs:
if name in done:
continue
target, stamp = folder / f'{name}.nsp', folder / f'{name}.source'
remove(stamp)
try:
folder.mkdir(parents=True, exist_ok=True)
ncz.unpack(path, target, lambda d, t, n=name: rep.progress('check', 'unpack_' + n, d, t, 'bytes'))
stamp.write_text(tag, encoding='utf-8')
except ncz.DamagedFile as e:
raise InstallError('check.input_damaged', input=name, path=str(e.path), detail=e.detail) from None
except ValueError as e:
raise_if_damaged(e, name, path)
raise InstallError('check.unpack_failed', input=name, detail=str(e)) from None
except OSError as e:
raise InstallError('check.unpack_failed', input=name, detail=str(e)) from None
ctx.unpacked[name] = target
@staticmethod
def open_input(f, path, keys, name, is_update):
try:
@@ -539,9 +608,10 @@ class Check(Step):
out = work / name
remove(out)
try:
extract.extract_exefs(Path(getattr(o, name)), Path(o.keys), out, logs, update=is_update)
extract.extract_exefs(game_file(ctx, name), Path(o.keys), out, logs, update=is_update)
except extract.DamagedFile as e:
raise InstallError('check.input_damaged', input=name, path=str(e.path), detail=e.detail) from None
raise InstallError('check.input_damaged', input=name, path=str(getattr(o, name)),
detail=e.detail) from None
except ValueError:
raise InstallError('check.exefs_failed', input=name, log=str(logs / f'extract_exefs-{name}.log')) \
from None
@@ -660,9 +730,9 @@ class Extract(Step):
def run(self, ctx):
o, dest, rep = ctx.options, ctx.dest, ctx.reporter
expected = expected_update(ctx)
base, update, keys = Path(o.base), Path(o.update), Path(o.keys)
keys = Path(o.keys)
inputs = {}
for name, path in (('base', base), ('update', update)):
for name, path in (('base', Path(o.base)), ('update', Path(o.update))):
known = ctx.state['inputs'].get(name, {})
st = path.stat()
if known.get('size') == st.st_size and known.get('mtime_ns') == st.st_mtime_ns and known.get('sha256'):
@@ -672,6 +742,7 @@ class Extract(Step):
inputs[name] = {'name': path.name, 'size': st.st_size, 'mtime_ns': st.st_mtime_ns, 'sha256': digest}
ctx.state['inputs'] = inputs
ctx.save()
base, update = game_file(ctx, 'base'), game_file(ctx, 'update')
dest.mkdir(parents=True, exist_ok=True)
for stale in dest.glob('update-merge-*'): # a killed merge leaves its NCA copies (and key copies) behind
@@ -683,7 +754,7 @@ class Extract(Step):
try:
extract.extract_exefs(update, keys, ctx.temp / 'check' / 'update', ctx.temp / 'logs', update=True)
except extract.DamagedFile as e:
raise InstallError('check.input_damaged', input='update', path=str(e.path), detail=e.detail) from None
raise InstallError('check.input_damaged', input='update', path=str(o.update), detail=e.detail) from None
except ValueError:
raise InstallError('check.exefs_failed', input='update',
log=str(ctx.temp / 'logs' / 'extract_exefs-update.log')) from None
@@ -705,6 +776,7 @@ class Extract(Step):
raise InstallError('extract.romfs_mismatch', version=game_version(ctx), size=report['romfs_size'],
sha256=report['romfs_sha256'][:16])
(dest / 'profile').mkdir(exist_ok=True)
remove(ctx.temp / 'unpacked') # the NSPs unpacked from an NSZ or XCZ (Check.unpack)
st = (romfs / 'romfs.bin').stat()
return {'romfs_size': st.st_size, 'romfs_mtime_ns': st.st_mtime_ns}
@@ -726,7 +798,8 @@ class Extract(Step):
try:
extract.merge_update_romfs(base, update, keys, ctx.layout.hactool, romfs, logs, phase=phase)
except extract.DamagedFile as e:
raise InstallError('check.input_damaged', input='base' if e.path == base else 'update', path=str(e.path),
name = 'base' if e.path == base else 'update'
raise InstallError('check.input_damaged', input=name, path=str(getattr(ctx.options, name)),
detail=e.detail) from None
except ValueError as e:
raise InstallError('extract.romfs_merge_failed', detail=str(e), log=str(logs / 'merge_romfs-'
@@ -1043,7 +1116,82 @@ class Prepare(Step):
return {'xlate_sha256': file_sha256(xlate)}
# ---- 7. clean up
# ---- 7. NVIDIA DLSS
# NVIDIA's DLSS library, which NGX loads from the folder of totk.exe (runtime/vk_dlaa.c). The setup does not ship it:
# with --dlss it downloads NVIDIA's own copy from the DLSS SDK repository (NVIDIA RTX SDK license) at a pinned tag and
# checks its hash. The game runs without it; dlaa and the DLSS modes then do nothing.
DLSS_DLL = 'nvngx_dlss.dll'
DLSS_VERSION = '310.9.1'
DLSS_URL = f'https://raw.githubusercontent.com/NVIDIA/DLSS/v{DLSS_VERSION}/lib/Windows_x86_64/rel/{DLSS_DLL}'
DLSS_SIZE = 58956912
DLSS_SHA256 = '3975567b8943c53acce397f2b72380092f84f162d00b0d2c7d08a1025c563983'
DLSS_LICENSE = 'https://github.com/NVIDIA/DLSS/blob/main/LICENSE.txt'
def nvidia_rtx(host):
"""An NVIDIA RTX GPU among the host check's Vulkan devices: the GPUs DLSS runs on."""
return any('NVIDIA' in d['name'].upper() and 'RTX' in d['name'].upper() for d in (host or {}).get('devices', []))
class Dlss(Step):
"""With --dlss on Windows and an NVIDIA RTX GPU: nvngx_dlss.dll next to totk.exe. A DLSS library the player put
there (another version) is kept. A failed download warns and is tried again on the next run; the game runs
without it."""
id = 'dlss'
def key(self, ctx):
if not getattr(ctx.options, 'dlss', False):
return fingerprint('dlss-1', 'off')
return fingerprint('dlss-1', DLSS_VERSION, DLSS_SHA256)
def current(self, ctx, record):
if record.get('failed'):
return False
return not record.get('installed') or (ctx.dest / DLSS_DLL).is_file()
def run(self, ctx):
if not getattr(ctx.options, 'dlss', False):
return {}
if os.name != 'nt':
return {'skipped': 'not_windows'}
if not nvidia_rtx(getattr(ctx, 'host', None)):
# The box is ticked by default, so most players without an RTX GPU never chose it: a log line, no warning
ctx.reporter.log('dlss: no NVIDIA RTX GPU among the Vulkan devices, nothing downloaded')
return {'skipped': 'no_rtx'}
rep, target = ctx.reporter, ctx.dest / DLSS_DLL
if target.is_file():
if file_sha256(target) == DLSS_SHA256:
return {'installed': DLSS_VERSION}
rep.warning('warn.dlss_kept', path=str(target))
return {'kept': True}
part = ctx.temp / (DLSS_DLL + '.part')
rep.progress('dlss', 'download', 0, DLSS_SIZE, 'bytes', force=True)
rep.log(f'dlss: {DLSS_URL} (license {DLSS_LICENSE})')
try:
part.parent.mkdir(parents=True, exist_ok=True)
h, done = hashlib.sha256(), 0
with urllib.request.urlopen(DLSS_URL, timeout=60) as r, open(part, 'wb') as f:
while block := r.read(1 << 20):
f.write(block)
h.update(block)
done += len(block)
rep.progress('dlss', 'download', min(done, DLSS_SIZE), DLSS_SIZE, 'bytes')
if done != DLSS_SIZE or h.hexdigest() != DLSS_SHA256:
raise OSError(f'{done} bytes, SHA-256 {h.hexdigest()[:16]}, not the expected file')
replace_file(part, target)
remove(part)
# urllib's errors, timeouts and a bad file; a connection cut mid-file raises http.client.IncompleteRead,
# which is not an OSError
except (OSError, http.client.HTTPException) as e:
remove(part)
rep.warning('warn.dlss_download_failed', url=DLSS_URL, detail=str(e)[:300])
return {'failed': True}
return {'installed': DLSS_VERSION}
# ---- 8. clean up
class Cleanup(Step):
id = 'cleanup'
@@ -1061,7 +1209,7 @@ class Cleanup(Step):
rep.progress('cleanup', 'temp', force=True)
for stale in dest.glob('update-merge-*'):
remove(stale)
for name in ('check', 'shader_list.tsv', 'shared-fonts'): # gen/ and build/ stay for the next rebuild
for name in ('check', 'unpacked', 'shader_list.tsv', 'shared-fonts'): # gen/ and build/ stay for the next rebuild
remove(ctx.temp / name)
romfs_problem(dest / 'romfs', expected_update(ctx))
write_rebuild_command(ctx)
@@ -1111,4 +1259,4 @@ def romfs_problem(romfs, expected):
def all_steps():
return [Check(), Fonts(), Extract(), Lift(), Compile(), Prepare(), Cleanup()]
return [Check(), Fonts(), Extract(), Lift(), Compile(), Prepare(), Dlss(), Cleanup()]
+15 -6
View File
@@ -5,12 +5,15 @@
"step.lift": "Translating the game code to C",
"step.compile": "Compiling {game_exe}",
"step.prepare": "Preparing the shaders",
"step.dlss": "Downloading NVIDIA DLSS",
"step.cleanup": "Cleaning up",
"phase.check.host": "Checking the processor and the graphics card",
"phase.check.verify": "Checking the files already installed",
"phase.check.verify_romfs": "Checking the installed game data",
"phase.check.inputs": "Opening the game files with the keys",
"phase.check.unpack_base": "Unpacking the compressed base game",
"phase.check.unpack_update": "Unpacking the compressed update",
"phase.check.versions": "Checking the game versions",
"phase.fonts.extract": "Extracting the shared fonts from the firmware",
"phase.fonts.copy": "Copying the shared fonts into the install folder",
@@ -30,6 +33,7 @@
"phase.prepare.translate": "Translating the shaders",
"phase.prepare.libraries": "Building the shaders for the graphics driver",
"phase.prepare.pack": "Packing the shader cache into one file",
"phase.dlss.download": "Downloading nvngx_dlss.dll from NVIDIA's DLSS SDK",
"phase.cleanup.ini": "Writing totk.ini",
"phase.cleanup.temp": "Removing temporary files",
@@ -58,14 +62,15 @@
"check.keys_no_header_key": "The keys file has no header_key. Dump prod.keys from your console again.",
"check.keys_wrong": "The keys cannot open the {input} file. Use the prod.keys of your console.",
"check.keys_too_old": "The keys file is too old for the {input} file (needs key generation {generation}). Dump prod.keys from a console with newer firmware.",
"check.not_game_file": "The {input} file is not an XCI or NSP game file.",
"check.not_game_file": "The {input} file is not an XCI, NSP, XCZ or NSZ game file.",
"check.file_unreadable": "The {input} file cannot be read: {detail}",
"check.unpack_failed": "The compressed {input} file could not be unpacked: {detail}",
"check.ticket_personalized": "The {input} file carries a personalized ticket; a common ticket is needed.",
"check.ticket_missing": "The {input} file lacks the ticket its content needs.",
"check.base_not_totk": "The base game file is not The Legend of Zelda: Tears of the Kingdom ({detail}).",
"check.base_is_update": "The base game file is an update. Choose the base game NSP or XCI (version 1.0.0) here and this file as the update.",
"check.base_combined": "The base game file holds several packages (base game, update or DLC in one file). Choose a base game dump that contains only the game.",
"check.update_not_nsp": "The update must be an NSP file.",
"check.update_not_nsp": "The update must be an NSP or NSZ file.",
"check.update_is_base": "The update file holds a base game. Choose the 1.1.0 or 1.2.1 or 1.4.3 update NSP.",
"check.update_not_totk": "The update file is not a Tears of the Kingdom update ({detail}).",
"check.exefs_failed": "The game code of the {input} file failed its checks; see {log}.",
@@ -101,6 +106,8 @@
"warn.xbox_prompts_failed": "The Xbox button prompts could not be prepared (exit code {code}); the game keeps the Switch prompts. Details: {log}",
"warn.shaders_skipped": "The graphics driver crashed while building some shaders, so {count} were left out of the preparation. The game builds them when it first needs them, and the driver may crash there too; a driver update may fix it. Details: {log}",
"warn.kernel_old": "Linux {version} lacks the write tracking the game relies on (6.7 or later); the game may run very slowly. Updating the system fixes this.",
"warn.dlss_kept": "A DLSS library is already in the install folder ({path}); the setup keeps it.",
"warn.dlss_download_failed": "NVIDIA DLSS could not be downloaded from {url}: {detail}. The game runs without it; the next setup run tries again.",
"ui.title": "TotK Setup",
"ui.rebuild_title": "Rebuilding the game for your mods",
@@ -109,8 +116,8 @@
"ui.intro": "Builds a playable Tears of the Kingdom folder from your own game files. This takes a while and runs once.",
"ui.notice": "The folder this setup builds contains the game. It is for your own use only; do not share or upload it.",
"ui.keys_private": "The keys file is read in place. Its contents are never shown, copied or stored.",
"ui.base": "Base game (NSP or XCI)",
"ui.update": "Update 1.1.0 or 1.2.1 or 1.4.3 (NSP)",
"ui.base": "Base game (NSP, XCI, NSZ or XCZ)",
"ui.update": "Update 1.1.0 or 1.2.1 or 1.4.3 (NSP or NSZ)",
"ui.keys": "prod.keys",
"ui.firmware": "Firmware folder (optional)",
"ui.game_language": "Game language",
@@ -122,8 +129,10 @@
"ui.pick_firmware": "Choose the Switch firmware folder (Contents/registered)",
"ui.pick_dest": "Choose the install folder",
"ui.firmware_help": "For Chinese, Korean and Traditional Chinese text: the firmware's Contents/registered folder, for example from Ryujinx or Yuzu. The setup copies the Switch system fonts from it into the install folder.",
"ui.type_game": "Game files (NSP, XCI)",
"ui.type_nsp": "NSP files",
"ui.dlss": "Download NVIDIA DLSS (nvngx_dlss.dll) for NVIDIA RTX graphics cards",
"ui.dlss_help": "From NVIDIA's DLSS SDK on GitHub, under NVIDIA's RTX SDK license (github.com/NVIDIA/DLSS). It makes the dlaa and DLSS settings work; without it they do nothing.",
"ui.compressed_help": "Compressed NSZ and XCZ files work as they are: the setup unpacks them by itself (until the game data is extracted, that needs as much free space as the unpacked files take).",
"ui.type_game": "Game files (NSP, XCI, NSZ, XCZ)",
"ui.type_keys": "Key files (*.keys)",
"ui.type_all": "All files",
"ui.start": "Start",
+15 -6
View File
@@ -5,12 +5,15 @@
"step.lift": "Traduzindo o código do jogo para C",
"step.compile": "Compilando o {game_exe}",
"step.prepare": "Preparando os shaders",
"step.dlss": "Baixando o NVIDIA DLSS",
"step.cleanup": "Finalizando",
"phase.check.host": "Verificando o processador e a placa de vídeo",
"phase.check.verify": "Verificando os arquivos já instalados",
"phase.check.verify_romfs": "Verificando os dados do jogo instalados",
"phase.check.inputs": "Abrindo os arquivos do jogo com as chaves",
"phase.check.unpack_base": "Descompactando o jogo base compactado",
"phase.check.unpack_update": "Descompactando a atualização compactada",
"phase.check.versions": "Verificando as versões do jogo",
"phase.fonts.extract": "Extraindo as fontes compartilhadas do firmware",
"phase.fonts.copy": "Copiando as fontes compartilhadas para a pasta de instalação",
@@ -30,6 +33,7 @@
"phase.prepare.translate": "Traduzindo os shaders",
"phase.prepare.libraries": "Montando os shaders para o driver de vídeo",
"phase.prepare.pack": "Juntando o cache de shaders num único arquivo",
"phase.dlss.download": "Baixando o nvngx_dlss.dll do DLSS SDK da NVIDIA",
"phase.cleanup.ini": "Gravando o totk.ini",
"phase.cleanup.temp": "Removendo os arquivos temporários",
@@ -58,14 +62,15 @@
"check.keys_no_header_key": "O arquivo de chaves não tem header_key. Extraia o prod.keys do seu console de novo.",
"check.keys_wrong": "As chaves não abrem o arquivo de {input}. Use o prod.keys do seu console.",
"check.keys_too_old": "O arquivo de chaves é antigo demais para o arquivo de {input} (precisa da geração de chaves {generation}). Extraia o prod.keys de um console com firmware mais novo.",
"check.not_game_file": "O arquivo de {input} não é um arquivo de jogo XCI ou NSP.",
"check.not_game_file": "O arquivo de {input} não é um arquivo de jogo XCI, NSP, XCZ ou NSZ.",
"check.file_unreadable": "Não foi possível ler o arquivo de {input}: {detail}",
"check.unpack_failed": "Não foi possível descompactar o arquivo compactado de {input}: {detail}",
"check.ticket_personalized": "O arquivo de {input} tem um ticket personalizado; é preciso um ticket comum.",
"check.ticket_missing": "O arquivo de {input} não tem o ticket de que o conteúdo precisa.",
"check.base_not_totk": "O arquivo do jogo base não é The Legend of Zelda: Tears of the Kingdom ({detail}).",
"check.base_is_update": "O arquivo do jogo base é uma atualização. Escolha aqui o NSP ou XCI do jogo base (versão 1.0.0) e este arquivo como a atualização.",
"check.base_combined": "O arquivo do jogo base contém vários pacotes (jogo base, atualização ou DLC em um só arquivo). Escolha um dump do jogo base que contenha só o jogo.",
"check.update_not_nsp": "A atualização precisa ser um arquivo NSP.",
"check.update_not_nsp": "A atualização precisa ser um arquivo NSP ou NSZ.",
"check.update_is_base": "O arquivo de atualização contém um jogo base. Escolha o NSP da atualização 1.1.0, 1.2.1 ou 1.4.3.",
"check.update_not_totk": "O arquivo de atualização não é uma atualização de Tears of the Kingdom ({detail}).",
"check.exefs_failed": "O código do jogo no arquivo de {input} falhou nas verificações; veja {log}.",
@@ -101,6 +106,8 @@
"warn.xbox_prompts_failed": "Não foi possível preparar os ícones de botão do Xbox (código de saída {code}); o jogo mantém os ícones do Switch. Detalhes: {log}",
"warn.shaders_skipped": "O driver gráfico travou ao compilar alguns shaders, então {count} ficaram fora da preparação. O jogo os compila quando precisar deles pela primeira vez, e o driver pode travar ali também; uma atualização do driver pode resolver. Detalhes: {log}",
"warn.kernel_old": "O Linux {version} não tem o rastreamento de escrita de que o jogo depende (6.7 ou mais novo); o jogo pode rodar muito devagar. Atualizar o sistema resolve.",
"warn.dlss_kept": "Já existe uma biblioteca DLSS na pasta de instalação ({path}); a instalação a mantém.",
"warn.dlss_download_failed": "Não foi possível baixar o NVIDIA DLSS de {url}: {detail}. O jogo funciona sem ele; a próxima execução da instalação tenta de novo.",
"ui.title": "Instalação do TotK",
"ui.rebuild_title": "Recompilando o jogo para os seus mods",
@@ -109,8 +116,8 @@
"ui.intro": "Monta uma pasta jogável de Tears of the Kingdom a partir dos seus próprios arquivos do jogo. Isso demora um pouco e só precisa ser feito uma vez.",
"ui.notice": "A pasta que esta instalação monta contém o jogo. Ela é só para seu uso pessoal; não compartilhe nem envie para a internet.",
"ui.keys_private": "O arquivo de chaves é lido no lugar. O conteúdo dele nunca é mostrado, copiado nem guardado.",
"ui.base": "Jogo base (NSP ou XCI)",
"ui.update": "Atualização 1.1.0, 1.2.1 ou 1.4.3 (NSP)",
"ui.base": "Jogo base (NSP, XCI, NSZ ou XCZ)",
"ui.update": "Atualização 1.1.0, 1.2.1 ou 1.4.3 (NSP ou NSZ)",
"ui.keys": "prod.keys",
"ui.firmware": "Pasta de firmware (opcional)",
"ui.game_language": "Idioma do jogo",
@@ -122,8 +129,10 @@
"ui.pick_firmware": "Escolha a pasta de firmware do Switch (Contents/registered)",
"ui.pick_dest": "Escolha a pasta de instalação",
"ui.firmware_help": "Para textos em chinês, coreano e chinês tradicional: a pasta Contents/registered do firmware, por exemplo do Ryujinx ou do Yuzu. A instalação copia dela as fontes do sistema do Switch para a pasta de instalação.",
"ui.type_game": "Arquivos de jogo (NSP, XCI)",
"ui.type_nsp": "Arquivos NSP",
"ui.dlss": "Baixar o NVIDIA DLSS (nvngx_dlss.dll) para placas de vídeo NVIDIA RTX",
"ui.dlss_help": "Do DLSS SDK da NVIDIA no GitHub, sob a licença NVIDIA RTX SDK (github.com/NVIDIA/DLSS). Sem ele as opções dlaa e DLSS não fazem nada.",
"ui.compressed_help": "Arquivos compactados NSZ e XCZ funcionam do jeito que estão: a instalação os descompacta sozinha (para isso, até os dados do jogo serem extraídos, ela precisa de tanto espaço livre quanto ocupam os arquivos descompactados).",
"ui.type_game": "Arquivos de jogo (NSP, XCI, NSZ, XCZ)",
"ui.type_keys": "Arquivos de chaves (*.keys)",
"ui.type_all": "Todos os arquivos",
"ui.start": "Iniciar",
+15 -6
View File
@@ -5,12 +5,15 @@
"step.lift": "Перевод кода игры в C",
"step.compile": "Сборка {game_exe}",
"step.prepare": "Подготовка шейдеров",
"step.dlss": "Загрузка NVIDIA DLSS",
"step.cleanup": "Уборка",
"phase.check.host": "Проверка процессора и видеокарты",
"phase.check.verify": "Проверка уже установленных файлов",
"phase.check.verify_romfs": "Проверка установленных данных игры",
"phase.check.inputs": "Открытие игровых файлов с помощью ключей",
"phase.check.unpack_base": "Распаковка сжатой базовой игры",
"phase.check.unpack_update": "Распаковка сжатого обновления",
"phase.check.versions": "Проверка версий игры",
"phase.fonts.extract": "Извлечение общих шрифтов из прошивки",
"phase.fonts.copy": "Копирование общих шрифтов в папку установки",
@@ -30,6 +33,7 @@
"phase.prepare.translate": "Трансляция шейдеров",
"phase.prepare.libraries": "Сборка шейдеров для драйвера видеокарты",
"phase.prepare.pack": "Упаковка кэша шейдеров в один файл",
"phase.dlss.download": "Загрузка nvngx_dlss.dll из DLSS SDK от NVIDIA",
"phase.cleanup.ini": "Запись totk.ini",
"phase.cleanup.temp": "Удаление временных файлов",
@@ -58,14 +62,15 @@
"check.keys_no_header_key": "В файле ключей нет header_key. Снова получите prod.keys со своей консоли.",
"check.keys_wrong": "Эти ключи не открывают файл «{input}». Используйте prod.keys со своей консоли.",
"check.keys_too_old": "Файл ключей слишком старый для файла «{input}» (нужно поколение ключей {generation}). Получите prod.keys с консоли с более новой прошивкой.",
"check.not_game_file": "Файл «{input}» не является игровым файлом XCI или NSP.",
"check.not_game_file": "Файл «{input}» не является игровым файлом XCI, NSP, XCZ или NSZ.",
"check.file_unreadable": "Не удаётся прочитать файл «{input}»: {detail}",
"check.unpack_failed": "Не удалось распаковать сжатый файл «{input}»: {detail}",
"check.ticket_personalized": "В файле «{input}» личный тикет; нужен общий тикет.",
"check.ticket_missing": "В файле «{input}» нет тикета, который нужен его содержимому.",
"check.base_not_totk": "Файл базовой игры — не The Legend of Zelda: Tears of the Kingdom ({detail}).",
"check.base_is_update": "Файл базовой игры — это обновление. Выберите здесь NSP или XCI базовой игры (версия 1.0.0), а этот файл укажите как обновление.",
"check.base_combined": "Файл базовой игры содержит несколько пакетов (игра, обновление или DLC в одном файле). Выберите дамп, в котором только базовая игра.",
"check.update_not_nsp": "Обновление должно быть файлом NSP.",
"check.update_not_nsp": "Обновление должно быть файлом NSP или NSZ.",
"check.update_is_base": "Файл обновления содержит базовую игру. Выберите NSP с обновлением 1.1.0 или 1.2.1 или 1.4.3.",
"check.update_not_totk": "Файл обновления — не обновление Tears of the Kingdom ({detail}).",
"check.exefs_failed": "Код игры из файла «{input}» не прошёл проверку; подробности в {log}.",
@@ -101,6 +106,8 @@
"warn.xbox_prompts_failed": "Не удалось подготовить подсказки кнопок Xbox (код выхода {code}); игра оставит подсказки Switch. Подробности: {log}",
"warn.shaders_skipped": "Графический драйвер аварийно завершился при сборке части шейдеров, поэтому {count} из них пропущены при подготовке. Игра соберёт их, когда они впервые понадобятся, и драйвер может упасть и там; обновление драйвера может это исправить. Подробности: {log}",
"warn.kernel_old": "В Linux {version} нет отслеживания записи, на которое опирается игра (нужна версия 6.7 или новее); игра может работать очень медленно. Обновление системы это исправит.",
"warn.dlss_kept": "В папке установки уже есть библиотека DLSS ({path}); установка её оставляет.",
"warn.dlss_download_failed": "Не удалось загрузить NVIDIA DLSS с {url}: {detail}. Игра работает и без него; при следующем запуске установка попробует снова.",
"ui.title": "Установка TotK",
"ui.rebuild_title": "Пересборка игры под ваши моды",
@@ -109,8 +116,8 @@
"ui.intro": "Собирает готовую к запуску папку с Tears of the Kingdom из ваших игровых файлов. Занимает много времени и выполняется один раз.",
"ui.notice": "Папка, которую создаёт установка, содержит игру. Она предназначена только для личного использования: не передавайте и не выкладывайте её.",
"ui.keys_private": "Файл ключей читается на месте. Его содержимое не показывается, не копируется и не сохраняется.",
"ui.base": "Базовая игра (NSP или XCI)",
"ui.update": "Обновление 1.1.0 или 1.2.1 или 1.4.3 (NSP)",
"ui.base": "Базовая игра (NSP, XCI, NSZ или XCZ)",
"ui.update": "Обновление 1.1.0 или 1.2.1 или 1.4.3 (NSP или NSZ)",
"ui.keys": "prod.keys",
"ui.firmware": "Папка прошивки (необязательно)",
"ui.game_language": "Язык игры",
@@ -122,8 +129,10 @@
"ui.pick_firmware": "Выберите папку прошивки Switch (Contents/registered)",
"ui.pick_dest": "Выберите папку установки",
"ui.firmware_help": "Для текста на китайском и корейском: папка Contents/registered прошивки, например из Ryujinx или Yuzu. Установка скопирует из неё системные шрифты Switch в папку установки.",
"ui.type_game": "Файлы игры (NSP, XCI)",
"ui.type_nsp": "Файлы NSP",
"ui.dlss": "Загрузить NVIDIA DLSS (nvngx_dlss.dll) для видеокарт NVIDIA RTX",
"ui.dlss_help": "Из DLSS SDK от NVIDIA на GitHub, по лицензии NVIDIA RTX SDK (github.com/NVIDIA/DLSS). Без него настройки dlaa и DLSS ничего не делают.",
"ui.compressed_help": "Сжатые файлы NSZ и XCZ подходят как есть: установка сама их распакует (для этого, пока извлекаются данные игры, нужно столько свободного места, сколько занимают распакованные файлы).",
"ui.type_game": "Файлы игры (NSP, XCI, NSZ, XCZ)",
"ui.type_keys": "Файлы ключей (*.keys)",
"ui.type_all": "Все файлы",
"ui.start": "Начать",
+15 -6
View File
@@ -5,12 +5,15 @@
"step.lift": "将游戏代码转换为 C 代码",
"step.compile": "编译 {game_exe}",
"step.prepare": "准备着色器",
"step.dlss": "下载 NVIDIA DLSS",
"step.cleanup": "清理",
"phase.check.host": "检查处理器和显卡",
"phase.check.verify": "检查已安装的文件",
"phase.check.verify_romfs": "检查已安装的游戏数据",
"phase.check.inputs": "用密钥打开游戏文件",
"phase.check.unpack_base": "正在解压压缩的游戏本体",
"phase.check.unpack_update": "正在解压压缩的更新",
"phase.check.versions": "检查游戏版本",
"phase.fonts.extract": "正在从固件中提取共享字体",
"phase.fonts.copy": "正在将共享字体复制到安装文件夹",
@@ -30,6 +33,7 @@
"phase.prepare.translate": "转译着色器",
"phase.prepare.libraries": "为显卡驱动构建着色器",
"phase.prepare.pack": "将着色器缓存打包成一个文件",
"phase.dlss.download": "从 NVIDIA 的 DLSS SDK 下载 nvngx_dlss.dll",
"phase.cleanup.ini": "写入 totk.ini",
"phase.cleanup.temp": "删除临时文件",
@@ -58,14 +62,15 @@
"check.keys_no_header_key": "密钥文件中没有 header_key。请从你的主机重新导出 prod.keys。",
"check.keys_wrong": "这些密钥无法打开{input}文件。请使用你自己主机的 prod.keys。",
"check.keys_too_old": "密钥文件对{input}文件来说太旧(需要密钥代数 {generation})。请从固件更新的主机导出 prod.keys。",
"check.not_game_file": "{input}文件不是 XCI 或 NSP 游戏文件。",
"check.not_game_file": "{input}文件不是 XCI、NSP、XCZ 或 NSZ 游戏文件。",
"check.file_unreadable": "无法读取{input}文件:{detail}",
"check.unpack_failed": "无法解压压缩的{input}文件:{detail}",
"check.ticket_personalized": "{input}文件带有个人化票据(personalized ticket);需要通用票据(common ticket)。",
"check.ticket_missing": "{input}文件缺少其内容所需的票据(ticket)。",
"check.base_not_totk": "游戏本体文件不是《塞尔达传说 王国之泪》({detail})。",
"check.base_is_update": "基础游戏文件是一个更新。请在这里选择基础游戏的 NSP 或 XCI(版本 1.0.0),并把此文件选为更新。",
"check.base_combined": "基础游戏文件包含多个内容包(基础游戏、更新或 DLC 合在一个文件中)。请选择只包含游戏本体的基础游戏转储。",
"check.update_not_nsp": "更新必须是 NSP 文件。",
"check.update_not_nsp": "更新必须是 NSP 或 NSZ 文件。",
"check.update_is_base": "更新文件里是游戏本体。请选择 1.1.0、1.2.1 或 1.4.3 的更新 NSP。",
"check.update_not_totk": "更新文件不是《王国之泪》的更新({detail})。",
"check.exefs_failed": "{input}文件的游戏代码未通过检查;详见 {log}。",
@@ -101,6 +106,8 @@
"warn.xbox_prompts_failed": "无法准备 Xbox 按键提示(退出代码 {code});游戏将继续使用 Switch 按键提示。详情:{log}",
"warn.shaders_skipped": "显卡驱动在编译部分着色器时崩溃,因此有 {count} 个着色器未被预先准备。游戏会在首次需要时编译它们,驱动在那时也可能崩溃;更新驱动可能会解决。详情:{log}",
"warn.kernel_old": "Linux {version} 缺少游戏依赖的写入跟踪功能(需要 6.7 或更新版本);游戏可能运行得很慢。更新系统即可解决。",
"warn.dlss_kept": "安装文件夹中已有 DLSS 库({path});安装程序将保留它。",
"warn.dlss_download_failed": "无法从 {url} 下载 NVIDIA DLSS:{detail}。游戏无需它也能运行;下次运行安装程序时会重试。",
"ui.title": "TotK 安装程序",
"ui.rebuild_title": "正在为你的模组重新构建游戏",
@@ -109,8 +116,8 @@
"ui.intro": "用你自己的游戏文件构建一个可以游玩的《王国之泪》文件夹。耗时较长,只需运行一次。",
"ui.notice": "此安装程序构建的文件夹包含游戏本身,仅供你个人使用;请勿分享或上传。",
"ui.keys_private": "密钥文件只在原位置读取,其内容绝不会被显示、复制或保存。",
"ui.base": "游戏本体(NSP 或 XCI)",
"ui.update": "1.1.0、1.2.1 或 1.4.3 更新(NSP)",
"ui.base": "游戏本体(NSP、XCI、NSZ 或 XCZ)",
"ui.update": "1.1.0、1.2.1 或 1.4.3 更新(NSP 或 NSZ)",
"ui.keys": "prod.keys",
"ui.firmware": "固件文件夹(可选)",
"ui.game_language": "游戏语言",
@@ -122,8 +129,10 @@
"ui.pick_firmware": "选择 Switch 固件文件夹(Contents/registered)",
"ui.pick_dest": "选择安装文件夹",
"ui.firmware_help": "用于中文、韩文和繁体中文文本:固件的 Contents/registered 文件夹,例如来自 Ryujinx 或 Yuzu。安装程序会从中把 Switch 系统字体复制到安装文件夹。",
"ui.type_game": "游戏文件(NSP、XCI)",
"ui.type_nsp": "NSP 文件",
"ui.dlss": "为 NVIDIA RTX 显卡下载 NVIDIA DLSS(nvngx_dlss.dll)",
"ui.dlss_help": "来自 GitHub 上 NVIDIA 的 DLSS SDK,遵循 NVIDIA RTX SDK 许可(github.com/NVIDIA/DLSS)。没有它,dlaa 和 DLSS 设置不起作用。",
"ui.compressed_help": "压缩的 NSZ 和 XCZ 文件可以直接使用:安装程序会自行解压(在游戏数据提取完成之前,需要与解压后文件大小相同的可用空间)。",
"ui.type_game": "游戏文件(NSP、XCI、NSZ、XCZ)",
"ui.type_keys": "密钥文件(*.keys)",
"ui.type_all": "所有文件",
"ui.start": "开始",
+77 -1
View File
@@ -3,6 +3,7 @@ with a newer release, totk.ini and the event stream. No game files, no tools; st
Run: python tests/installer_test.py
"""
import http.client
import io
import json
import re
@@ -11,6 +12,7 @@ import tempfile
import unittest
from pathlib import Path
from types import SimpleNamespace
from unittest import mock
ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(ROOT))
@@ -481,7 +483,7 @@ class FontsTest(unittest.TestCase):
def test_all_steps_order(self):
self.assertEqual([s.id for s in steps.all_steps()],
['check', 'fonts', 'extract', 'lift', 'compile', 'prepare', 'cleanup'])
['check', 'fonts', 'extract', 'lift', 'compile', 'prepare', 'dlss', 'cleanup'])
def test_extracts_into_the_install_folder(self):
self.fake_tool()
@@ -575,5 +577,79 @@ class EventsTest(unittest.TestCase):
self.assertEqual(sorted(i for i in used if i not in catalog), [])
@unittest.skipUnless(steps.os.name == 'nt', 'DLSS is Windows-only')
class DlssTest(unittest.TestCase):
"""The DLSS step: off without --dlss, needs an RTX GPU, keeps a player's library, verifies the download."""
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
base = Path(self.tmp.name)
self.dest, temp = base / 'dest', base / 'temp'
self.dest.mkdir()
temp.mkdir()
self.src = base / 'nvngx_dlss.dll'
self.src.write_bytes(b'dlss' * 1000)
self.saved = steps.DLSS_URL, steps.DLSS_SIZE, steps.DLSS_SHA256
steps.DLSS_URL = self.src.as_uri() # urllib reads file: URLs as it reads https: ones
steps.DLSS_SIZE, steps.DLSS_SHA256 = 4000, steps.file_sha256(self.src)
rtx = {'devices': [{'name': 'NVIDIA GeForce RTX 4080 SUPER', 'api': [1, 4, 0], 'type': 2}]}
self.ctx = SimpleNamespace(dest=self.dest, temp=temp, host=rtx, reporter=Reporter(out=io.StringIO()),
options=SimpleNamespace(dlss=True))
def tearDown(self):
steps.DLSS_URL, steps.DLSS_SIZE, steps.DLSS_SHA256 = self.saved
self.ctx.reporter.close()
self.tmp.cleanup()
def warnings(self):
return [json.loads(line)['id'] for line in self.ctx.reporter.out.getvalue().splitlines()
if json.loads(line)['type'] == 'warning']
def test_off_without_the_option(self):
self.ctx.options.dlss = False
self.assertEqual(steps.Dlss().run(self.ctx), {})
self.assertFalse((self.dest / 'nvngx_dlss.dll').exists())
self.assertNotEqual(steps.Dlss().key(self.ctx), steps.Dlss().key(SimpleNamespace(options=SimpleNamespace(dlss=True))))
def test_downloads_and_checks_the_library(self):
rec = steps.Dlss().run(self.ctx)
self.assertEqual(rec, {'installed': steps.DLSS_VERSION})
self.assertEqual((self.dest / 'nvngx_dlss.dll').read_bytes(), self.src.read_bytes())
self.assertFalse(list(self.ctx.temp.iterdir()))
self.assertTrue(steps.Dlss().current(self.ctx, rec))
(self.dest / 'nvngx_dlss.dll').unlink()
self.assertFalse(steps.Dlss().current(self.ctx, rec))
def test_a_wrong_file_is_not_installed_and_tried_again(self):
steps.DLSS_SHA256 = '0' * 64
rec = steps.Dlss().run(self.ctx)
self.assertEqual(rec, {'failed': True})
self.assertFalse((self.dest / 'nvngx_dlss.dll').exists())
self.assertEqual(self.warnings(), ['warn.dlss_download_failed'])
self.assertFalse(steps.Dlss().current(self.ctx, rec))
def test_a_cut_connection_warns(self):
class Cut(io.BytesIO): # a response whose connection drops mid-file
def read(self, n=-1):
raise http.client.IncompleteRead(b'dl', 3998)
with mock.patch.object(steps.urllib.request, 'urlopen', return_value=Cut()):
self.assertEqual(steps.Dlss().run(self.ctx), {'failed': True})
self.assertFalse((self.dest / 'nvngx_dlss.dll').exists())
self.assertFalse(list(self.ctx.temp.iterdir()))
self.assertEqual(self.warnings(), ['warn.dlss_download_failed'])
def test_keeps_the_players_library(self):
(self.dest / 'nvngx_dlss.dll').write_bytes(b'newer')
self.assertEqual(steps.Dlss().run(self.ctx), {'kept': True})
self.assertEqual((self.dest / 'nvngx_dlss.dll').read_bytes(), b'newer')
self.assertEqual(self.warnings(), ['warn.dlss_kept'])
def test_skipped_without_an_rtx_gpu(self):
self.ctx.host = {'devices': [{'name': 'AMD Radeon RX 7800 XT', 'api': [1, 3, 0], 'type': 2}]}
self.assertEqual(steps.Dlss().run(self.ctx), {'skipped': 'no_rtx'})
self.assertEqual(self.warnings(), [])
if __name__ == '__main__':
unittest.main()
+173
View File
@@ -0,0 +1,173 @@
"""tools/ncz.py and the installer's unpacking of NSZ input (installer/steps.py, Check.unpack) on small made-up files:
an NCA with an AES-CTR section and a plain one, stored as an NCZ with one zstd stream or as NCZBLOCK blocks.
Run: .venv-py314\\Scripts\\python tests/ncz_test.py
"""
import hashlib
import io
import os
import struct
import sys
import tempfile
import unittest
from compression import zstd
from pathlib import Path
from types import SimpleNamespace
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(ROOT))
sys.path.insert(0, str(ROOT / 'tools'))
import ncz # noqa: E402
from installer import steps # noqa: E402
from installer.events import Reporter # noqa: E402
KEY, COUNTER = bytes(range(16)), bytes(range(16, 32))
# (offset, size, crypto type): an AES-CTR section, then a plain one
SECTIONS = ((0x4000, 0x9000, 3), (0xD000, 0x3200, 1))
def pfs0(files):
"""An NSP of (name, data) pairs."""
strings = b''.join(name.encode() + b'\0' for name, _ in files)
strings += bytes(-(16 + 0x18 * len(files) + len(strings)) % 0x20)
head = b'PFS0' + struct.pack('<III', len(files), len(strings), 0)
offset = name_at = 0
for name, data in files:
head += struct.pack('<QQII', offset, len(data), name_at, 0)
offset += len(data)
name_at += len(name) + 1
return head + strings + b''.join(data for _, data in files)
def made_up_nca():
"""(the NCA as stored on the console, its body after the header with the sections decrypted)."""
rand = os.urandom
plain = rand(0x2000) + bytes(0x7000) # half random, half compressible
rest = bytes(0x1200) + rand(0x2000)
counter = COUNTER[:8] + struct.pack('>Q', 0x4000 >> 4)
secret = Cipher(algorithms.AES(KEY), modes.CTR(counter)).encryptor().update(plain)
header = rand(0x4000)
return header + secret + rest, plain + rest
def ncz_file(nca, body, blocks=False):
out = nca[:0x4000] + b'NCZSECTN' + struct.pack('<Q', len(SECTIONS))
for offset, size, crypto in SECTIONS:
out += struct.pack('<QQQQ', offset, size, crypto, 0) + KEY + COUNTER
if not blocks:
return out + zstd.compress(body)
exponent = 14
pieces = [body[i:i + (1 << exponent)] for i in range(0, len(body), 1 << exponent)]
stored = [zstd.compress(p) if p.count(0) > len(p) // 2 else p for p in pieces] # random blocks stay as they are
out += b'NCZBLOCK' + bytes([2, 1, 0, exponent]) + struct.pack('<IQ', len(pieces), len(body))
out += struct.pack(f'<{len(stored)}I', *map(len, stored))
return out + b''.join(stored)
class UnpackTest(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.dir = Path(self.tmp.name)
self.nca, self.body = made_up_nca()
self.name = hashlib.sha256(self.nca).hexdigest()[:32]
self.tik = b'ticket' * 40
def tearDown(self):
self.tmp.cleanup()
def nsz(self, blocks=False, damage=None):
data = ncz_file(self.nca, self.body, blocks)
if damage:
data = damage(bytearray(data))
path = self.dir / 'game.nsz'
path.write_bytes(pfs0([(self.name + '.ncz', bytes(data)), ('a.tik', self.tik)]))
return path
def expected(self):
return pfs0([(self.name + '.nca', self.nca), ('a.tik', self.tik)])
def test_stream_comes_back_byte_for_byte(self):
src = self.nsz()
self.assertEqual(ncz.compressed(src)[0], 'nsz')
self.assertEqual(ncz.unpacked_size(src), len(self.expected()))
seen = []
self.assertEqual(ncz.unpack(src, self.dir / 'out.nsp', lambda d, t: seen.append((d, t))), 'nsz')
self.assertEqual((self.dir / 'out.nsp').read_bytes(), self.expected())
self.assertEqual(seen[-1], (len(self.expected()), len(self.expected())))
self.assertFalse((self.dir / 'out.nsp.part').exists())
def test_blocks_come_back_byte_for_byte(self):
ncz.unpack(self.nsz(blocks=True), self.dir / 'out.nsp')
self.assertEqual((self.dir / 'out.nsp').read_bytes(), self.expected())
def test_plain_nsp_is_not_compressed(self):
path = self.dir / 'plain.nsp'
path.write_bytes(self.expected())
self.assertIsNone(ncz.compressed(path))
def test_changed_byte_fails_the_hash(self):
def flip(data):
data[0x4000 + 16 + 0x40 * len(SECTIONS) + 0x100] ^= 1 # inside a stored (random) block
return data
with self.assertRaises(ncz.DamagedFile) as e:
ncz.unpack(self.nsz(blocks=True, damage=flip), self.dir / 'out.nsp')
self.assertIn('hash mismatch', e.exception.detail)
def test_cut_stream_is_damaged(self):
with self.assertRaises(ValueError):
ncz.unpack(self.nsz(damage=lambda d: d[:-40]), self.dir / 'out.nsp')
self.assertFalse((self.dir / 'out.nsp').exists())
class CheckUnpackTest(unittest.TestCase):
"""Check.unpack: an NSZ base becomes <temp>/unpacked/base.nsp, a plain NSP update is read in place, a second run
reuses the NSP."""
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
base = Path(self.tmp.name)
nca, body = made_up_nca()
name = hashlib.sha256(nca).hexdigest()[:32]
self.base, self.update = base / 'base.nsz', base / 'update.nsp'
self.base.write_bytes(pfs0([(name + '.ncz', ncz_file(nca, body))]))
self.update.write_bytes(pfs0([('a.tik', b'x' * 64)]))
self.expected = pfs0([(name + '.nca', nca)])
self.out = io.StringIO()
self.ctx = SimpleNamespace(options=SimpleNamespace(base=str(self.base), update=str(self.update)),
temp=base / 'temp', reporter=Reporter(out=self.out, log_path=base / 'x.log'))
def tearDown(self):
self.ctx.reporter.close()
self.tmp.cleanup()
def test_unpacks_the_base_and_reuses_it(self):
steps.Check().unpack(self.ctx)
target = self.ctx.temp / 'unpacked' / 'base.nsp'
self.assertEqual(target.read_bytes(), self.expected)
self.assertEqual(steps.game_file(self.ctx, 'base'), target)
self.assertEqual(steps.game_file(self.ctx, 'update'), self.update)
self.assertIn('"phase": "unpack_base"', self.out.getvalue())
mtime = target.stat().st_mtime_ns
steps.Check().unpack(self.ctx)
self.assertEqual(target.stat().st_mtime_ns, mtime)
def test_content_decides_not_the_name(self):
renamed = self.base.with_name('base.nsp')
self.base.rename(renamed)
self.ctx.options.base = str(renamed)
steps.Check().unpack(self.ctx)
self.assertEqual((self.ctx.temp / 'unpacked' / 'base.nsp').read_bytes(), self.expected)
def test_damaged_base_names_the_chosen_file(self):
data = bytearray(self.base.read_bytes())
data[-100] ^= 0xFF
self.base.write_bytes(bytes(data))
with self.assertRaises(steps.InstallError) as e:
steps.Check().unpack(self.ctx)
self.assertIn(e.exception.id, ('check.input_damaged', 'check.unpack_failed'))
if __name__ == '__main__':
unittest.main()
+4 -3
View File
@@ -105,9 +105,10 @@ LLVM_SKIP_DIRS = {'aarch64-w64-mingw32', 'armv7-w64-mingw32', 'i686-w64-mingw32'
LLVM_SKIP_SUFFIXES = {'.idl', '.tlb'}
# tools/ files the installer steps run (and what they import), copied from the repo.
TOOLS_FILES = ['build_game.py', 'build_shaders.py', 'button_icons.py', 'check_content.py', 'check_isa.py', 'extract.py', 'hostmem.py',
'make_romfs_patches.py', 'pack_shader_cache.py', 'prepare_shaders.py', 'romfs_index.py', 'shader_census.py',
'shader_list.py', 'ultrawide_layout.py', 'cheats.py', 'exefs_patches.py']
TOOLS_FILES = ['build_game.py', 'build_shaders.py', 'button_icons.py', 'check_content.py', 'check_isa.py', 'extract.py',
'hostmem.py', 'make_romfs_patches.py', 'ncz.py', 'pack_shader_cache.py', 'prepare_shaders.py',
'romfs_index.py', 'shader_census.py', 'shader_list.py', 'ultrawide_layout.py', 'cheats.py',
'exefs_patches.py']
TOOLS_DIRS = ['vendor/macnativegames', 'data'] # data/: expected hashes
RUNTIME_SUFFIXES = {'.h', '.inc', '.ico'} # .ico: the installer window icon
+231
View File
@@ -0,0 +1,231 @@
"""Unpack an NSZ or XCZ (a game file compressed by the nsz tool) into a plain NSP.
An NSZ is an NSP whose NCAs are stored as NCZ files: the first 0x4000 bytes of the NCA (its encrypted header) as they
are, then an NCZSECTN table of the NCA's sections with the AES-CTR key and counter of each, then the rest of the NCA
decrypted and compressed with zstd, either as one stream or as an NCZBLOCK table of independently compressed blocks.
An XCZ is an XCI whose secure partition holds NCZ files. Unpacking encrypts the sections again, so every NCA comes back
byte for byte; its file name is the first half of its SHA-256, which is checked. The output is an NSP of the secure
partition (an XCZ) or of the NSZ's files, with .ncz renamed to .nca. Format: nicoboss/nsz, nsz/NszDecompressor.py
(__decompressNcz) and nsz/BlockDecompressorReader.py; this is our own code.
usage: ncz.py <file.nsz|file.xcz> <out.nsp>. Runs under .venv-py314 (compression.zstd).
"""
import hashlib
import io
import struct
import sys
from compression import zstd
from pathlib import Path
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
sys.path.insert(0, str(Path(__file__).resolve().parent / 'vendor' / 'macnativegames'))
from extract_exefs import container_entries, read_at, require # noqa: E402
HEADER = 0x4000 # the NCA header, kept encrypted and uncompressed
CHUNK = 1 << 20
CRYPTO_CTR = (3, 4) # AES-CTR and BKTR sections; the other types are stored as they are
class DamagedFile(ValueError):
"""An NCZ whose unpacked NCA does not match its name's hash, or whose data ends early."""
def __init__(self, path, detail):
self.path, self.detail = Path(path), detail
super().__init__(f'{self.path} is damaged or incomplete ({detail})')
def compressed(path):
"""('nsz' or 'xcz', entries) for a game file holding NCZ files, else None. Raises ValueError for a file that is
neither an NSP nor an XCI."""
with Path(path).open('rb') as f:
kind, entries = container_entries(f, Path(path).stat().st_size)
if not any(name.lower().endswith('.ncz') for name, _, _ in entries):
return None
return ('nsz' if kind == 'nsp' else 'xcz'), entries
def sections(f, start):
"""[(offset, size, crypto type, key, counter)] of the NCZ at start, and the offset of its data after the table."""
head = read_at(f, start + HEADER, 16)
require(head[:8] == b'NCZSECTN', 'NCZ without its NCZSECTN table')
count = struct.unpack_from('<Q', head, 8)[0]
require(0 < count <= 0x10000, 'Invalid NCZ section count')
raw = read_at(f, start + HEADER + 16, count * 0x40)
out = []
for i in range(count):
offset, size, crypto = struct.unpack_from('<QQQ', raw, i * 0x40)
out.append((offset, size, crypto, raw[i * 0x40 + 0x20:i * 0x40 + 0x30], raw[i * 0x40 + 0x30:i * 0x40 + 0x40]))
out.sort()
require(out[0][0] >= HEADER, 'NCZ section inside the NCA header')
for (a, size, *_), (b, *_) in zip(out, out[1:]):
require(a + size <= b, 'Overlapping NCZ sections')
return out, start + HEADER + 16 + count * 0x40
def nca_size(f, start):
table, _ = sections(f, start)
return table[-1][0] + table[-1][1]
def layout(source):
"""(entries, NSP header, NSP size) of the NSP that unpack writes for the NSZ or XCZ source: its files in order,
.ncz renamed to .nca, the PFS0 table padded to 0x20 bytes."""
_, entries = compressed(source)
with Path(source).open('rb') as f:
sizes = [nca_size(f, start) if name.lower().endswith('.ncz') else length for name, start, length in entries]
names = [name[:-4] + '.nca' if name.lower().endswith('.ncz') else name for name, _, _ in entries]
strings = b''.join(n.encode('utf-8') + b'\0' for n in names)
strings += bytes(-(16 + 0x18 * len(names) + len(strings)) % 0x20)
head = b'PFS0' + struct.pack('<III', len(names), len(strings), 0)
offset, name_at = 0, 0
for name, size in zip(names, sizes):
head += struct.pack('<QQII', offset, size, name_at, 0)
offset += size
name_at += len(name.encode('utf-8')) + 1
head += strings
return entries, head, len(head) + offset
def unpacked_size(source):
"""Bytes of the NSP that unpack writes for the NSZ or XCZ source."""
return layout(source)[2]
class Slice(io.RawIOBase):
"""Bytes start..end of a file, for zstd.ZstdFile: a stream may hold several frames and must stop at the entry's
end."""
def __init__(self, f, start, end):
self.f, self.pos, self.end = f, start, end
def readable(self):
return True
def readinto(self, buffer):
n = min(len(buffer), self.end - self.pos)
if n <= 0:
return 0
self.f.seek(self.pos)
data = self.f.read(n)
buffer[:len(data)] = data
self.pos += len(data)
return len(data)
def blocks(f, pos, end, total):
"""The NCZBLOCK data at pos, decompressed, in pieces. A block whose stored size is not smaller than its
decompressed size is stored as it is."""
head = read_at(f, pos, 24)
exponent, count = head[11], struct.unpack_from('<I', head, 12)[0]
size = struct.unpack_from('<Q', head, 16)[0]
require(head[8] == 2 and head[9] == 1 and 14 <= exponent <= 32, 'Unsupported NCZBLOCK header')
require(size == total and count == -(-size // (1 << exponent)), 'NCZBLOCK sizes do not match the sections')
stored = struct.unpack_from(f'<{count}I', read_at(f, pos + 24, 4 * count))
pos += 24 + 4 * count
for i, length in enumerate(stored):
plain = min(1 << exponent, size - (i << exponent))
require(pos + length <= end, 'Truncated NCZ block')
data = read_at(f, pos, length)
pos += length
yield zstd.decompress(data) if length < plain else data
def stream(f, pos, end, total):
"""The decompressed NCA body after the header, in pieces of up to CHUNK bytes."""
if read_at(f, pos, 8) == b'NCZBLOCK':
yield from blocks(f, pos, end, total)
return
with zstd.ZstdFile(io.BufferedReader(Slice(f, pos, end), CHUNK)) as z:
while data := z.read(CHUNK):
yield data
def unpack_ncz(f, start, end, out, name, source, progress):
"""Write the NCA of the NCZ at start..end to out; check it against name's hash. progress(bytes) after each piece."""
table, pos = sections(f, start)
total = table[-1][0] + table[-1][1]
digest = hashlib.sha256()
header = read_at(f, start, HEADER)
out.write(header)
digest.update(header)
progress(HEADER)
at = HEADER
pending = b''
try:
pieces = stream(f, pos, end, total - HEADER)
for piece in pieces:
pending += piece
while pending and at < total:
section = next((s for s in table if s[0] <= at < s[0] + s[1]), None)
stop = section[0] + section[1] if section else next((s[0] for s in table if s[0] > at), total)
n = min(len(pending), stop - at)
crypto = section and section[2] in CRYPTO_CTR
if crypto and at + n < stop:
n -= n % 16 # the counter of the next piece starts at a whole AES block
if not n:
break
data, pending = pending[:n], pending[n:]
if crypto:
require(at % 16 == 0, 'Unaligned NCZ section')
counter = section[4][:8] + struct.pack('>Q', at >> 4)
data = Cipher(algorithms.AES(section[3]), modes.CTR(counter)).encryptor().update(data)
out.write(data)
digest.update(data)
at += n
progress(n)
if at == total:
break
except (zstd.ZstdError, EOFError) as e: # EOFError: the stream ends before its end-of-frame marker
raise DamagedFile(source, f'{name}: {e}') from None
if at != total or pending:
raise DamagedFile(source, f'{name}: {at - HEADER:,} of {total - HEADER:,} bytes')
stem = name[:-4]
if len(stem) == 32 and digest.hexdigest()[:32] != stem.lower():
raise DamagedFile(source, f'{name}: hash mismatch')
def unpack(source, target, progress=lambda done, total: None):
"""Write the NSP of the NSZ or XCZ source to target (through target.part). progress(done, total) in bytes."""
source, target = Path(source), Path(target)
kind = compressed(source)[0]
entries, head, total = layout(source)
part = target.with_name(target.name + '.part')
done = 0
def step(n):
nonlocal done
done += n
progress(done, total)
with source.open('rb') as f, part.open('wb') as out:
out.write(head)
step(len(head))
for name, start, length in entries:
if name.lower().endswith('.ncz'):
unpack_ncz(f, start, start + length, out, name, source, step)
continue
for at in range(start, start + length, CHUNK):
data = read_at(f, at, min(CHUNK, start + length - at))
out.write(data)
step(len(data))
part.replace(target)
return kind
def main():
source, target = sys.argv[1:3]
shown = [-1]
def progress(done, total):
percent = done * 100 // total
if percent != shown[0]:
shown[0] = percent
print(f'\r{percent}%', end='', flush=True)
kind = unpack(source, target, progress)
print(f'\n{kind}: wrote {target}')
if __name__ == '__main__':
main()