diff --git a/AGENTS.md b/AGENTS.md index 4792a3ef9b..6126d4887f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -84,7 +84,7 @@ An **asset** is any non-code material that the project includes, ships, embeds, ### 1.2 Only open, provable licences Every asset must be one of: -- **Open source:** ISC, MIT, BSD-2-Clause, BSD-3-Clause, Apache-2.0, OFL-1.1, Ubuntu-Font-1.0 or Bitstream-Vera. +- **Open source:** ISC, MIT, BSD-2-Clause, BSD-3-Clause, Apache-2.0, OFL-1.1, Ubuntu-Font-1.0, Bitstream-Vera or Unicode-3.0 (Unicode, Inc. data such as ICU4X's line-breaking dictionaries; already allowed for code in `deny.toml`). - **Creative Commons:** CC0-1.0, CC-BY-4.0 or CC-BY-SA-4.0. - **Public domain.** - **Contributor-original:** created, photographed or generated by a contributor who owns it and licenses it under the project licence (`MIT OR Apache-2.0`). @@ -150,7 +150,7 @@ If you find an asset that breaks these rules, stop and remove it from the reposi ## 3. Agent control (automation and MCP) - **Everything is reachable headlessly.** Every user-facing feature must be callable without the GUI: an engine API, then a tool in `crates/automation` (`pdfcraft-cli run` and MCP use the same table). A feature is not done until its tool exists and has an end-to-end test (`crates/automation/README.md`, "Adding a tool"). -- **The UI control channel is opt-in too.** It runs only when the app is launched with `--control `. It binds to loopback, uses a fresh random token per launch, and writes it to an owner-only file. Never enable it by default, widen the bind address, or log the token. +- **The UI control channel is opt-in too.** It runs only when the app is launched with `--control `. It binds to loopback, uses a fresh random token per launch, and writes it to an owner-only file (mode 0600 on Unix; on Windows an access list of the user and SYSTEM only, which `pdfcraft_platform::owner_only` sets and checks, and which the CLI checks again before reading the token). Never enable it by default, widen the bind address, or log the token. - **The MCP server is opt-in.** Never make the app, the CLI or an installer start it, register it with an agent, or listen on a port by default. It runs only when a user (or an agent the user configured) launches `pdfcraft-cli mcp`. Any future network transport must be off by default, bound to loopback and authenticated with a token. - **Safe by default.** Tools must never discard unsaved work silently, must write atomically, must honour `--root`, and must return errors an agent can act on (say what was wrong and what is allowed). - **Track parity honestly.** Every user-facing feature has an entry in `parity/acrobat-features.toml`. Mark it `shipped` only when it works end to end and the entry cites a passing test (`evidence`); `cargo xtask parity` (part of `xtask ci`) rejects claims whose commands, tools or tests don't exist. diff --git a/ATTRIBUTION.md b/ATTRIBUTION.md index a8de85e07a..cfc16f78f4 100644 --- a/ATTRIBUTION.md +++ b/ATTRIBUTION.md @@ -267,7 +267,7 @@ Every asset PdfCraft includes, bundles or uses to build its published material, | `assets/icons/a-large-small.svg` | Lucide icon "a-large-small" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/a-large-small.svg) | UI icon, embedded via crates/ui-egui/src/widgets/icon_data.rs | | `assets/icons/move-horizontal.svg` | Lucide icon "move-horizontal" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/move-horizontal.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs | -## Compiled in through dependencies (6) +## Compiled in through dependencies (7) | Crate | File | Title | Author | Licence | Source | Used for | |---|---|---|---|---|---|---| @@ -277,6 +277,7 @@ Every asset PdfCraft includes, bundles or uses to build its published material, | `epaint_default_fonts` 0.37.0 | `fonts/emoji-icon-font.ttf` | emoji-icon-font | John Slegers | MIT | https://github.com/jslegers/emoji-icon-font | egui monochrome fallback font (egui feature monochrome_emoji_fonts) | | `epaint_default_fonts` 0.37.0 | `fonts/egui-icons.ttf` | egui icons (monochrome emoji-icon-font subset) | John Slegers | MIT | https://github.com/jslegers/emoji-icon-font | Small special-icon subset embedded by egui default_fonts; retained in FontDefinitions::default(). Provenance: https://github.com/storytold/egui/blob/6d518fc799681571963842f95b55a6b68431b15a/crates/epaint_default_fonts/fonts/egui-icons.txt; complete MIT copyright and permission text in NOTICE. | | `hayro-interpret` 0.7.0 | `src/font/generated/metrics.rs` | Standard-14 font metrics and encodings (as Rust tables) (non-visual data, AGENTS.md §1.1) | Adobe (Core 14 AFM metrics, PDF specification encodings); tables by The Hayro Authors | MIT OR Apache-2.0 | ISO 32000-2 Annex D; Adobe Core14 AFM files (via vendored hayro-interpret) | Glyph widths and encodings for non-embedded standard fonts. Non-visual data, allowed by AGENTS.md §1.1 | +| `icu_segmenter_data` 2.3.0 | `data/segmenter_dictionary_extended_v1.rs.data` | Southeast Asian word dictionaries for line breaking (Thai, Lao, Khmer, Burmese) | The ICU4X Project Developers; word lists from the ICU project (Unicode, Inc.) | Unicode-3.0 | https://github.com/unicode-org/icu4x (via icu_segmenter_data) | Word boundaries for wrapping Thai text in Edit text and Add text (crates/edit/src/linebreak.rs). Non-visual data; the line segmenter loads only these dictionaries, not the Chinese and Japanese one | ## Downloaded at build time (22) diff --git a/ATTRIBUTION.toml b/ATTRIBUTION.toml index 608bb601a5..be935422ce 100644 --- a/ATTRIBUTION.toml +++ b/ATTRIBUTION.toml @@ -2263,6 +2263,18 @@ adobe_data = true usage = "Glyph widths and encodings for non-embedded standard fonts. Non-visual data, allowed by AGENTS.md §1.1" sha256 = "2ec9e608573eda459de1e1d9bda3bb494315e36bee885acc88912a11fe0caa41" +[[bundled]] +crate = "icu_segmenter_data" +version = "2.3.0" +path = "data/segmenter_dictionary_extended_v1.rs.data" +title = "Southeast Asian word dictionaries for line breaking (Thai, Lao, Khmer, Burmese)" +author = "The ICU4X Project Developers; word lists from the ICU project (Unicode, Inc.)" +source = "https://github.com/unicode-org/icu4x (via icu_segmenter_data)" +licence = "Unicode-3.0" +kind = "data" +usage = "Word boundaries for wrapping Thai text in Edit text and Add text (crates/edit/src/linebreak.rs). Non-visual data; the line segmenter loads only these dictionaries, not the Chinese and Japanese one" +sha256 = "cad3821ca44fdb88ed856ff3336444aa200afe7b9d19e34ac28ae6a4459d4c6d" + [[fetched]] file = "PlayfairDisplay[wght].ttf" url = "https://raw.githubusercontent.com/google/fonts/9710da1eacb3be272583c3224dcb70f9da6eadbb/ofl/playfairdisplay/PlayfairDisplay%5Bwght%5D.ttf" @@ -3232,4 +3244,3 @@ licence_file = "assets/icons/LICENSE-lucide.txt" kind = "icon" usage = "UI icon, embedded via crates/ui-egui/src/icon_data.rs" sha256 = "864ab2f19254e6ac7240bc6a98323197332e8f714d4d3f94831f41309e5ef2f7" - diff --git a/Cargo.lock b/Cargo.lock index 89cba1d591..fafd037d33 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1289,7 +1289,7 @@ dependencies = [ [[package]] name = "craft-update" version = "0.1.0" -source = "git+https://github.com/storytold/craft-libs?rev=787a7a0f972fd1d7a7618fdbeca3a5267a0407ff#787a7a0f972fd1d7a7618fdbeca3a5267a0407ff" +source = "git+https://github.com/storytold/craft-libs?rev=1b34eba41e62138fc2438d2734523b302840b45d#1b34eba41e62138fc2438d2734523b302840b45d" dependencies = [ "chrono", "dunce", @@ -3133,6 +3133,28 @@ dependencies = [ "zerovec", ] +[[package]] +name = "icu_segmenter" +version = "2.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "82d07aafccd67af15d02512a6adf5896fbc5ed00f2e99b471d2efa14016db3db" +dependencies = [ + "icu_collections", + "icu_locale_fallback", + "icu_provider", + "icu_segmenter_data", + "potential_utf", + "smallvec", + "utf8_iter", + "zerovec", +] + +[[package]] +name = "icu_segmenter_data" +version = "2.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae293c039020f9ec10710af98d29ce6aa2051486638b49c9a6409f3b4a9e98ad" + [[package]] name = "idna" version = "1.1.0" @@ -4246,6 +4268,7 @@ dependencies = [ "fmv-macos-events", "log", "pdfcraft-engine", + "pdfcraft-platform", "pdfcraft-ui-egui", "pollster 1.0.1", "raw-window-handle", @@ -4400,6 +4423,7 @@ name = "pdfcraft-edit" version = "0.18.1" dependencies = [ "hayro-syntax", + "icu_segmenter", "kurbo 0.13.1", "pdfcraft-annot", "pdfcraft-content", @@ -4569,6 +4593,7 @@ dependencies = [ "pdfcraft-render", "proptest", "thiserror 2.0.21", + "thread-priority", "zune-jpeg", ] @@ -4656,8 +4681,10 @@ dependencies = [ "pdfcraft-testkit", "pdfcraft-xfa", "regex", + "roxmltree 0.21.2", "sha2 0.11.1", "thiserror 2.0.21", + "unicode-normalization", ] [[package]] @@ -4672,12 +4699,16 @@ dependencies = [ "kurbo 0.13.1", "log", "lopdf", + "pdfcraft-content", "pdfcraft-cos", "pdfcraft-fonts", "pdfcraft-geom", + "pdfcraft-testkit", "serde", "skrifa 0.44.0", "thiserror 2.0.21", + "unicode-bidi", + "url", ] [[package]] @@ -4741,6 +4772,7 @@ dependencies = [ "egui", "egui_extras", "egui_kittest", + "flate2", "fontconfig-parser", "getrandom 0.4.3", "image", @@ -4811,10 +4843,12 @@ dependencies = [ name = "pdfcraft-xfdf" version = "0.18.1" dependencies = [ + "base64 0.23.1", "pdfcraft-annot", "pdfcraft-cos", "pdfcraft-forms", "pdfcraft-model", + "proptest", "roxmltree 0.21.2", "thiserror 2.0.21", ] @@ -6932,6 +6966,20 @@ dependencies = [ "syn 3.0.6", ] +[[package]] +name = "thread-priority" +version = "3.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8d2e834949be5111506bb252643498af1514f600d9e1dceedaa42afae155b67f" +dependencies = [ + "bitflags 2.13.2", + "cfg-if", + "libc", + "log", + "rustversion", + "windows", +] + [[package]] name = "tiff" version = "0.11.3" @@ -8724,7 +8772,6 @@ dependencies = [ "nix", "typed-path", "windows-sys 0.61.2", - "zopfli", ] [[package]] @@ -8739,18 +8786,6 @@ version = "1.0.23" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b" -[[package]] -name = "zopfli" -version = "0.8.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "aaf7fc5d30c28483d93805c4a5e12b05bbb52407fa67c5f8bd552374cd01fb11" -dependencies = [ - "bumpalo", - "crc32fast", - "log", - "simd-adler32", -] - [[package]] name = "zune-core" version = "0.5.3" diff --git a/Cargo.toml b/Cargo.toml index d919a90383..b0f3411624 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -28,7 +28,7 @@ craft-print = { git = "https://github.com/storytold/craft-libs", version = "0.1. # Whether a path stored in a document may be opened (GoToR link targets). craft-docpath = { version = "0.1.0", git = "https://github.com/storytold/craft-libs", rev = "787a7a0f972fd1d7a7618fdbeca3a5267a0407ff" } # craft-update: app-specific pretend versions apply to startup and manual checks. -craft-update = { git = "https://github.com/storytold/craft-libs", version = "=0.1.0", rev = "787a7a0f972fd1d7a7618fdbeca3a5267a0407ff", features = ["egui"] } +craft-update = { git = "https://github.com/storytold/craft-libs", version = "=0.1.0", rev = "1b34eba41e62138fc2438d2734523b302840b45d", features = ["egui"] } # internal pdfcraft-geom = { path = "crates/geom", version = "0.18.1" } pdfcraft-filters = { path = "crates/filters", version = "0.18.1" } diff --git a/LICENSES.toml b/LICENSES.toml index 0617ca5d57..ac54d9f702 100644 --- a/LICENSES.toml +++ b/LICENSES.toml @@ -16,7 +16,7 @@ sha256 = "cd7032ea33b71427f188e34ffb4aee9d6868a57fd196037c5c8b2be03a2a0352" [[document]] source = "NOTICE" destination = "NOTICE" -sha256 = "9f854f730719af962cd840529324d089e05e68a30bd245c50e33f96e009b4314" +sha256 = "d9ba162ad2bc95d22ab9885065e253bb823bd6795068828e7b39858344b660eb" [[document]] source = "assets/app-icon/LICENSE-document.txt" @@ -60,6 +60,13 @@ destination = "licenses/CMap-LICENSE.txt" sha256 = "742665db9c8e1bc72603c6d319ca3e90b83bd6d95202f0cd4ef11068a07a9c29" origin = "https://static.crates.io/crates/hayro-cmap/hayro-cmap-0.1.0.crate#assets/LICENSE.txt" +[[document]] +source = "licenses/ICU4X-LICENSE.txt" +licences = ["Unicode-3.0"] +destination = "licenses/ICU4X-LICENSE.txt" +sha256 = "f367c1b8e1aa262435251e442901da4607b4650e0e63a026f5044473ecfb90f2" +origin = "https://static.crates.io/crates/icu_segmenter_data/icu_segmenter_data-2.3.0.crate#LICENSE" + [[document]] source = "licenses/egui-Hack-Regular.txt" licences = ["MIT", "Bitstream-Vera"] @@ -164,3 +171,9 @@ crate = "hayro-interpret" path = "src/font/generated/metrics.rs" licence = "MIT OR Apache-2.0" documents = ["vendor/hayro-interpret/LICENSE-MIT", "vendor/hayro-interpret/LICENSE-APACHE"] + +[[bundled]] +crate = "icu_segmenter_data" +path = "data/segmenter_dictionary_extended_v1.rs.data" +licence = "Unicode-3.0" +documents = ["licenses/ICU4X-LICENSE.txt"] diff --git a/NOTICE b/NOTICE index e8f31ade2f..eed2d654b0 100644 --- a/NOTICE +++ b/NOTICE @@ -36,6 +36,9 @@ list (author, source, license) is in ATTRIBUTION.md. - egui default fonts (epaint_default_fonts, via eframe) — Ubuntu Light (Ubuntu Font Licence 1.0), Hack (MIT / Bitstream Vera licence), Noto Emoji (SIL OFL 1.1), emoji-icon-font (MIT). - Colour profile in hayro-interpret — CGATS001Compat-v2-micro.icc (CC0-1.0, Clinton Ingram). +- ICU4X line-breaking data (icu_segmenter_data, via crates.io) — Unicode-3.0. Its Thai, Lao, Khmer + and Burmese word dictionaries are compiled in for wrapping Thai text; notice below, full licence in + licenses/ICU4X-LICENSE.txt. Required notices: @@ -48,6 +51,32 @@ Required notices: (https://github.com/adobe-type-tools/cmap-resources/blob/master/LICENSE.txt). Neither the name of Adobe nor the names of its contributors are used to endorse or promote this product. + ICU4X segmenter data (bundled by icu_segmenter_data 2.3.0; word lists, not visual assets): + Copyright © 2020-2024 Unicode, Inc. Portions adapted from ICU4C and/or ICU4J: + ICU 1.8.1 to ICU 57.1 © 1995-2016 International Business Machines Corporation and others. + + Permission is hereby granted, free of charge, to any person obtaining a copy of data files and any + associated documentation (the "Data Files") or software and any associated documentation (the + "Software") to deal in the Data Files or Software without restriction, including without + limitation the rights to use, copy, modify, merge, publish, distribute, and/or sell copies of the + Data Files or Software, and to permit persons to whom the Data Files or Software are furnished to + do so, provided that either (a) this copyright and permission notice appear with all copies of + the Data Files or Software, or (b) this copyright and permission notice appear in associated + Documentation. + + THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR + PURPOSE AND NONINFRINGEMENT OF THIRD PARTY RIGHTS. + + IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE BE LIABLE FOR ANY CLAIM, + OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF + USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, + ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THE DATA FILES OR SOFTWARE. + + Except as contained in this notice, the name of a copyright holder shall not be used in + advertising or otherwise to promote the sale, use or other dealings in these Data Files or + Software without prior written authorization of the copyright holder. + Showcase PDF (built by `cargo xtask demo-pdf`, not committed): fonts are downloaded at build time from google/fonts at a pinned commit, verified by SHA-256, all under the SIL Open Font License 1.1 — see the "Downloaded at build time" table in ATTRIBUTION.md. diff --git a/README.md b/README.md index f488d160cc..4b534f6e17 100644 --- a/README.md +++ b/README.md @@ -331,7 +331,7 @@ Edits stay in memory, undoable, until `doc_save`. Saving to the same file append Start the desktop app with `pdfcraft --control ~/.pdfcraft-control.json` and an agent can see and operate the real interface: the widget tree with labels and positions (from the accessibility tree), clicks, typing, keys, commands, view options and screenshots. This is also off by default. It listens only on loopback, and every connection must present the random token written to that file, which only you can read. -Keep the control file in a folder only you can write, not a shared one such as `/tmp`: another user could create the file there first and receive your commands. `pdfcraft-cli ui` refuses a control file that is a symbolic link, and on macOS, Linux and FreeBSD one that another user owns or can read or write. The app doesn't start if it can't write the file. +Keep the control file in a folder only you can write, not a shared one such as `/tmp`: another user could create the file there first and receive your commands. `pdfcraft-cli ui` refuses a control file that is a symbolic link, or one that another user owns, or that another user or a group such as Administrators can read or write: on macOS, Linux and FreeBSD by its owner and mode (the app writes it with mode 600), on Windows by its owner and access list (the app gives it one with only you and SYSTEM on it). The app doesn't start if it can't write the file. On Windows, cmd.exe doesn't expand `~`: use `%USERPROFILE%\.pdfcraft-control.json` there. ```sh pdfcraft-cli ui --control ~/.pdfcraft-control.json inspect query=rotate # find widgets @@ -472,11 +472,14 @@ Installers and executables are code-signed. The portable zip runs from any folder, a USB stick included. Its `portable.txt` keeps the settings, logs and crash recovery in a `PdfCraftData` folder next to `pdfcraft.exe`, so nothing is written to `%APPDATA%`; delete that file to use the normal per-user folders. +Windows doesn't offer the portable copy as a default PDF app on its own, since it registers nothing; +`portable.txt` has the PowerShell lines that register it for your user only, and how to undo them. The MSI installs for your account only by default, without administrator privileges. Choose "Everyone on this computer" on its first page, or pass `ALLUSERS=1` from an elevated terminal, to install for all users. It adds "Edit with PdfCraft" -to the right-click menu of PDF files and "Create PDF with PdfCraft…" to the right-click menu of +and "Combine files in PdfCraft" (select several PDFs: they land in the Combine files tab of the +running PdfCraft) to the right-click menu of PDF files and "Create PDF with PdfCraft…" to the right-click menu of images (.png .jpg .jpeg .tif .tiff .gif .bmp .jp2 .j2k .jpx) in File Explorer (on Windows 11, under "Show more options"), without changing which app opens them by default. For unattended deployment without a desktop shortcut (add `ALLUSERS=1`, from an elevated terminal, for an all-users install): @@ -488,7 +491,7 @@ msiexec /i "pdfcraft--windows-x64.msi" /qn /norestart INSTALLDESKTOPSHORTCU | Property | `=0` leaves out | |---|---| | `INSTALLDESKTOPSHORTCUT` | the desktop shortcut | -| `INSTALLPDFCONTEXTMENU` | "Edit with PdfCraft" on PDF files | +| `INSTALLPDFCONTEXTMENU` | "Edit with PdfCraft" and "Combine files in PdfCraft" on PDF files | | `INSTALLIMAGECONTEXTMENU` | "Create PDF with PdfCraft…" on images | All three are on for a first install. An upgrade keeps what the installed version has: a shortcut @@ -499,6 +502,12 @@ didn't have a menu yet (0.6.0 and earlier for the PDF menu, 0.2.1 and earlier fo adds it, as a first install would. Use the MSI for your architecture. Per-user installation overrides are not supported. +To install somewhere other than the default `ArtCraft\PdfCraft` folder, change the folder on the +welcome screen (type it, or Browse…) or pass it on the command line: +`INSTALLFOLDER="D:\Apps\PdfCraft"`. It is installed for whoever the first screen chose (`ALLUSERS=1` +on the command line for everyone). An upgrade keeps the folder the installed version uses unless +the command line gives another one. + **Pin the checksum, not just the version.** Release candidates of a version share its MSI version (MSI versions are numbers only, so 1.2.3-rc.1 and 1.2.3 are both 1.2.3), and the installer lets an MSI of the same version replace the installed one without asking. Two different `0.7.1` MSIs can't diff --git a/ROADMAP.md b/ROADMAP.md index 47344cb07a..92324ebccb 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -199,27 +199,28 @@ What's strong and missing in each: [target-app-parity.md](docs/target-app-parity ## Languages -Twelve key languages (historical % snapshot vs 2,143 keys, except Vietnamese, Brazilian -Portuguese and Spanish vs the current union). Detail and script notes: [localization-parity.md](docs/localization-parity.md). +Twelve key languages: the share of the 2,897 strings in the i18n report's denominator (the interface's strings plus every catalog key), from +`cargo xtask i18n-report` on 2026-11-03. Detail and script notes: [localization-parity.md](docs/localization-parity.md). | Language | Code | Strings | Status | |---|---|---|---| | English | `en` | 100% | full | -| Simplified Chinese | `zh-hans` | 97.4% | partial (native LXGW WenKai Lite; no Hans face in the web build, #826) | -| Spanish | `es` | 100% of current union | partial (dialogs / tooltips; no help) | -| Hindi | `hi` | 0% | none | -| Arabic | `ar` | 100% | partial (logical catalog, explicit RTL labels, mirrored stock chrome; native/browser/IME workflows unqualified) | -| French | `fr` | 99.3% | partial | -| Portuguese (Brazil) | `pt-br` | 100% of current union | partial | -| Indonesian | `id` | 0% | none | -| Japanese | `ja` | 99.1% | partial | -| German | `de` | 99.3% | partial | -| Korean | `ko` | 0% | none | -| Vietnamese | `vi` | 100% of current union | partial (dialogs / tooltips; no help) | +| Simplified Chinese | `zh-hans` | 88.4% | partial (native LXGW WenKai Lite; no Hans face in the web build, #826) | +| Spanish | `es` | 98.4% | partial (dialogs / tooltips; no help) | +| Hindi | `hi` | 97.5% | partial (Devanagari rendering unqualified) | +| Arabic | `ar` | 97.9% | partial (logical catalog, explicit RTL labels, mirrored stock chrome; native/browser/IME workflows unqualified) | +| French | `fr` | 98.4% | partial | +| Portuguese (Brazil) | `pt-br` | 98.4% | partial | +| Indonesian | `id` | 97.5% | partial | +| Japanese | `ja` | 89.6% | partial | +| German | `de` | 89.7% | partial | +| Korean | `ko` | 97.5% | partial (no Hangul UI face) | +| Vietnamese | `vi` | 98.4% | partial (dialogs / tooltips; no help) | Also shipped: Traditional Chinese, Russian, Bulgarian, Telugu, Hungarian, Ukrainian, Italian, -Kazakh, Polish, Slovenian, Greek, Turkish, Lithuanian, Romanian and Czech: **15 more catalogs** -(24 interface languages including English). +Kazakh, Polish, Slovenian, Greek, Turkish, Lithuanian, Finnish, Albanian, Catalan, Romanian, +Norwegian Bokmål, Persian, Central Kurdish, Thai, Lao and Czech: **23 more catalogs** +(35 interface languages including English). ## Upcoming @@ -240,6 +241,12 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md ## Progress log - **2026-10-13 (M8, redact by regex):** `redact_mark` takes `regex` (caller expression, linear-time engine, length/size/nesting caps, empty-matching refused) with optional `validate` (luhn, mod97, iban). Headless only; issue #1886. +- **2026-11-08 (M5, comment exchange):** Comments keep their formatting and appearance through XFDF and FDF. XFDF export and import carry rich text, default style, border width, style and dash pattern, rotation, justification, fringe, line leaders and captions, the caret symbol and redaction overlay text as XFDF 3.0 (ISO 19444-1) names them, and a stamp's appearance as `` (the DICT/STREAM/DATA encoding, bounded); copy and paste carries every comment's appearance. FDF export keeps a stamp's /AP with its forms and images, as XFDF does, and FDF import keeps any /AP the file carries (ISO 32000-2 §12.7.8.3.4). The intent goes out as both `intent` and `IT`. An unreadable XFDF appearance is left out, said in the import report and the repair log, and the comment is drawn by PdfCraft; an FDF past the import budget is refused whole. A redaction mark is always drawn by PdfCraft. + +- **2026-11-07 (M2, #6449):** A non-embedded Times New Roman set as a TrueType CID font (`TimesNewRoman,Bold`, `TimesNewRomanPSMT` and the other faces) draws the glyphs its codes name, as the non-embedded Arial does: its first 258 glyphs follow the same standard Macintosh order, which the embedded Times New Roman subsets of a public French form confirm in 941 of 942 `/ToUnicode` entries (vendored hayro-interpret patch 88). That form drew "Salarié(e) du régime général" as "Salari²*g+f w't²i lo g'ì²p²tcn" and "changement" as "ej angement". Times and Times New Roman MT Std are not read this way. + +- **2026-11-05 (M2, #5889):** A non-embedded Arial set as a TrueType CID font (`CIDFontType2`, `Identity-H`, as Word and PDFMaker write it) draws the glyphs its codes name. Those codes are glyph indexes in the author's Arial, whose first 258 glyphs follow the standard Macintosh order, so the standard substitute now draws each glyph by name (vendored hayro-interpret patch 76). Before, it took each code as one of its own glyph indexes, and "Low Power" drew as "@ck 'Dck Yf". Arial Narrow, Black and Unicode MS, and `CIDFontType0` fonts, are not read this way. Copying or searching those runs still needs a `/ToUnicode`. + - **2026-11-07 (M7, #2964 / #3185):** Supported Form text can be prepared for direct editing; groups left grouped (unsafe compositing, text nested past eight levels) are reported. Legacy Thai keeps the original font programs, Regular/Bold and text settings, also across same-face subsets; the narrow THSarabunPSK map repair is checked glyph by glyph against the font program; cross-family replacement is explicit. Builds on #3067 and #2811. Unsafe groups and mixed contextual styles remain partial; estimates unchanged. - **2026-10-28 (M10, #4119 partial):** Scan & OCR ▸ Enhance scanned file cleans scanned pages in place: background removal (level 0–100, as the camera clean-up), edge shadow removal (book gutters, copier borders) and despeckle. Each scan image is decoded, cleaned and written back into its own object at the same size and colour space (JPEG stays JPEG, others lossless, 1-bit stays 1-bit), so the OCR text layer, comments and links stay; one undo step, shared images once, unsupported images (CMYK, indexed, JPEG 2000, JBIG2, CCITT, broken data) reported. `page_enhance_scan` drives it headlessly. Not yet an option of Recognize text. Estimates unchanged. - **2026-11-07 (M10, `ocr.ocr-web`, port onto #977):** Web OCR rebuilt on main's engines and preprocessing (#977). `OcrJob::run` is now `OcrJob::steps` a page at a time: the page renders, crops to the region and cleans each page, a Web Worker runs ocrs on the result, and `OcrShot::place` maps the words back through the region and preprocessing. Every strategy reads like this when only one engine is present, and the web build has only ocrs. The downloaded models are installed into the engine cache (`ocr::install`), so the Verify screen and the engine-status checks see them too. The batch keeps main's skipped-files bucket, low-confidence notes and cancel. The translations moved to craft-i18n (storytold/craft-i18n#141). The Verify screen starts the model download itself and runs once the models land; before, on a fresh page it failed with the desktop's "set PDFCRAFT_MODELS" message. Known limits: the cleaning steps, and the Verify screen's Run OCR and re-reads, run on the page. Estimates unchanged. @@ -250,14 +257,17 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md - **2026-11-03 (M5, #5850):** Fill & Sign Enter inserts a line break in a growing editor; clicking outside commits and Escape cancels. Nonempty text keeps leading, internal and trailing line breaks. The headless fill_sign_add path sizes its typewriter box for the same wrapped lines so saved appearances show every row. Production-order keyboard and engine save/reopen/render regressions exercise the UI and headless callers. Estimates unchanged. +- **2026-11-03 (community, M7, Thai word breaking):** Thai rows end between words, found with ICU4X's dictionary line segmenter (`icu_segmenter` 2.3, compiled dictionary data, no LSTM model; Unicode-3.0), instead of only at spaces when rewrapping and at any cluster in Add text. A row never starts with a following vowel, tone or other mark, Mai Yamok or Paiyannoi, nor ends with a leading vowel; a word wider than the row still breaks between clusters, and the spaces where an Add text row ends are dropped as in Latin text. Rewrapping existing text works out the breaks once per run of non-space characters. Rewriting a paragraph into Thai still needs a source font with the Thai glyphs (there is no Thai fallback face for it yet). The compiled dictionaries (Thai, Lao, Khmer, Burmese) add about 1.8 MB to a stripped release binary. Estimates unchanged. - **2026-11-03 (M7, #5835):** In image crop mode, a right-click on the image, its crop box or its handles keeps the crop and opens no menu, so the menu no longer switches as the pointer moves. A right-click elsewhere clears the selection and opens the page menu. UI tests cover all four page rotations, overlapping images, a 45° image, and a check that the Done/Cancel/Remove Crop pills at a page edge still keep the crop. -- **2026-11-02 (M12, #1297):** `model_add` embeds a U3D or PRC file, recognised by its header, as a 3D annotation with a chosen activation and a poster (an image or PDF page fitted in the box, or a labelled frame); an older file's catalog `/Version` is raised to what the model needs. Views, a rendered poster and a UI command are later steps. Estimates unchanged. +- **2026-11-03 (M14 docs, #5805):** `docs/localization-parity.md` and the Languages table here count every catalog with `cargo xtask i18n-report`: the share of the 2,897 strings in its denominator, on `14ae839a`. The catalog-union measure it replaces had grown to 2,895 keys and left four catalogs at "100%" that were 44 keys short. Hindi, Indonesian and Korean are listed with their catalogs (97.5% each) instead of 0, and Thai, Lao and Central Kurdish join the table. The ≈ 68% localization score is not re-scored. - **2026-11-02 (M10, #5743):** macOS print jobs name their title and owner (`com.apple.print.JobInfo.PMJobName`, `PMJobOwner`), as the system print dialog does, because a Kyocera filter crashes on a job without them. Checked by a test and through `doc_print` against a stand-in `lp`; not yet on a physical printer. - **2026-10-31 (M3, #5452):** Native Home Recent rows offer Reveal in Finder / File Explorer from their context menu. They reuse the tab action’s platform dispatch and missing-file notification without opening the PDF or changing Recent. Browser behavior and milestone estimates unchanged. +- **2026-11-02 (M3, #5708):** Pages shorter than the window sit in its middle, one page or row at a time and when scrolling continuously with every page in view. +- **2026-11-02 (M3, #5716):** A snapshot is drawn at 300 ppi, or at the zoom if that is finer, so one taken zoomed out stays readable; an area too large for one 4096-pixel tile comes out as fine as a tile allows (about 147 ppi for 2000 pt). A choice of resolution and a selection that stays active remain. - **2026-11-02 (M14, #5737, #5460):** Preferences can turn off the update check at start. It stays on by default, and Help ▸ Check for updates still asks. - **2026-11-01 (M7, #5657):** Header/footer, watermark and background additions and replacements stage the complete call before committing. Synthetic later-page failures preserve exact incremental bytes, object count, dirty state and allocation cursor; success controls retain other marks and shared/inherited resources. Estimates unchanged. @@ -306,6 +316,7 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md - **2026-10-23 (M11, JBIG2):** The PDF Optimizer can store black-and-white images as lossless JBIG2 (a generic region, written from the JBIG2 draft standard and checked against its arithmetic-coder test sequence), about a third smaller than Group 4 on 300 dpi text, and Export ▸ Images reads JBIG2 images. JBIG2 symbol coding, where its largest gains on text are, is next. - **2026-10-23 (M11, monochrome compression):** The PDF Optimizer and Reduce File Size store black-and-white images (scans, image masks) losslessly as CCITT Group 4 when that is smaller, with a Monochrome Images setting in the Optimizer and `doc_optimize` `mono`. JBIG2 and monochrome downsampling are next. - **2026-10-23 (M6, calculation checks):** The calculation order now flags a formula that reads a field the form doesn't have (a field deleted from under a sum, a typo in notation, a script's `getField`) and a script that builds the names it reads at run time. Prepare a form ▸ Calculation order marks both on the row and in a card that names the broken formulas, in place of its all-clear. `form_calculation_order` gives each field's formula, and `form_fields` shows a custom Calculate script. +- **2026-10-23 (M4, #3923):** A recovered document's original path is only a suggestion: the first Save after recovering opens Save As at that path instead of writing it, since anyone who can write the recovery folder (a shared portable `PdfCraftData`, for example) can name any file in a sidecar. The file chosen is the document's file from then on. Sidecars are read through a 64 KiB cap; a longer one is not read, and its snapshot recovers without metadata. Estimates unchanged. - **2026-10-23 (M7, #4280):** Edit text names each standard stand-in font after its face (`PCEdTimesBold`, ...) instead of one shared `PCEdHelv`. Rewriting a bold paragraph whose subset lacks an accent no longer repoints the paragraphs set earlier in a regular stand-in at Times-Bold, and the editor's frame stays on the paragraph being edited. Pages saved by older builds keep their shared name and are not repaired. Estimates unchanged. - **2026-10-23 (M8, #3205 follow-up):** Phone-pattern redaction matches complete labelled Italian geographic numbers such as `Tel. +39 0426/123456`, with synthetic matcher and mark/apply/save/reopen coverage; unlabelled international slash numbers remain unsupported. - **2026-10-23 (M7, #2636 partial):** A new `PasteContent` edit pastes an added text box or image copied in the same document or another, keeping its style, picture, turns, flips and crop, as one undo step; `content_copy` drives it headlessly. A paste lands beside earlier copies (18 pt steps) and stays on the page. A pasted image reuses its picture object in its own document and after the first paste into another; there a colour profile is dropped and CMYK colours stored losslessly are turned to RGB. The desktop keys and menus follow in separate PRs. Estimates unchanged. @@ -435,6 +446,7 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md - **2026-10-16 (M11, #1201):** Optimize decodes a JPEG with its colour transform: an Adobe APP14 marker's flag, else the filter's `/ColorTransform` (ISO 32000-2 §7.4.8). An RGB JPEG stored untransformed (`/ColorTransform 0`, no marker) used to be converted from YCbCr anyway and came out in other colours, recompressed or merged from strips; it now keeps them. A JPEG whose marker and `/ColorTransform` disagree, or whose components are named R, G and B without `/ColorTransform 0`, is left as it is, since readers differ on it. Estimates unchanged. - **2026-10-15 (M2, gap 12):** `cargo xtask text-oracle` scores reading order as well as word F1: the longest common subsequence of the two word sequences over the longer one. F1 is a bag of words and cannot see order at all -- measuring #2431, a change that moves text and nothing else, it called 26 of the 29 files whose output changed identical, where the order score separated 18 improvements from 6 regressions. Reports, does not gate; a disagreement with Poppler, not a verdict. Estimates unchanged. - **2026-10-15 (M2, #2305 follow-up):** WinAnsiEncoding 0xA0 and 0xAD and MacRomanEncoding 0xCA extract as the ordinary space and hyphen Annex D.2 names them, in the vendored hayro tables (patch 30) as well as `crates/fonts`, so search, copy and redaction see characters that were hidden behind U+00A0 and U+00AD. Across the pdf.js corpus the soft hyphens we extract fall 117 to 13 and the non-breaking spaces 30 to 26; the rest come from `/ToUnicode`. An explicit `/Differences` name keeps its glyph list value, as pdftotext does. Estimates unchanged. +- **2026-10-15 (M14, gap 16, #1683):** Split view. View ▸ Split (⌘\\ / Ctrl+\\) and a tab's Split right / Split down show two panes, each with its own tab strip, scroll and zoom; a tab dropped on the right half of the page area splits right, and the divider drags. One document in both panes is one document with one Undo; closing one of its views keeps it open. The pane last clicked is the one menus, keys and panels act on. Nothing about the panes is kept across restarts. `view.split-window` planned → shipped. Multiple windows stay planned. Estimates unchanged. - **2026-10-15 (M12, XFA, #2574):** A `pageArea` with no `` repeats without limit instead of taking a subform's default of one use, so a later page area with `occur max="1"` is no longer drawn (with a copy of its fields) over every body page. A page area's `occur min` is honoured too: one the body never reached gets the pages it still needs after the body (the once-only closing page of a form, with its button, comes last). Acrobat Pro also puts that page last, and when printing keeps it but leaves it blank (`relevant="-print"` on the page area), which PdfCraft does not do yet; both re-observed on a synthetic fixture, see the 2026-10-19 line. Synthetic regression tests. Estimates unchanged. - **2026-10-19 (M12, XFA, #2574):** The Acrobat observations behind the line above were first made on a public real-world form, which the Privacy rule does not allow; they are replaced by ones made on a synthetic form (three `breakBefore pageArea` sections, then a page area with `occur min="1" max="1"`, with and without `relevant="-print"`). Acrobat Pro shows the closing page last in both and prints it in both, blank only with `relevant="-print"`: confirmed. The synthetic form also shows a difference not fixed yet: Acrobat opens it as 5 pages, with an empty first body page before the first section, where PdfCraft lays out 4. Two more synthetic forms narrow it down: with `breakBefore targetType="pageArea"` on the first section, with or without `startNew`, Acrobat opens an empty body page first; without a break on the first section it lays out 4, as PdfCraft does (so `startNew` is not the cause). Not fixed yet: `layout.rs` skips a break while the page is still empty. Estimates unchanged. - **2026-10-15 (M8, #2447):** Removing links or comments (Remove Hidden Information, Sanitize) also drops the tagged-PDF structure tree's object references to them, so the full rewrite no longer keeps a removed link's URL or a comment's text. Regression seen to fail without the fix. Estimates unchanged. @@ -445,6 +457,9 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md - **2026-10-14 (M12, reading colours, #2059):** Pending page backgrounds use the document's reading colour immediately, preventing white flashes while scrolling to uncached pages. A headless canvas regression covers three pending pages in every preset and a custom background. Estimates unchanged. - **2026-10-15 (community, M6, #2608):** In Prepare a form the selected field moves with the arrow keys: a point per press, ten with Shift, as the object nudge and dragging it do. Each press is one undo step ("Change field properties"), and the keys no longer turn the page while a field is selected. Estimates unchanged. - **2026-10-15 (M3, closable community card):** The ArtCraft community card on Home has a close button that hides it for the rest of the run; it returns on the next launch, and its links stay in the top bar's Discord button and the Help menu. A UI regression covers closing and the fresh-launch default. Estimates unchanged. +- **2026-10-15 (M14, Combine files from File Explorer, #585):** The Windows MSI adds "Combine files in PdfCraft" to PDF files' right-click menu (with the Edit verb, same switch); each selected file starts `pdfcraft --combine`, and the single-instance hand-off (`"combine": true`, `OsEvent::Combine`) collects them into the running PdfCraft's Combine files tab. Tests: the request round trip and hand-off in `single_instance.rs`, the OS event in `tests/ui.rs`, the verb rows in `test-msi.ps1`. Estimates unchanged. +- **2026-10-15 (M14, MSI install folder, #1139):** The Windows installer's welcome dialog has an "Install to" row with a native folder browser, starting on the folder chosen for the per-user or per-machine scope (the recorded one on an update). `test-msi.ps1` checks the dialog wiring: both Install buttons set the folder before the dialog closes, and Browse's OK and Cancel. Estimates unchanged. +- **2026-10-15 (M8, #2448):** Applying redactions also removes the document information entries and the document's and pages' XMP metadata that repeat the removed text, so a redacted word can't survive in `/Title`, `/Keywords` or the XMP that mirrors them. Estimates unchanged. - **2026-10-13 (community, M3, #1569 review):** Pixel-only Wayland wheel, wheel-tilt and continuous input use standalone moved events, so Ctrl zoom does not remain latched after the modifier is released (#991); vendored winit patch (13), with a policy test. Estimates unchanged. - **2026-10-18 (M1, PR #1143 repair policy):** A live in-file reference resolves when its generation matches the xref row or a same-number header at that indexed offset. Neither-match and free slots remain null, including when old object bytes remain. Opening records bounded xref/header disagreements without parsing object bodies or expanding compact xref rows. Synthetic controls cover read order, clones, overlays, dictionary edits, full classic/packed saves and an independent reader of incremental output. **Correction to the 2026-10-15 note below:** its async-opening-test sentence was not evidenced by this COS change; no fresh async-opening or GUI validation is claimed. The full-reference stream-length recursion key is defensive hardening, not an independently isolated regression fix. Estimates unchanged. - **2026-10-15 (M1, PR #1143 refresh):** COS reference lookups and full saves distinguish complete object references, so stale generations cannot expose replacement content. Snapshot restoration also keeps edits to objects with nonzero generations before incremental saves. Synthetic regressions cover caches, overlays, snapshots, stream lengths and classic/packed saves. Current upstream async-opening test corrections are included. Estimates unchanged. @@ -515,6 +530,10 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md - **2026-10-15 (M3, #1248):** A fresh install follows the desktop's appearance: the theme preference defaults to System (dark on a dark desktop, Light where the desktop says nothing); saved settings and `--theme` still win. Dark page rendering is Reading colours, and the page reading themes are #1732. - **2026-10-15 (M3, #1767):** Restore last view settings when reopening documents, as Acrobat's preference does: a file opened again shows the page, zoom and page display it was last shown with (remembered for the last 100 files when a tab closes or settings are saved), over the file's own initial view; on by default, Preferences ▸ Documents and view turns it off. +- **2026-10-14 (M10, #2001):** The Print dialog remembers two-sided, grayscale, collate, page sizing (with its custom percent) and Comments & forms across restarts, under the names `doc_print` uses; copies, pages, layout and orientation still start fresh for each job. The option names now live in one place (`pdfcraft_print::names`) for the tool and the settings file. +- **2026-10-14 (M2, #623):** A page whose content was cut at the renderer's limits no longer looks complete: a notice names it (and how many), `ui.state` lists the affected pages under `active.cut_pages`, and `text_extract` marks its text `truncated`, since the cut part's text can't be found or copied. The renderer already reported the cut; nothing showed it. +- **2026-10-14 (M5, comment pop-ups; reworked 2026-11-01):** A double-click on a comment opens its pop-up window on the page: a title bar with the comment's kind, author and date; its text and its replies' typed in place, the replies as panels each inside the one it answers, a Reply button under each; the window keeps its `/Rect` size on screen as the page zooms, scrolls its thread, drags by its title bar, resizes by its corner and closes with ×, each an undo step saved in the file's `/Popup` so other viewers agree. A faint wedge in the comment's colour joins the comment to its open window on roll-over or selection. `comment_popup` does the same headlessly and `comment_list` reports the pop-up. +- **2026-10-15 (M6, #1741):** Custom stamps can be dynamic: Create Custom Stamp adds an optional line under the picture, a template with {name}, {date} and {time} filled in when the stamp is placed (the built-in dynamic stamps' format), kept in the stamp library; `stamp_custom` takes the same `line`. - **2026-10-13 (M3, native-opening candidate):** Chosen tools survive preparation, conversion and password retries; startup intake waits for pending preparation. Regression tests cover completion, cancellation and document identity. Estimates unchanged. - **2026-10-13 (community, M3, #531 touchpad momentum):** On Wayland a two-finger swipe keeps gliding after the fingers lift and slows to a stop, measured from the gesture's last 0.1 s and decaying with a 0.35 s time constant through the #759 1:1 path. Any wheel input, click, key, zoom or edge ends it (fingers resting on the touchpad don't yet: the vendored winit binds no Wayland hold gestures); single-page paging, macOS (which sends its own momentum), the web, and Animation effects Off never glide. Unit tests for the physics and two headless view tests. Estimates unchanged. - **2026-10-13 (M3, native-open admission):** Later prepared replies stay queued while a modal or native picker owns the foreground. Picker completion, cancellation and abandonment wake retained opens. Regressions cover recipient ownership, exactly-once admission, source/history preservation and one-shot repaint requests. Estimates unchanged. @@ -886,6 +905,7 @@ Newest first: the date, what moved, and the new overall estimate. - **2026-10-21 (M4, #3865):** A new bookmark records where the reader is: the top-left of the selected text, or the top of the window, as `/XYZ left top null` (zoom kept), instead of the page top (`Edit::AddBookmark` takes `left` and `top`; `bookmark_add` takes `top` and `left`; `PageInfo::xyz_for_view`). - **2026-10-18 (renderer, hayro patches 19/35):** A mesh shading (types 4–7) is sampled over its own triangles instead of the whole path: a BricsCAD plan with 310 one-triangle `sh` fills took 2 s per 1024 px tile at every zoom (and hit the 20 s watchdog on a busy machine); now about 70 ms, so zooming into dense topology drawings stays fluid. Estimates unchanged. +- **2026-10-22 (M3, #3530, builds on PR #3547):** A find step keeps its match in the middle of the window after zooming: a zoom pending in the same frame no longer puts the page-top jump off to the next frame, and the rest of an eased ⌘/Ctrl-wheel zoom anchors on the match instead of the pointer. Synthetic UI regressions step through matches on pages larger and smaller than the window at 10–3200% and in every page display. Estimates unchanged. - **2026-10-22 (PR #3227 integration):** M11: integrate upstream `80b7d215`; retain partial target-version compatibility and upstream content-boundary and image-size regression coverage. Scope and estimates unchanged. diff --git a/apps/pdfcraft-cli/build.rs b/apps/pdfcraft-cli/build.rs index ee77f13f6a..9a6500b7f3 100644 --- a/apps/pdfcraft-cli/build.rs +++ b/apps/pdfcraft-cli/build.rs @@ -6,12 +6,25 @@ //! cross-compile from macOS or Linux still links, unless `PDFCRAFT_REQUIRE_WINRES=1` turns it //! into an error (for release builds). +/// The in-box DLLs only printing uses (Windows.Data.Pdf draws the job, prntvpt reads the +/// driver's print capabilities). Trimmed Windows installations (NTLite) can lack them, and a +/// load-time import of a missing DLL stops the program before `main` (#2630). Delay-loaded, they +/// load on the first print; `pdfcraft_print::spool` checks they exist before calling in. +/// Same list in `apps/pdfcraft{,-cli}/build.rs`, `crates/print/build.rs` and `crates/print/src/spool.rs`. +const DELAY_LOADED: [&str; 2] = ["windows.data.pdf.dll", "prntvpt.dll"]; + fn main() { println!("cargo:rerun-if-changed=build.rs"); println!("cargo:rerun-if-env-changed=PDFCRAFT_REQUIRE_WINRES"); if std::env::var("CARGO_CFG_TARGET_OS").as_deref() != Ok("windows") { return; } + if std::env::var("CARGO_CFG_TARGET_ENV").as_deref() == Ok("msvc") { + for dll in DELAY_LOADED { + println!("cargo:rustc-link-arg-bins=/DELAYLOAD:{dll}"); + } + println!("cargo:rustc-link-arg-bins=delayimp.lib"); + } let mut res = winresource::WindowsResource::new(); res.set("ProductName", "PdfCraft") .set("FileDescription", "PdfCraft command-line tool") diff --git a/apps/pdfcraft-cli/src/check.rs b/apps/pdfcraft-cli/src/check.rs index 62993ad6dc..4521181b6f 100644 --- a/apps/pdfcraft-cli/src/check.rs +++ b/apps/pdfcraft-cli/src/check.rs @@ -43,7 +43,18 @@ pub(crate) fn check_one(args: &[String]) -> Result<(), CliError> { let path = *positional(args, CHECK_ONE_OPTIONS).first().ok_or("check-one: missing file")?; let dpi = parse_dpi(flag(args, "--dpi").unwrap_or("36"))?; let start = Instant::now(); - let bytes = read(path)?; + // A file that can't be read (past the input cap, say) is this file's record, not a crash of the child. + let bytes = match read(path) { + Ok(bytes) => bytes, + Err(e) => { + let status = if e.contains("larger than") { "input-too-large" } else { "read-error" }; + let line = serde_json::json!({ "file": path, "status": status, "pages": 0, "failed_pages": 0, "warnings": 0, + "ms": start.elapsed().as_millis() as u64, "detail": e.chars().take(400).collect::() }); + stdout_line(format_args!("{line}"))?; + // (the record is the answer `check` reads; the error and exit status are for a direct run) + return Err(e.into()); + } + }; let (status, pages, failed, detail, warnings) = match inspect(bytes.clone(), None) { Err(e) => (format!("open-error:{}", short(&e.to_string())), 0, 0, e.to_string(), 0), Ok(info) => { @@ -65,7 +76,7 @@ pub(crate) fn check_one(args: &[String]) -> Result<(), CliError> { // `--edit` (fuzzing): also run our own object layer end to end, with no panic safety net, // so a panic in cos shows up as a crash of this process. if args.iter().any(|a| a == "--edit") { - edit_round_trip(&std::fs::read(path).map_err(|e| e.to_string())?); + edit_round_trip(&read(path)?); } let line = serde_json::json!({ "file": path, "status": status, "pages": pages, "failed_pages": failed, "warnings": warnings, "ms": start.elapsed().as_millis() as u64, "detail": detail.chars().take(400).collect::() }); @@ -76,13 +87,19 @@ pub(crate) fn check_one(args: &[String]) -> Result<(), CliError> { /// Open with cos, touch every object, edit the catalog, save incrementally and in full (both /// output styles), and reopen each result. Errors are fine; panics, hangs and aborts are bugs. fn edit_round_trip(bytes: &[u8]) { - use pdfcraft_cos::{Document, Object, SaveOptions, write_full, write_incremental}; + use pdfcraft_cos::{DecodeBudget, Document, Object, SaveOptions, write_full, write_incremental}; let Ok(mut doc) = Document::open(Arc::new(bytes.to_vec())) else { return }; + // One allowance for every stream, charged for what each decode did (failed ones too), so + // many small bombs can't each inflate to the per-stream limit. + let mut budget = DecodeBudget::page(); for num in doc.object_numbers().into_iter().take(20_000) { + if budget.exhausted() { + break; + } if let Ok(o) = doc.try_get(num) && let Object::Stream(s) = &*o { - let _ = s.decoded(); + let _ = budget.decode(s); } } if let Some(root) = doc.root() { @@ -113,7 +130,7 @@ fn collect(paths: &[&str]) -> Vec { let identity = match p.canonicalize() { Ok(identity) => identity, Err(error) => { - eprintln!("{}: could not resolve directory: {error}", p.display()); + eprintln!("{}: could not resolve directory: {error}", p.display().to_string().escape_debug()); continue; } }; @@ -158,12 +175,16 @@ pub(crate) fn check(args: &[String]) -> Result<(), CliError> { let status = r["status"].as_str().unwrap_or("?").to_string(); *counts.entry(status.split(':').next().unwrap_or("?").to_string()).or_default() += 1; if status != "ok" { + // One record per line: a file name or detail with a line break or a terminal control in + // it is written escaped, so it can't forge another record (the JSON report keeps both). + let detail: String = r["detail"].as_str().unwrap_or("").chars().take(160).collect(); eprintln!( - "[{}/{}] {status:<28} {} {}", + "[{}/{}] {:<28} {} {}", n + 1, files.len(), - f.display(), - r["detail"].as_str().unwrap_or("").chars().take(160).collect::() + status.escape_debug().to_string(), + f.display().to_string().escape_debug(), + detail.escape_debug() ); } results.push(r); diff --git a/apps/pdfcraft-cli/src/commands.rs b/apps/pdfcraft-cli/src/commands.rs index 3118c901b4..cb3365a4fe 100644 --- a/apps/pdfcraft-cli/src/commands.rs +++ b/apps/pdfcraft-cli/src/commands.rs @@ -4,10 +4,24 @@ use super::options::{ }; use super::{Arc, CliError, ImageFormat, PageRenderer, Path, PathBuf, RenderConfig, RenderRequest, RequestKind, Write, inspect, stdout_line}; +/// Largest input file read into memory (512 MiB, as automation's `doc_open` reads). +pub(crate) const MAX_INPUT: u64 = 512 * 1024 * 1024; + +/// The file at `path`, read no further than [`MAX_INPUT`]: a name that leads to something +/// endless (a device, a pipe nobody closes) fails at the cap instead of filling memory. pub(crate) fn read(path: &str) -> Result>, String> { - std::fs::read(path).map(Arc::new).map_err(|e| format!("{path}: {e}")) + use std::io::Read as _; + let mut bytes = Vec::new(); + std::fs::File::open(path).and_then(|f| f.take(MAX_INPUT + 1).read_to_end(&mut bytes)).map_err(|e| format!("{path}: {e}"))?; + if bytes.len() as u64 > MAX_INPUT { + return Err(format!("{path}: the file is larger than 512 MiB")); + } + Ok(Arc::new(bytes)) } +/// `info FILE`: a JSON summary of the document as the app opens it (pages, fields, metadata, the +/// standards it declares, and the notes about damage repaired while opening it), with none of +/// its scripts run. pub(crate) fn info(args: &[String]) -> Result<(), CliError> { validate_options("info", args, INFO_OPTIONS)?; let path = *positional(args, INFO_OPTIONS).first().ok_or("info: missing file")?; @@ -18,24 +32,33 @@ pub(crate) fn info(args: &[String]) -> Result<(), CliError> { let mut info = inspect(bytes.clone(), render_password.as_deref()).map_err(|e| e.to_string())?; let pdf_pages = info.pages.len(); let acroform_fields = info.fields.len(); + // The document as the app opens it: the notes about damage repaired while opening, which the + // app shows in Document Properties, and for a dynamic XFA form the laid-out pages and fields. + // Inspection runs none of the document's scripts: an XFA form's initialize and calculate + // scripts, which the app runs on opening, are left alone, as with JavaScript turned off. + let mut session = pdfcraft_engine::Session::new(); + session.set_javascript(false); + let opened = session.open("info.pdf", None, bytes.clone(), password).ok().and_then(|id| session.get(id)); // Dynamic XFA forms: say what laying the template out gives (what the app shows). - let xfa_layout = (info.xfa == Some(pdfcraft_render::Xfa::Dynamic)) - .then(|| { - let mut s = pdfcraft_engine::Session::new(); - let id = s.open("info.pdf", None, bytes.clone(), password).ok()?; - let d = s.get(id)?; - d.xfa.as_ref().map(|x| { - info = d.info.clone(); - // The in-memory layout is an appended revision, not the file being inspected. - info.file_size = bytes.len(); - serde_json::json!({ "pages": x.pages, "fields": x.fields, "warnings": x.warnings }) - }) + let xfa_layout = opened.filter(|_| info.xfa == Some(pdfcraft_render::Xfa::Dynamic)).and_then(|d| { + d.xfa.as_ref().map(|x| { + info = d.info.clone(); + // The in-memory layout is an appended revision, not the file being inspected. + info.file_size = bytes.len(); + serde_json::json!({ "pages": x.pages, "fields": x.fields, "warnings": x.warnings }) }) - .flatten(); - // What the document declares as a standard, read the way the Standards panel reads it, so - // `info` and the app agree. A file that declares nothing leaves these null and empty. - let declared = - pdfcraft_cos::Document::open_with_password(bytes.clone(), password).map(|doc| pdfcraft_preflight::declared(&doc)).unwrap_or_default(); + }); + // What the document declares as a standard, as the Standards panel reads it, and the notes + // Document Properties shows, which include what the app repaired after reading the objects (a + // page tree rebuilt from the pages found by scanning), both from the session's own reading. A + // file the app cannot open is read once more for what the object reader declares and repaired. + // A file that declares nothing leaves the standards null and empty. + let (declared, repairs) = match opened { + Some(d) => (d.standards(), d.repair_log()), + None => pdfcraft_cos::Document::open_with_password(bytes.clone(), password) + .map(|cos| (pdfcraft_preflight::declared(&cos), cos.repair_log().to_vec())) + .unwrap_or_default(), + }; let json = serde_json::json!({ "file": path, "pdf_version": info.pdf_version, @@ -56,6 +79,7 @@ pub(crate) fn info(args: &[String]) -> Result<(), CliError> { "custom_properties": info.custom.len(), "page_labels": info.pages.iter().take(8).map(|p| p.label.clone()).collect::>(), "warnings": info.warnings, + "repairs": repairs, }); stdout_line(format_args!("{}", serde_json::to_string_pretty(&json).unwrap_or_default()))?; Ok(()) @@ -374,6 +398,14 @@ pub(crate) fn render(args: &[String]) -> Result<(), CliError> { // The renderer takes only the user password of R2–R4 files; `Session::open` and the display // copy take either. let render_password = pdfcraft_engine::render_password_for(bytes.clone(), password); + // A page image is an image taken out of the document, as a snapshot or Export is in the app: + // like `text`, it needs the document to allow copying (or the permissions password). + let mut session = pdfcraft_engine::Session::new(); + if let Some(doc) = session.open(path, None, bytes.clone(), password).ok().and_then(|id| session.get(id)) { + doc.check_copying().map_err( + |_| "render: the document's permissions don't allow copying its pages as images; pass the owner password with --password to render it", + )?; + } let display = if inspect(bytes.clone(), render_password.as_deref()).is_ok_and(|info| info.xfa == Some(pdfcraft_render::Xfa::Dynamic)) { let mut session = pdfcraft_engine::Session::new(); let id = session.open(path, None, bytes, password).map_err(|e| e.to_string())?; @@ -408,6 +440,18 @@ pub(crate) fn render(args: &[String]) -> Result<(), CliError> { let _ = writeln!(std::io::stderr().lock(), "warning: page {page}: a colour that could not be computed was painted in a fallback colour"); } + pdfcraft_render::RenderWarning::MalformedIndexedImage => { + let _ = writeln!( + std::io::stderr().lock(), + "warning: page {page}: an image with a malformed Indexed colour table may be drawn wrongly or not at all" + ); + } + pdfcraft_render::RenderWarning::Type3WidthsMissing => { + let _ = writeln!( + std::io::stderr().lock(), + "warning: page {page}: a Type 3 font has no /Widths; its glyphs were spaced by the widths their glyph procedures give" + ); + } } } Ok(()) diff --git a/apps/pdfcraft-cli/src/ui.rs b/apps/pdfcraft-cli/src/ui.rs index dc419171b0..0ad18c154b 100644 --- a/apps/pdfcraft-cli/src/ui.rs +++ b/apps/pdfcraft-cli/src/ui.rs @@ -50,18 +50,23 @@ pub(crate) fn ui(args: &[String]) -> Result<(), CliError> { /// Most a control file may hold. The app writes under 100 bytes. const CONTROL_FILE_MAX: u64 = 64 * 1024; +#[cfg(not(windows))] const CONTROL_FILE_ADVICE: &str = "Start the app with a control file in a folder only you can write, such as `pdfcraft --control ~/.pdfcraft-control.json`"; +// cmd.exe doesn't expand `~` (#3922). +#[cfg(windows)] +const CONTROL_FILE_ADVICE: &str = + r"Start the app with a control file in a folder only you can write, such as `pdfcraft --control %USERPROFILE%\.pdfcraft-control.json`"; /// Read the file written by `pdfcraft --control FILE`. /// /// It names the port that receives the token and every command, typed text included, so it must /// be the user's own. In a shared folder such as `/tmp`, another local user can create the file /// first; the app then can't write it, and the CLI would talk to the other user's listener. -/// Refused: a symbolic link, anything but a regular file, and on Unix a file owned by someone -/// else or open to other users (the app writes it with mode 600). +/// Refused: a symbolic link, anything but a regular file, and a file owned by someone else or +/// open to other users: on Unix by its owner and mode (the app writes it with mode 600), on +/// Windows by its owner and access list (the app gives it one of its user and SYSTEM, #3922). fn read_control_file(file: &str) -> Result { - use std::io::Read as _; let at_path = std::fs::symlink_metadata(file).map_err(|e| format!("{file}: {e}"))?; if at_path.file_type().is_symlink() { return Err(format!("{file}: is a symbolic link, which could point at another user's file. {CONTROL_FILE_ADVICE}")); @@ -69,21 +74,41 @@ fn read_control_file(file: &str) -> Result { if !at_path.is_file() { return Err(format!("{file}: not a regular file. {CONTROL_FILE_ADVICE}")); } - let f = std::fs::File::open(file).map_err(|e| format!("{file}: {e}"))?; - #[cfg(unix)] - { - use std::os::unix::fs::MetadataExt; - // Checked on the open file, so nothing swapped in after `symlink_metadata` gets through. - let opened = f.metadata().map_err(|e| format!("{file}: {e}"))?; - if (opened.dev(), opened.ino()) != (at_path.dev(), at_path.ino()) { - return Err(format!("{file}: was replaced while being opened. {CONTROL_FILE_ADVICE}")); + #[cfg(not(windows))] + let text = { + use std::io::Read as _; + let f = std::fs::File::open(file).map_err(|e| format!("{file}: {e}"))?; + #[cfg(unix)] + { + use std::os::unix::fs::MetadataExt; + // Checked on the open file, so nothing swapped in after `symlink_metadata` gets through. + let opened = f.metadata().map_err(|e| format!("{file}: {e}"))?; + if (opened.dev(), opened.ino()) != (at_path.dev(), at_path.ino()) { + return Err(format!("{file}: was replaced while being opened. {CONTROL_FILE_ADVICE}")); + } + if let Some(problem) = control_file_problem(opened.uid(), opened.mode(), current_uid()) { + return Err(format!("{file}: {problem}. {CONTROL_FILE_ADVICE}")); + } } - if let Some(problem) = control_file_problem(opened.uid(), opened.mode(), current_uid()) { + let mut text = String::new(); + f.take(CONTROL_FILE_MAX.saturating_add(1)).read_to_string(&mut text).map_err(|e| format!("{file}: {e}"))?; + text + }; + // Windows: the same owner and access check, on the security descriptor of the very handle the + // contents are read through, so nothing swapped in at the path gets through (#3922). A file + // planted by another user, or one other users can read, is refused; if the check itself can't + // run, the file isn't trusted either. + #[cfg(windows)] + let text = { + use pdfcraft_platform::owner_only; + let me = owner_only::current_user_sid().map_err(|e| format!("{file}: can't check who owns it: {e}. {CONTROL_FILE_ADVICE}"))?; + let (sddl, bytes) = owner_only::read_with_sddl(std::path::Path::new(file), CONTROL_FILE_MAX) + .map_err(|e| format!("{file}: can't check who owns it: {e}. {CONTROL_FILE_ADVICE}"))?; + if let Some(problem) = owner_only::foreign_access(&sddl, &me) { return Err(format!("{file}: {problem}. {CONTROL_FILE_ADVICE}")); } - } - let mut text = String::new(); - f.take(CONTROL_FILE_MAX.saturating_add(1)).read_to_string(&mut text).map_err(|e| format!("{file}: {e}"))?; + String::from_utf8(bytes).map_err(|e| format!("{file}: {e}"))? + }; if u64::try_from(text.len()).unwrap_or(u64::MAX) > CONTROL_FILE_MAX { return Err(format!("{file}: too large for a control file (over {} KiB)", CONTROL_FILE_MAX / 1024)); } @@ -112,6 +137,38 @@ fn current_uid() -> u32 { mod tests { use super::control_file_problem; + /// #3922: on Windows the CLI checks the control file's owner and access list. A file other + /// users can read is refused; the same file restricted to its owner is accepted. + #[cfg(windows)] + #[test] + fn on_windows_a_control_file_other_users_can_read_is_refused() { + let dir = std::env::temp_dir().join(format!("pdfcraft-cli-control-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let users = std::process::Command::new("icacls").arg(&dir).arg("/grant").arg("*S-1-5-32-545:(OI)(CI)R").output().unwrap(); + assert!(users.status.success()); + let file = dir.join("ctl.json"); + std::fs::write(&file, r#"{"port":4242,"token":"t0ken"}"#).unwrap(); + let path = file.to_string_lossy().into_owned(); + let refused = super::read_control_file(&path).unwrap_err(); + assert!(refused.contains("another account") && refused.contains("%USERPROFILE%"), "{refused}"); + pdfcraft_platform::owner_only::restrict_to_owner(&file, &pdfcraft_platform::owner_only::current_user_sid().unwrap()).unwrap(); + assert_eq!(super::read_control_file(&path).unwrap()["token"].as_str(), Some("t0ken")); + // The contents come with the check, capped: exactly the limit is read, one byte more is refused. + let padded = |len: usize| { + let json = r#"{"port":4242,"token":"t0ken"}"#; + json.to_string() + &" ".repeat(len - json.len()) + }; + std::fs::write(&file, padded(64 * 1024)).unwrap(); + assert_eq!(super::read_control_file(&path).unwrap()["port"].as_u64(), Some(4242)); + std::fs::write(&file, padded(64 * 1024 + 1)).unwrap(); + assert!(super::read_control_file(&path).unwrap_err().contains("too large")); + // Valid JSON once the byte is replaced by U+FFFD, so only a strict UTF-8 read refuses it. + std::fs::write(&file, b"{\"port\":4242,\"token\":\"\xFF\"}").unwrap(); + assert!(super::read_control_file(&path).is_err(), "not UTF-8"); + let _ = std::fs::remove_dir_all(&dir); + } + #[test] fn a_control_file_must_be_yours_and_private() { // Regular file type bits (0o100000) don't matter, only the owner and the permission bits. diff --git a/apps/pdfcraft-cli/tests/it/check.rs b/apps/pdfcraft-cli/tests/it/check.rs index 7a6236419c..3f32459089 100644 --- a/apps/pdfcraft-cli/tests/it/check.rs +++ b/apps/pdfcraft-cli/tests/it/check.rs @@ -129,3 +129,18 @@ fn a_bad_timeout_or_dpi_is_refused_before_scanning() { ); } } + +/// A failing file's name goes to stderr escaped: a line break in it can't start a second, +/// forged record. The JSON report keeps the name as it is. +#[cfg(unix)] +#[test] +fn a_file_name_cannot_forge_a_stderr_record() { + let dir = Workdir::new("escaped-name"); + let name = "bad\n[9] ok forged.pdf"; + std::fs::write(dir.0.join("input").join(name), b"not a pdf").unwrap(); + let out = Command::new(BIN).arg("check").arg(dir.0.join("input")).args(["--dpi", "36", "--timeout", "5"]).output().unwrap(); + let stderr = String::from_utf8_lossy(&out.stderr); + let records: Vec<&str> = stderr.lines().filter(|l| l.starts_with('[')).collect(); + assert_eq!(records.len(), 1, "{stderr}"); + assert!(records[0].contains("bad\\n[9] ok forged.pdf"), "{stderr}"); +} diff --git a/apps/pdfcraft-cli/tests/it/info_repairs.rs b/apps/pdfcraft-cli/tests/it/info_repairs.rs new file mode 100644 index 0000000000..72e8bed1a3 --- /dev/null +++ b/apps/pdfcraft-cli/tests/it/info_repairs.rs @@ -0,0 +1,75 @@ +//! `info` lists the notes about damage repaired while opening, as the app shows them in Document +//! Properties and as `doc_info` reports them in `repairs`: a rebuilt cross-reference index, and a +//! page tree rebuilt from the pages found by scanning (a note the app adds after reading). + +use std::process::Command; + +const BIN: &str = env!("CARGO_BIN_EXE_pdfcraft-cli"); + +/// A PDF with `pages` pages, each drawing a rectangle (a page found by scanning has content), and +/// `catalog` as object 1; `startxref` points at its cross-reference table, or at `bad_startxref`. +fn fixture(catalog: &str, pages: usize, bad_startxref: Option) -> Vec { + let content = "0 0 0 rg 10 10 40 20 re f"; + let kids: String = (0..pages).map(|i| format!("{} 0 R ", 4 + i)).collect(); + let mut objs = vec![ + catalog.to_string(), + format!("<< /Type /Pages /Kids [{kids}] /Count {pages} >>"), + format!("<< /Length {} >>\nstream\n{content}\nendstream", content.len()), + ]; + objs.extend((0..pages).map(|_| "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 120 60] /Contents 3 0 R >>".to_string())); + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offsets.push(out.len()); + out.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes()); + } + let xref = out.len(); + out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for off in &offsets { + out.extend_from_slice(format!("{off:010} 00000 n \n").as_bytes()); + } + let start = bad_startxref.unwrap_or(xref); + out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{start}\n%%EOF\n", objs.len() + 1).as_bytes()); + out +} + +const CATALOG: &str = "<< /Type /Catalog /Pages 2 0 R >>"; + +fn info_of(bytes: &[u8], name: &str) -> serde_json::Value { + let dir = std::env::temp_dir().join(format!("pdfcraft-info-repairs-{}-{name}", std::process::id())); + std::fs::create_dir_all(&dir).unwrap(); + let path = dir.join("f.pdf"); + std::fs::write(&path, bytes).unwrap(); + let out = Command::new(BIN).args(["info", path.to_str().unwrap()]).output().unwrap(); + assert!(out.status.success(), "{}", String::from_utf8_lossy(&out.stderr)); + let json = serde_json::from_slice(&out.stdout).expect("info prints JSON"); + let _ = std::fs::remove_dir_all(&dir); + json +} + +fn repairs(json: &serde_json::Value) -> Vec { + json["repairs"].as_array().unwrap_or_else(|| panic!("no repairs array: {json}")).iter().map(|l| l.as_str().unwrap().to_string()).collect() +} + +#[test] +fn info_lists_a_rebuilt_cross_reference_index() { + // `startxref` points into the middle of an object, so the index is rebuilt from the headers. + let json = info_of(&fixture(CATALOG, 1, Some(20)), "xref"); + assert_eq!(json["pages"], serde_json::json!(1), "{json}"); + let lines = repairs(&json); + assert!(lines.iter().any(|l| l.contains("rebuilt cross-reference index")), "{lines:?}"); +} + +#[test] +fn info_lists_a_page_tree_the_app_rebuilt_by_scanning() { + // The catalog names a page tree object the file does not have (#1656). + let json = info_of(&fixture("<< /Type /Catalog /Pages 9 0 R >>", 2, None), "pagetree"); + let lines = repairs(&json); + assert!(lines.iter().any(|l| l.contains("page tree is missing")), "{lines:?}"); +} + +#[test] +fn info_lists_no_repairs_for_a_sound_file() { + let json = info_of(&fixture(CATALOG, 1, None), "sound"); + assert_eq!(json["repairs"], serde_json::json!([]), "{json}"); +} diff --git a/apps/pdfcraft-cli/tests/it/main.rs b/apps/pdfcraft-cli/tests/it/main.rs index 30a6e25882..7b7292fe45 100644 --- a/apps/pdfcraft-cli/tests/it/main.rs +++ b/apps/pdfcraft-cli/tests/it/main.rs @@ -8,6 +8,7 @@ mod check; mod edit; mod extract; mod help; +mod info_repairs; mod info_standards; mod mcp_compact; mod options; diff --git a/apps/pdfcraft-cli/tests/it/options.rs b/apps/pdfcraft-cli/tests/it/options.rs index 7955157e28..d414922998 100644 --- a/apps/pdfcraft-cli/tests/it/options.rs +++ b/apps/pdfcraft-cli/tests/it/options.rs @@ -113,3 +113,14 @@ fn an_argument_that_is_not_utf8_is_an_error_not_a_panic() { assert!(!stderr.contains("panicked"), "{stderr}"); } } + +/// A file `check-one` cannot read is a record on stdout with its own status and the reason, which +/// `check` relays, not an empty stdout that reads as a crash. +#[test] +fn check_one_reports_an_unreadable_file_as_a_record() { + let out = Command::new(BIN).args(["check-one", "missing.pdf"]).output().expect("run cli"); + let stdout = String::from_utf8_lossy(&out.stdout); + let record: serde_json::Value = serde_json::from_str(stdout.trim()).unwrap_or_else(|e| panic!("{e}: {stdout:?}")); + assert_eq!(record["status"], "read-error"); + assert!(record["detail"].as_str().is_some_and(|d| d.contains("missing.pdf")), "{record}"); +} diff --git a/apps/pdfcraft-cli/tests/it/owner_password.rs b/apps/pdfcraft-cli/tests/it/owner_password.rs index 6796980982..5a3e061728 100644 --- a/apps/pdfcraft-cli/tests/it/owner_password.rs +++ b/apps/pdfcraft-cli/tests/it/owner_password.rs @@ -130,6 +130,12 @@ fn render_accepts_the_owner_password() { .output() .unwrap(); let stderr = String::from_utf8_lossy(&out.stderr); + if pw == "user" { + // The user password opens it, but a page image is a copy the permissions forbid. + assert!(!out.status.success() && stderr.contains("don't allow copying"), "{alg:?} with {pw:?}: {stderr}"); + assert!(!png.exists(), "{alg:?}: no image is written"); + continue; + } assert!(out.status.success(), "{alg:?} with {pw:?}: {stderr}"); assert_rendered_page(&std::fs::read(&png).unwrap(), DPI, &format!("{alg:?} with {pw:?}")); } diff --git a/apps/pdfcraft-cli/tests/it/ui_control.rs b/apps/pdfcraft-cli/tests/it/ui_control.rs index f49771b001..082faf3426 100644 --- a/apps/pdfcraft-cli/tests/it/ui_control.rs +++ b/apps/pdfcraft-cli/tests/it/ui_control.rs @@ -86,6 +86,24 @@ fn write_private(path: &Path, text: &str) { opts.mode(0o600); } opts.open(path).unwrap().write_all(text.as_bytes()).unwrap(); + // On Windows the file inherits its folder's access list, as the app's would before it + // restricts it (#3922). + #[cfg(windows)] + { + use pdfcraft_platform::owner_only; + owner_only::restrict_to_owner(path, &owner_only::current_user_sid().unwrap()).unwrap(); + // The file the CLI is handed must already pass its check, or a refusal below would be + // the helper's fault, not the CLI's. + let sddl = owner_only::sddl_of(path).unwrap(); + let listing = |p: &Path| Command::new("icacls").arg(p).output().map(|o| String::from_utf8_lossy(&o.stdout).into_owned()); + let folder = path.parent().map(listing); + assert_eq!( + owner_only::foreign_access(&sddl, &owner_only::current_user_sid().unwrap()), + None, + "restricted file: {sddl}; icacls: {:?}; folder: {folder:?}", + listing(path) + ); + } } fn ui(control: &Path, method: &str) -> Output { diff --git a/apps/pdfcraft-cli/tests/it/xfa.rs b/apps/pdfcraft-cli/tests/it/xfa.rs index 1b5ff7bc9d..4d5df67535 100644 --- a/apps/pdfcraft-cli/tests/it/xfa.rs +++ b/apps/pdfcraft-cli/tests/it/xfa.rs @@ -7,6 +7,11 @@ use std::sync::Arc; const BIN: &str = env!("CARGO_BIN_EXE_pdfcraft-cli"); fn fixture(test: &str) -> PathBuf { + fixture_with(test, "") +} + +/// The two-page form, with `event` inside its `name` field. +fn fixture_with(test: &str, event: &str) -> PathBuf { let xdp = r#" "#; +"# + .replace("{event}", event); let objects = [ "<< /Type /Catalog /Pages 2 0 R /NeedsRendering true /AcroForm << /XFA 5 0 R /Fields [] >> >>".to_string(), "<< /Type /Pages /Kids [3 0 R] /Count 1 >>".into(), @@ -67,6 +73,21 @@ fn info_reports_the_laid_out_pages_and_fields() { assert_eq!(std::fs::read(path).unwrap(), original, "inspection must not write the layout to the input"); } +/// `info` inspects an untrusted file: a form's initialize script, which the app runs on opening +/// and which here would change the field's value, does not run. +#[test] +fn info_runs_none_of_the_forms_scripts() { + let script = r#""#; + let path = fixture_with("info-script", script); + let output = Command::new(BIN).arg("info").arg(&path).output().unwrap(); + success(&output); + let info: serde_json::Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(info["xfa_layout"]["pages"], 2, "the form is still laid out: {info}"); + let warnings = info["warnings"].as_array().unwrap(); + assert!(!warnings.iter().any(|w| w.as_str().is_some_and(|w| w.contains("scripts changed it on opening"))), "{warnings:?}"); + assert_eq!(info["repairs"], serde_json::json!([]), "{info}"); +} + #[test] fn text_reads_the_layout_including_a_page_absent_from_the_placeholder() { let path = fixture("text"); @@ -134,3 +155,21 @@ fn a_laid_out_form_keeps_the_original_copy_permissions() { assert_eq!(info["file_size"], original.len()); assert_eq!(std::fs::read(path).unwrap(), original); } + +/// A damaged dynamic XFA form: `info` lists the cross-reference index rebuilt while opening it, +/// as Document Properties does after the form is laid out as one more revision. +#[test] +fn info_lists_the_repairs_of_a_laid_out_form() { + let path = fixture("info-damaged"); + let bytes = std::fs::read(&path).unwrap(); + let at = bytes.windows(10).rposition(|w| w == b"startxref\n").unwrap() + 10; + let mut damaged = bytes[..at].to_vec(); + damaged.extend_from_slice(b"20\n%%EOF\n"); + std::fs::write(&path, &damaged).unwrap(); + let output = Command::new(BIN).arg("info").arg(&path).output().unwrap(); + success(&output); + let info: serde_json::Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(info["xfa_layout"]["pages"], 2, "the form is still laid out: {info}"); + let repairs = info["repairs"].as_array().unwrap(); + assert!(repairs.iter().any(|l| l.as_str().is_some_and(|l| l.contains("rebuilt cross-reference index"))), "{repairs:?}"); +} diff --git a/apps/pdfcraft/Cargo.toml b/apps/pdfcraft/Cargo.toml index 966af354d1..5e4010fcad 100644 --- a/apps/pdfcraft/Cargo.toml +++ b/apps/pdfcraft/Cargo.toml @@ -14,6 +14,8 @@ workspace = true pdfcraft-ui-egui = { workspace = true } # `build_info`: the version and compile-time build label for `--version` and the log. pdfcraft-engine = { workspace = true } +# `owner_only`: the --control file is restricted to its user on Windows (#3922). +pdfcraft-platform = { workspace = true } # wgpu draws; OpenGL (glow) is the fallback when wgpu can't start on an old or unusual GPU or # driver (#461, #435, #392), and can be chosen with PDFCRAFT_RENDERER=gl. eframe = { workspace = true, features = ["default_fonts", "wgpu", "glow", "accesskit", "wayland", "x11", "persistence", "links", "clipboard"] } @@ -62,6 +64,10 @@ windows = { version = "0.62", default-features = false, features = ["std", "UI_V # socket. `std` has no Unix sockets on Windows, and calling Winsock directly needs unsafe, which # our workspace forbids. Already in the lockfile through zbus; pinned exactly, review before bumping. uds_windows = "=1.2.1" +# Full-resolution taskbar and Alt+Tab icon (`ICON_BIG`, issue #3818). Set both window icons +# directly so eframe cannot replace ICON_BIG with a system-sized copy. Already in the tree +# through eframe and covered by the workspace's winit patch. +winit = { version = "0.30", default-features = false } [build-dependencies] # Icon + VERSIONINFO for pdfcraft.exe; build.rs only uses it when targeting Windows. diff --git a/apps/pdfcraft/build.rs b/apps/pdfcraft/build.rs index 35d519d99e..b6892e7b9d 100644 --- a/apps/pdfcraft/build.rs +++ b/apps/pdfcraft/build.rs @@ -9,6 +9,13 @@ #[path = "src/windows_manifest.rs"] mod windows_manifest; +/// The in-box DLLs only printing uses (Windows.Data.Pdf draws the job, prntvpt reads the +/// driver's print capabilities). Trimmed Windows installations (NTLite) can lack them, and a +/// load-time import of a missing DLL stops the program before `main` (#2630). Delay-loaded, they +/// load on the first print; `pdfcraft_print::spool` checks they exist before calling in. +/// Same list in `apps/pdfcraft{,-cli}/build.rs`, `crates/print/build.rs` and `crates/print/src/spool.rs`. +const DELAY_LOADED: [&str; 2] = ["windows.data.pdf.dll", "prntvpt.dll"]; + fn main() { println!("cargo:rerun-if-changed=build.rs"); println!("cargo:rerun-if-changed=../../assets/app-icon/pdfcraft.ico"); @@ -18,6 +25,12 @@ fn main() { if std::env::var("CARGO_CFG_TARGET_OS").as_deref() != Ok("windows") { return; } + if std::env::var("CARGO_CFG_TARGET_ENV").as_deref() == Ok("msvc") { + for dll in DELAY_LOADED { + println!("cargo:rustc-link-arg-bins=/DELAYLOAD:{dll}"); + } + println!("cargo:rustc-link-arg-bins=delayimp.lib"); + } let mut res = winresource::WindowsResource::new(); res.set_icon("../../assets/app-icon/pdfcraft.ico") .set_icon_with_id("../../assets/app-icon/pdfcraft-document.ico", "2") diff --git a/apps/pdfcraft/src/logging.rs b/apps/pdfcraft/src/logging.rs index bef27de169..4c0f39e85c 100644 --- a/apps/pdfcraft/src/logging.rs +++ b/apps/pdfcraft/src/logging.rs @@ -102,25 +102,12 @@ pub fn timestamp(t: SystemTime) -> String { format!("{y:04}-{m:02}-{d:02}T{:02}:{:02}:{:02}.{:03}Z", rem / 3600, rem % 3600 / 60, rem % 60, since.subsec_millis()) } -/// Days since 1970-01-01 → proleptic Gregorian (year, month, day). -/// -/// `timestamp` clamps a pre-1970 clock to the epoch, so `days` is an unsigned count that can run -/// past `NaiveDate`'s ±262143-year range. The Gregorian calendar repeats every 400 years -/// (146_097 days), so chrono converts the day within one cycle and whole cycles go back into the -/// year (an `i128`, then saturated into `i64`), keeping chrono as the only calendar logic and -/// reproducing the old `civil_from_days` result exactly without hand-rolled day math. 1970-01-01 -/// is `num_days_from_ce` 719_163. +/// Days since 1970-01-01 → proleptic Gregorian (year, month, day): `pdfcraft_engine::civil_from_days`, +/// which keeps the date past chrono's range by whole 400-year cycles. `days` is a `u64` second +/// count over 86_400, at most about 2.1e14, so it always fits an `i64`. fn civil_from_days(days: u64) -> (i64, u32, u32) { - use chrono::Datelike; - let (cycles, rest) = (days / 146_097, days % 146_097); - // `rest + 719_163` is 719_163..865_259, always a valid CE day. - match i32::try_from(rest + 719_163).ok().and_then(chrono::NaiveDate::from_num_days_from_ce_opt) { - Some(d) => { - let y = i128::from(d.year()) + i128::from(cycles) * 400; - (i64::try_from(y).unwrap_or(i64::MAX), d.month(), d.day()) - } - None => (1970, 1, 1), // unreachable - } + // The fallback is unreachable: `days` is at most `u64::MAX / 86_400`, far below `i64::MAX`. + pdfcraft_engine::civil_from_days(i64::try_from(days).unwrap_or(i64::MAX)) } fn numbered(dir: &Path, n: usize) -> PathBuf { diff --git a/apps/pdfcraft/src/main.rs b/apps/pdfcraft/src/main.rs index 4de423db7f..837a803e22 100644 --- a/apps/pdfcraft/src/main.rs +++ b/apps/pdfcraft/src/main.rs @@ -12,6 +12,8 @@ //! --cover on|off --default-layout single|continuous|two-up|two-page --default-continuous on|off --default-zoom fit-width|fit-page| --always-default-view on|off //! --page-gaps on|off --default-page-gaps on|off --recent-files 0..50 //! --combine-view grid|list --combine-zoom 60..200` +//! `--combine files…` adds the files to the Combine files tab (File Explorer's "Combine files in +//! PdfCraft", #585), of the PdfCraft already running when there is one. //! //! `--new-window` opens a window of its own. Without it, on Windows, a launch that only names files //! hands them to the PdfCraft already running, where they open as tabs (`single_instance`). @@ -69,6 +71,7 @@ Options: --two-paged open command-line PDFs in noncontinuous two-page view --new-window open a window of its own instead of passing the files to a running PdfCraft --create-images [images…] put the images in one PDF and ask for the page DPI + --combine files… add the files to the Combine files tab, of a running PdfCraft if there is one --control enable the UI control channel (pdfcraft-cli ui --control …) --page N, --zoom 150, --layout continuous, --theme dark, … view settings applied once the files are open @@ -231,7 +234,7 @@ fn main() -> eframe::Result { } startup.files = rejoin_split_paths(startup.files, |p| std::path::Path::new(p).is_file()); let may_hand_off = startup.may_hand_off(); - let startup::Arguments { files, options, control_file, create_images, two_paged, .. } = startup; + let startup::Arguments { files, options, control_file, create_images, combine, two_paged, .. } = startup; let integrated = cfg!(target_os = "macos"); // GNOME on Wayland leaves the window frame to the app (src/decorations.rs). let window_controls = if cfg!(any(target_os = "macos", target_os = "windows")) { None } else { decorations::app_drawn() }; @@ -255,13 +258,13 @@ fn main() -> eframe::Result { // Absolute for this instance too: links relative to a document resolve against its folder, // never the working directory. let files = single_instance::absolute(&files); - let instance = match single_instance::claim(&files, may_hand_off) { + let instance = match single_instance::claim(&files, may_hand_off, combine) { single_instance::Claim::HandedOff => return Ok(()), single_instance::Claim::Primary(server) => Some(server), single_instance::Claim::Alone => None, }; let choice = renderer_choice(std::env::var("PDFCRAFT_RENDERER").ok().as_deref()); - let launch = Launch { files, options, control_file, create_images, integrated, window_controls, two_paged }; + let launch = Launch { files, options, control_file, create_images, combine, integrated, window_controls, two_paged }; // Finder, Open With and the Dock deliver files as Apple events, not arguments; catch the one // that launched us as well as later ones. Lives until the event loop returns. #[cfg(target_os = "macos")] @@ -337,7 +340,15 @@ fn native_options(integrated: bool, app_frame: bool, renderer: eframe::Renderer) .with_drag_and_drop(true) // Wayland app id: matches packaging/linux/ai.storyteller.pdfcraft.desktop. .with_app_id(APP_ID); + // eframe replaces ICON_BIG with a system-sized icon on its first active frame. + // An explicitly empty icon disables that setter (None selects eframe's default). + // Windows installs both sizes directly through winit in app_creator instead. + #[cfg(target_os = "windows")] + { + viewport = viewport.with_icon(egui::IconData::default()); + } // Dock, taskbar, Alt-Tab and launcher icon when running unbundled. + #[cfg(not(target_os = "windows"))] match eframe::icon_data::from_png_bytes(APP_ICON_PNG) { Ok(icon) => viewport = viewport.with_icon(icon), Err(e) => log::warn!("app icon: {e}"), @@ -368,6 +379,8 @@ struct Launch { options: Vec<(String, String)>, control_file: Option, create_images: bool, + /// `--combine`: the files go to the Combine files tab. + combine: bool, integrated: bool, /// The window buttons the app draws, when the desktop leaves the frame to it. window_controls: Option, @@ -386,13 +399,55 @@ fn native_wayland(_: &eframe::CreationContext<'_>) -> bool { false } +/// Decode `png` as a [`winit::window::Icon`] for the Windows taskbar and Alt+Tab (`ICON_BIG`). +#[cfg(target_os = "windows")] +fn taskbar_icon(png: &[u8]) -> Option { + let icon = match eframe::icon_data::from_png_bytes(png) { + Ok(icon) => icon, + Err(e) => { + log::warn!("taskbar icon: {e}"); + return None; + } + }; + match winit::window::Icon::from_rgba(icon.rgba, icon.width, icon.height) { + Ok(icon) => Some(icon), + Err(e) => { + log::warn!("taskbar icon: {e}"); + None + } + } +} + +/// Install the decoded [`winit::window::Icon`] from `png` via `set` when decoding succeeds. +#[cfg(target_os = "windows")] +fn install_taskbar_icon(png: &[u8], set: impl FnOnce(winit::window::Icon)) { + if let Some(icon) = taskbar_icon(png) { + set(icon); + } +} + +/// Install both window icons before the first frame. eframe's icon setter is disabled in +/// `native_options`, so it cannot replace the full-resolution taskbar/Alt+Tab icon (#3818). +#[cfg(target_os = "windows")] +fn set_taskbar_icon(window: &winit::window::Window) { + use winit::platform::windows::{IconExtWindows as _, WindowExtWindows as _}; + install_taskbar_icon(APP_ICON_PNG, |icon| window.set_taskbar_icon(Some(icon))); + // The title bar's icon: the small image drawn for this scale in the exe's own icon (build.rs). + let side = (16.0 * window.scale_factor()).round().clamp(16.0, 256.0) as u32; + match winit::window::Icon::from_resource(1, Some(winit::dpi::PhysicalSize::new(side, side))) { + Ok(icon) => window.set_window_icon(Some(icon)), + Err(e) => log::warn!("window icon: {e}"), + } +} + +/// Build the [`eframe::AppCreator`] closure that initializes [`PdfCraftApp`] once the window and renderer are ready. fn app_creator<'a>( launch: Launch, started: Rc>, instance: Option<&'a single_instance::Server>, #[cfg(target_os = "macos")] apple_events: &'a apple_events::AppleEvents, ) -> eframe::AppCreator<'a> { - let Launch { files, options, control_file, create_images, integrated, window_controls, two_paged } = launch; + let Launch { files, options, control_file, create_images, combine, integrated, window_controls, two_paged } = launch; Box::new(move |cc| { started.set(true); let mut app = PdfCraftApp::new(); @@ -413,6 +468,9 @@ fn app_creator<'a>( // it is set to On or Off. #[cfg(target_os = "windows")] { + if let Some(window) = cc.winit_window() { + set_taskbar_icon(window); + } app.motion.source = Some(motion::source()); } app.set_startup_update_check(true); @@ -466,6 +524,9 @@ fn app_creator<'a>( if let Err(e) = app.begin_image_import_paths(&files) { app.notify(e); } + } else if combine { + app.reopen_last_files(&[]); + app.combine_paths(&files); } else { // With the preference on, last session's files come back first; files named on // the command line open after them, in front (#442). @@ -506,7 +567,9 @@ fn open_control_channel(file: &str, endpoint: std::io::Result { - let message = format!("--control {file}: {e}. Use a control file in a folder only you can write, such as ~/.pdfcraft-control.json"); + // cmd.exe doesn't expand `~` (#3922). + let example = if cfg!(windows) { r"%USERPROFILE%\.pdfcraft-control.json" } else { "~/.pdfcraft-control.json" }; + let message = format!("--control {file}: {e}. Use a control file in a folder only you can write, such as {example}"); log::error!("{message}"); Err(message) } @@ -515,15 +578,25 @@ fn open_control_channel(file: &str, endpoint: std::io::Result std::io::Result<()> { - use std::io::{ErrorKind, Write}; + use std::io::ErrorKind; let json = serde_json::json!({ "port": port, "token": token, "pid": std::process::id() }).to_string(); let path = std::path::Path::new(path); let name = path.file_name().ok_or_else(|| std::io::Error::new(ErrorKind::InvalidInput, "not a file name"))?; + // Windows: the user's SID, read once for the private folder and for the file in it, since each + // lookup starts `whoami` (review on #3956). + #[cfg(windows)] + let me = pdfcraft_platform::owner_only::current_user_sid()?; + #[cfg(windows)] + let restrict = |file: &std::path::Path| pdfcraft_platform::owner_only::restrict_to_owner(file, &me); + #[cfg(not(windows))] + let restrict = |_: &std::path::Path| Ok::<(), std::io::Error>(()); // Unpredictable suffixes, so the name can't be planted in advance. `RandomState` is keyed from // the operating system's random source. let random = std::hash::RandomState::new(); @@ -532,35 +605,79 @@ fn write_control_file(path: &str, port: u16, token: &str) -> std::io::Result<()> staged.push(name); staged.push(format!(".{:016x}.tmp", std::hash::BuildHasher::hash_one(&random, i))); let staged = path.with_file_name(staged); - let mut opts = std::fs::OpenOptions::new(); - opts.write(true).create_new(true); - #[cfg(unix)] - { - use std::os::unix::fs::OpenOptionsExt; - opts.mode(0o600); - } - let file = match opts.open(&staged) { - Ok(f) => f, - Err(e) if e.kind() == ErrorKind::AlreadyExists => continue, - // Windows refuses `create_new` on a folder with "access denied"; the name is taken all - // the same. A folder we can't write to, with nothing at the name, still fails here. - Err(e) if e.kind() == ErrorKind::PermissionDenied && staged.symlink_metadata().is_ok() => continue, - Err(e) => return Err(e), + // Windows: a file created next to `path` would inherit that folder's access list, which may + // let other users read it, and restricting it afterwards leaves a window in which one of + // them opens a handle; Windows checks access only when a handle is opened, so a later + // change of the list doesn't revoke it. The file is created in a private folder instead, + // and that folder is removed once the file has left it. + #[cfg(windows)] + let (staging, staged) = { + match pdfcraft_platform::owner_only::create_private_dir(&staged, &me) { + Ok(()) => {} + Err(e) if e.kind() == ErrorKind::AlreadyExists => continue, + // As for a file below: something at the name we may not open still takes it. + Err(e) if e.kind() == ErrorKind::PermissionDenied && staged.symlink_metadata().is_ok() => continue, + Err(e) => return Err(e), + } + let file = staged.join(name); + (staged, file) }; - // The block closes the file before it is renamed. - let written = { - let mut file = file; - file.write_all(json.as_bytes()) - }; - let done = written.and_then(|()| std::fs::rename(&staged, path)); - if done.is_err() { - let _ = std::fs::remove_file(&staged); + let done = stage_control_file(&staged, path, &json, &restrict); + #[cfg(windows)] + let _ = std::fs::remove_dir(&staging); + match done { + Some(done) => return done, + None => continue, } - return done; } Err(std::io::Error::new(ErrorKind::AlreadyExists, "no free name for the control file's temporary copy")) } +/// Create `staged` fresh and owner-only, `restrict` it, write `json` to it and rename it to `path`. +/// `None` when something already has the name, so the caller tries another. On any other failure +/// the new file is removed and nothing is left at `path`'s side. +fn stage_control_file( + staged: &std::path::Path, + path: &std::path::Path, + json: &str, + restrict: &dyn Fn(&std::path::Path) -> std::io::Result<()>, +) -> Option> { + use std::io::{ErrorKind, Write}; + let mut opts = std::fs::OpenOptions::new(); + opts.write(true).create_new(true); + #[cfg(unix)] + { + use std::os::unix::fs::OpenOptionsExt; + opts.mode(0o600); + } + let file = match opts.open(staged) { + Ok(f) => f, + Err(e) if e.kind() == ErrorKind::AlreadyExists => return None, + // Windows refuses `create_new` on a folder with "access denied"; the name is taken all + // the same. A folder we can't write to, with nothing at the name, still fails here. + Err(e) if e.kind() == ErrorKind::PermissionDenied && staged.symlink_metadata().is_ok() => return None, + Err(e) => return Some(Err(e)), + }; + // Windows: the file inherited the private folder's list. It also gets a list of its own, so it + // stays private wherever the rename puts it, and that is checked before the token is written; + // nothing is written if that fails (#3922). Unix has mode 0600 above, and `restrict` does nothing. + if let Err(e) = restrict(staged) { + drop(file); + let _ = std::fs::remove_file(staged); + return Some(Err(e)); + } + // The block closes the file before it is renamed. + let written = { + let mut file = file; + file.write_all(json.as_bytes()) + }; + let done = written.and_then(|()| std::fs::rename(staged, path)); + if done.is_err() { + let _ = std::fs::remove_file(staged); + } + Some(done) +} + /// How wgpu finds a GPU. Each choice yields to its wgpu environment variable. /// /// - Draw on the integrated GPU unless `WGPU_POWER_PREF` says otherwise. A PDF viewer has no use diff --git a/apps/pdfcraft/src/single_instance.rs b/apps/pdfcraft/src/single_instance.rs index 751785d0c5..60f5d1d493 100644 --- a/apps/pdfcraft/src/single_instance.rs +++ b/apps/pdfcraft/src/single_instance.rs @@ -4,8 +4,9 @@ //! Outlook attachments, Explorer double-clicks and Open With all start `pdfcraft.exe ""`, and //! a file manager or `xdg-open` starts `pdfcraft ` the same way, so until now every file got //! its own process and window. The first PdfCraft listens on a Unix domain socket. On Windows (10 -//! 1803 and later) it is in `%LOCALAPPDATA%\PdfCraft`, or in the data folder of a portable copy, -//! and is named after the Windows session (`instance-Console.sock`), so two sign-ins of the same +//! 1803 and later) it is in `%LOCALAPPDATA%\PdfCraft` (a portable copy's too, as +//! `portable-instance-…` and never in its own folder, #889) and is named after the Windows +//! session (`instance-s1.sock` for session 1), so two sign-ins of the same //! user never open files in each other's windows. On Linux and the BSDs it is in //! `$XDG_RUNTIME_DIR/pdfcraft` and is named after the display (`instance-wayland-0.sock`), so a //! launch on another display or seat opens its own window there. Either way it lives in a folder @@ -13,7 +14,9 @@ //! intercept a launch or open files in this user's window. Inside a Flatpak, `$XDG_RUNTIME_DIR` is //! the app's own and is shared by every run of it, so the handoff works there too. The exchange is //! one line each way, a JSON request naming absolute paths and `ok`, and the running app opens the -//! paths exactly as File ▸ Open would. +//! paths exactly as File ▸ Open would. A `--combine` launch (File Explorer's "Combine files in +//! PdfCraft", #585, one launch per selected file) hands them over the same way with +//! `"combine": true`, and the running app adds them to its Combine files tab instead. //! //! Every failure falls back to the old behaviour: a launch that can't hand its files over within //! a few seconds opens its own window. `--new-window`, any other option, or no files at all also @@ -57,7 +60,10 @@ pub enum Claim { /// Files that later launches handed over, and how to wake the UI for them. #[derive(Default)] struct Inbox { - paths: Vec, + /// The requests in the order they arrived: [`OsEvent::Open`] or, for `--combine`, + /// [`OsEvent::Combine`]. One queue, so the UI handles an Open that came after a Combine after + /// it, and that Open comes to the front instead of opening behind the Combine files tab. + events: Vec, wake: Option>, } @@ -73,11 +79,20 @@ impl Server { Self { inbox: Arc::default() } } - /// Queue files from another launch and wake the UI. + /// Queue files from another launch (to open, or to combine) and wake the UI. #[cfg_attr(not(any(windows, all(unix, not(target_os = "macos")))), allow(dead_code))] - fn deliver(&self, paths: Vec) { + fn deliver(&self, paths: Vec, combine: bool) { + if paths.is_empty() { + return; + } let mut inbox = self.inbox.lock().unwrap_or_else(PoisonError::into_inner); - inbox.paths.extend(paths); + // A run of requests of one kind (File Explorer starts one launch per selected file) stays + // one event; a request of the other kind starts a new one after it. + match (inbox.events.last_mut(), combine) { + (Some(OsEvent::Combine(queued)), true) | (Some(OsEvent::Open(queued)), false) => queued.extend(paths), + (_, true) => inbox.events.push(OsEvent::Combine(paths)), + (_, false) => inbox.events.push(OsEvent::Open(paths)), + } if let Some(wake) = &inbox.wake { wake(); } @@ -94,8 +109,8 @@ impl Server { let inbox = Arc::clone(&self.inbox); let ctx = ctx.clone(); Box::new(move || { - let paths = std::mem::take(&mut inbox.lock().unwrap_or_else(PoisonError::into_inner).paths); - if paths.is_empty() { + let events = std::mem::take(&mut inbox.lock().unwrap_or_else(PoisonError::into_inner).events); + if events.is_empty() { return Vec::new(); } ctx.send_viewport_cmd(egui::ViewportCommand::Minimized(false)); @@ -103,27 +118,37 @@ impl Server { if native_wayland { ctx.send_viewport_cmd(egui::ViewportCommand::RequestUserAttention(egui::UserAttentionType::Informational)); } - vec![OsEvent::Open(paths)] + events }) } } -/// The request a launch sends: `{"open":[…]}` and a newline. +/// The request a launch sends: `{"open":[…]}` (with `"combine":true` for `--combine`) and a +/// newline. #[cfg_attr(not(any(windows, all(unix, not(target_os = "macos")))), allow(dead_code))] -pub fn request(paths: &[String]) -> String { - let mut line = serde_json::json!({ "open": paths }).to_string(); +pub fn request(paths: &[String], combine: bool) -> String { + let mut value = serde_json::json!({ "open": paths }); + if combine { + value["combine"] = serde_json::Value::Bool(true); + } + let mut line = value.to_string(); line.push('\n'); line } -/// The paths a request names, or `None` when it isn't a request: anything but one JSON object -/// whose `open` is a non-empty list of non-empty strings. +/// The paths a request names and whether they are for Combine files, or `None` when it isn't a +/// request: anything but one JSON object whose `open` is a non-empty list of non-empty strings +/// (`combine`, when there, must be a boolean). #[cfg_attr(not(any(windows, all(unix, not(target_os = "macos")))), allow(dead_code))] -pub fn parse_request(line: &str) -> Option> { +pub fn parse_request(line: &str) -> Option<(Vec, bool)> { let value: serde_json::Value = serde_json::from_str(line.trim_end()).ok()?; let open = value.get("open")?.as_array()?; let paths = open.iter().map(|p| p.as_str().filter(|s| !s.is_empty()).map(str::to_string)).collect::>>()?; - (!paths.is_empty()).then_some(paths) + let combine = match value.get("combine") { + None => false, + Some(c) => c.as_bool()?, + }; + (!paths.is_empty()).then_some((paths, combine)) } /// File arguments made absolute against this launch's working folder, which the running app @@ -133,18 +158,19 @@ pub fn absolute(paths: &[String]) -> Vec { } /// Decide what this launch does. `files` are absolute ([`absolute`]); `may_hand_off` is false -/// when the launch asked for anything besides opening files, so it gets its own window. -pub fn claim(files: &[String], may_hand_off: bool) -> Claim { +/// when the launch asked for anything besides opening (or, with `combine`, combining) files, so +/// it gets its own window. +pub fn claim(files: &[String], may_hand_off: bool, combine: bool) -> Claim { #[cfg(any(windows, all(unix, not(target_os = "macos"))))] { match socket_path() { - Some(path) => claim_at(&path, files, may_hand_off), + Some(path) => claim_at(&path, files, may_hand_off, combine), None => Claim::Alone, } } #[cfg(not(any(windows, all(unix, not(target_os = "macos")))))] { - let _ = (files, may_hand_off); + let _ = (files, may_hand_off, combine); Claim::Alone } } @@ -238,7 +264,7 @@ fn election(socket: &std::path::Path) -> Option { /// [`claim`] with the socket at `path`. #[cfg(any(windows, all(unix, not(target_os = "macos"))))] -fn claim_at(path: &std::path::Path, files: &[String], may_hand_off: bool) -> Claim { +fn claim_at(path: &std::path::Path, files: &[String], may_hand_off: bool, combine: bool) -> Claim { // Released when this function returns: by then this launch listens, or has handed over. let _election = election(path); // Two launches can start together: the one that loses the race to bind hands over to the @@ -249,7 +275,7 @@ fn claim_at(path: &std::path::Path, files: &[String], may_hand_off: bool) -> Cla if !may_hand_off { return Claim::Alone; } - return match hand_off(&stream, files) { + return match hand_off(&stream, files, combine) { Ok(()) => Claim::HandedOff, Err(e) => { log::warn!("couldn't hand the files to the running PdfCraft ({e}); opening a new window"); @@ -295,12 +321,12 @@ fn claim_at(path: &std::path::Path, files: &[String], may_hand_off: bool) -> Cla /// Send `files` over `stream` and wait for the running app's `ok`. #[cfg(any(windows, all(unix, not(target_os = "macos"))))] -fn hand_off(stream: &UnixStream, files: &[String]) -> std::io::Result<()> { +fn hand_off(stream: &UnixStream, files: &[String], combine: bool) -> std::io::Result<()> { use std::io::{BufRead, Read, Write}; stream.set_read_timeout(Some(TIMEOUT))?; stream.set_write_timeout(Some(TIMEOUT))?; let mut writer = stream; - writer.write_all(request(files).as_bytes())?; + writer.write_all(request(files, combine).as_bytes())?; let mut reply = String::new(); std::io::BufReader::new(stream.take(16)).read_line(&mut reply)?; if reply == "ok\n" { Ok(()) } else { Err(std::io::Error::other("the running PdfCraft didn't confirm")) } @@ -342,8 +368,8 @@ fn receive(stream: &UnixStream, server: &Server) -> std::io::Result<()> { if line.is_empty() { return Ok(()); } - let paths = parse_request(&line).ok_or_else(|| std::io::Error::new(std::io::ErrorKind::InvalidData, "not an open request"))?; - server.deliver(paths); + let (paths, combine) = parse_request(&line).ok_or_else(|| std::io::Error::new(std::io::ErrorKind::InvalidData, "not an open request"))?; + server.deliver(paths, combine); let mut writer = stream; writer.write_all(b"ok\n") } @@ -355,16 +381,31 @@ mod tests { #[test] fn a_request_names_the_paths_and_round_trips() { let paths = vec![r"C:\Users\me\Documents\Quote.pdf".to_string(), r"D:\Scans\Plan „A“ 1.pdf".to_string()]; - let line = request(&paths); + let line = request(&paths, false); assert!(line.ends_with('\n') && !line.trim_end().contains('\n'), "one line"); - assert_eq!(parse_request(&line), Some(paths)); + assert_eq!(parse_request(&line), Some((paths.clone(), false))); + assert!(!line.contains("combine"), "an open request says nothing about combining"); + let line = request(&paths, true); + assert_eq!(parse_request(&line), Some((paths, true)), "a --combine launch's request"); } #[test] fn anything_but_a_list_of_paths_is_refused() { - for line in ["", "ok\n", "{}", r#"{"open":[]}"#, r#"{"open":[""]}"#, r#"{"open":["a.pdf",3]}"#, r#"{"open":"a.pdf"}"#, "[\"a.pdf\"]"] { + for line in [ + "", + "ok\n", + "{}", + r#"{"open":[]}"#, + r#"{"open":[""]}"#, + r#"{"open":["a.pdf",3]}"#, + r#"{"open":"a.pdf"}"#, + "[\"a.pdf\"]", + r#"{"open":["a.pdf"],"combine":"yes"}"#, + r#"{"open":["a.pdf"],"combine":1}"#, + ] { assert_eq!(parse_request(line), None, "{line:?}"); } + assert_eq!(parse_request(r#"{"open":["a.pdf"],"combine":false}"#), Some((vec!["a.pdf".to_string()], false))); } #[test] @@ -397,13 +438,31 @@ mod tests { fn delivered_files_reach_the_ui_once_wake_it_and_bring_the_window_forward() { let server = Server::new(); // Before the window exists: queued, nothing to wake yet. - server.deliver(vec!["a.pdf".into()]); + server.deliver(vec!["a.pdf".into()], false); let ctx = egui::Context::default(); let mut poll = server.connect(&ctx, false); - server.deliver(vec!["b.pdf".into()]); + server.deliver(vec!["b.pdf".into()], false); let events = poll(); assert!(matches!(events.as_slice(), [OsEvent::Open(p)] if p == &["a.pdf", "b.pdf"])); assert!(poll().is_empty(), "each file is delivered once"); + // Files to combine (one launch each, from File Explorer) arrive as one Combine event, + // apart from files to open. + server.deliver(vec!["c.pdf".into()], true); + server.deliver(vec!["e.pdf".into()], true); + let events = poll(); + assert!(matches!(events.as_slice(), [OsEvent::Combine(c)] if c == &["c.pdf", "e.pdf"]), "{events:?}"); + // #2503 review: requests reach the UI in the order they arrived, so an Open after a + // Combine is handled after it and comes to the front instead of opening behind the + // Combine files tab. + server.deliver(vec!["f.pdf".into()], true); + server.deliver(vec!["g.pdf".into()], false); + server.deliver(vec!["h.pdf".into()], true); + let events = poll(); + assert!( + matches!(events.as_slice(), [OsEvent::Combine(f), OsEvent::Open(g), OsEvent::Combine(h)] if f == &["f.pdf"] && g == &["g.pdf"] && h == &["h.pdf"]), + "{events:?}" + ); + assert!(poll().is_empty()); } #[test] @@ -441,12 +500,15 @@ mod tests { #[test] fn a_second_launch_hands_its_files_to_the_first() { let path = socket("hand-off"); - let Claim::Primary(first) = claim_at(&path, &[], true) else { panic!("the first launch listens") }; + let Claim::Primary(first) = claim_at(&path, &[], true, false) else { panic!("the first launch listens") }; let mut poll = first.connect(&egui::Context::default(), false); - assert!(matches!(claim_at(&path, &["C:\\a.pdf".into()], true), Claim::HandedOff)); + assert!(matches!(claim_at(&path, &["C:\\a.pdf".into()], true, false), Claim::HandedOff)); assert!(matches!(poll().as_slice(), [OsEvent::Open(p)] if p == &["C:\\a.pdf"])); + // A --combine launch hands its file over for the Combine files tab (#585). + assert!(matches!(claim_at(&path, &["C:\\c.pdf".into()], true, true), Claim::HandedOff)); + assert!(matches!(poll().as_slice(), [OsEvent::Combine(p)] if p == &["C:\\c.pdf"])); // A launch with options keeps its own window and doesn't take over the socket. - assert!(matches!(claim_at(&path, &["C:\\b.pdf".into()], false), Claim::Alone)); + assert!(matches!(claim_at(&path, &["C:\\b.pdf".into()], false, false), Claim::Alone)); assert!(poll().is_empty()); } @@ -461,7 +523,7 @@ mod tests { let (path, barrier) = (path.clone(), Arc::clone(&barrier)); std::thread::spawn(move || { barrier.wait(); - claim_at(&path, &[format!("C:\\{i}.pdf")], true) + claim_at(&path, &[format!("C:\\{i}.pdf")], true, false) }) }) .collect(); @@ -482,6 +544,6 @@ mod tests { })); // `symlink_metadata`, not `exists`, which follows the socket's reparse point and can fail. assert!(std::fs::symlink_metadata(&path).is_ok(), "the socket file stays behind"); - assert!(matches!(claim_at(&path, &["C:\\a.pdf".into()], true), Claim::Primary(_))); + assert!(matches!(claim_at(&path, &["C:\\a.pdf".into()], true, false), Claim::Primary(_))); } } diff --git a/apps/pdfcraft/src/startup.rs b/apps/pdfcraft/src/startup.rs index be25da45b6..23db00a7b7 100644 --- a/apps/pdfcraft/src/startup.rs +++ b/apps/pdfcraft/src/startup.rs @@ -7,6 +7,8 @@ pub(super) struct Arguments { pub options: Vec<(String, String)>, pub control_file: Option, pub create_images: bool, + /// `--combine`: the files go to the Combine files tab. + pub combine: bool, pub new_window: bool, pub two_paged: bool, pub version: bool, @@ -33,6 +35,9 @@ impl Arguments { } "--control" => parsed.control_file = args.next(), "--create-images" => parsed.create_images = true, + // File Explorer's "Combine files in PdfCraft" (#585): the files go to the Combine + // files tab, of the running PdfCraft when there is one. + "--combine" => parsed.combine = true, "--new-window" => parsed.new_window = true, "--two-paged" => parsed.two_paged = true, flag if flag.starts_with("--two-paged=") => { diff --git a/apps/pdfcraft/src/tests.rs b/apps/pdfcraft/src/tests.rs index 3512f46b36..7c1f111b96 100644 --- a/apps/pdfcraft/src/tests.rs +++ b/apps/pdfcraft/src/tests.rs @@ -137,6 +137,31 @@ fn the_desktop_frame_is_turned_off_only_when_the_app_draws_one() { assert_eq!(native_options(false, true, eframe::Renderer::Glow).viewport.decorations, Some(false)); } +#[test] +fn the_window_and_windows_taskbar_icons_load_from_the_bundled_png() { + let icon = eframe::icon_data::from_png_bytes(super::APP_ICON_PNG).expect("the bundled PNG decodes as the window icon"); + assert!(icon.width > 0 && icon.height > 0); + assert_eq!(icon.rgba.len(), (icon.width as usize) * (icon.height as usize) * 4); + #[cfg(not(target_os = "windows"))] + assert_eq!(super::native_options(false, false, eframe::Renderer::Glow).viewport.icon.as_deref(), Some(&icon)); + #[cfg(target_os = "windows")] + { + // None would load eframe's default icon; only the explicit empty sentinel prevents + // its first-frame setter from overwriting our full-resolution ICON_BIG. + for renderer in [eframe::Renderer::Glow, eframe::Renderer::Wgpu] { + let viewport = super::native_options(false, false, renderer).viewport; + assert_eq!(viewport.icon.as_deref(), Some(&egui::IconData::default())); + } + assert_eq!((icon.width, icon.height), (256, 256)); + let mut installed = None; + super::install_taskbar_icon(super::APP_ICON_PNG, |icon| installed = Some(icon)); + assert!(installed.is_some(), "issue #3818: ICON_BIG loads and installs from APP_ICON_PNG"); + let mut bad = None; + super::install_taskbar_icon(b"not a png", |icon| bad = Some(icon)); + assert!(bad.is_none()); + } +} + #[test] fn a_software_renderer_is_shown_to_the_user() { use eframe::wgpu::DeviceType; @@ -654,6 +679,42 @@ fn the_control_file_replaces_the_one_a_previous_run_left() { let _ = std::fs::remove_dir_all(&dir); } +/// #3922: a control file created in a folder that lets every user read what's in it is still +/// readable by its user and SYSTEM only, like mode 0600 on Unix. +#[cfg(windows)] +#[test] +fn the_control_file_is_owner_only_in_a_folder_other_users_can_read() { + use pdfcraft_platform::owner_only; + let dir = scratch(); + let users = std::process::Command::new("icacls").arg(&dir).arg("/grant").arg("*S-1-5-32-545:(OI)(CI)R").output().unwrap(); + assert!(users.status.success()); + // What the folder hands down, for comparison: a plain file there is readable by Users. + std::fs::write(dir.join("plain.txt"), "x").unwrap(); + let me = owner_only::current_user_sid().unwrap(); + assert!(owner_only::foreign_access(&owner_only::sddl_of(&dir.join("plain.txt")).unwrap(), &me).is_some()); + super::write_control_file(&dir.join("ctl.json").to_string_lossy(), 4242, "t0ken").unwrap(); + let sddl = owner_only::sddl_of(&dir.join("ctl.json")).unwrap(); + assert_eq!(owner_only::foreign_access(&sddl, &me), None, "{sddl}"); + let (json, names) = written(&dir); + assert_eq!(json["token"].as_str(), Some("t0ken")); + assert_eq!(names, ["ctl.json", "plain.txt"], "no private staging folder left behind"); + let _ = std::fs::remove_dir_all(&dir); +} + +/// Review on #3956: writing the control file reads the user's SID once, for both the private +/// folder and the file in it, rather than starting `whoami` for each. +#[cfg(windows)] +#[test] +fn writing_the_control_file_looks_up_the_users_sid_once() { + use pdfcraft_platform::owner_only::sid_lookups_on_this_thread; + let dir = scratch(); + let before = sid_lookups_on_this_thread(); + super::write_control_file(&dir.join("ctl.json").to_string_lossy(), 4242, "t0ken").unwrap(); + assert_eq!(sid_lookups_on_this_thread().saturating_sub(before), 1, "whoami runs once per control file"); + assert_eq!(written(&dir).0["token"].as_str(), Some("t0ken")); + let _ = std::fs::remove_dir_all(&dir); +} + /// X11 startup notification and docks match a window to its launcher by `StartupWMClass`, /// so it has to be the class the window carries: the app id, on Wayland and X11 alike. #[test] diff --git a/crates/a11y/src/alt.rs b/crates/a11y/src/alt.rs index 55c0857878..29e48409ba 100644 --- a/crates/a11y/src/alt.rs +++ b/crates/a11y/src/alt.rs @@ -106,7 +106,13 @@ pub(crate) fn elements(doc: &Document) -> Vec { if !seen.insert(r) { continue; } - let Some(d) = doc.get(r).as_dict().cloned() else { continue }; + let target = doc.get(r); + if matches!(&*target, Object::Array(_) | Object::Ref(_)) { + // An indirect /K array is a container, not an element or a parent. + stack.push(((*target).clone(), pg, parent, depth)); + continue; + } + let Some(d) = target.as_dict().cloned() else { continue }; if d.contains(b"S") { let pg = d.get(b"Pg").and_then(Object::as_ref).or(pg); if let Some(kids) = d.get(b"K") { diff --git a/crates/a11y/src/autotag.rs b/crates/a11y/src/autotag.rs index 5c0dc2bd19..59d30930ff 100644 --- a/crates/a11y/src/autotag.rs +++ b/crates/a11y/src/autotag.rs @@ -1,14 +1,21 @@ -//! Create a valid, reviewable structure tree for PDFs whose painted content is text only. +//! Create a valid, reviewable structure tree for untagged PDFs. //! -//! The writer refuses mixed content and existing tags. Each text object becomes a paragraph, -//! which can then be retagged with the existing tag-property tool after human review. +//! Each text object becomes a paragraph, which can then be retagged with the existing +//! tag-property tool after human review. Painted paths, shadings and images outside text objects +//! are wrapped in `/Artifact BMC … EMC` so the resulting page passes the tagged-content check. -use pdfcraft_content::{Op, parse, serialize_ops}; +use std::collections::{HashMap, HashSet}; + +use pdfcraft_content::{Op, Pieces, serialize_ops}; use pdfcraft_cos::{DecodeBudget, Dict, Document, ObjRef, Object, Stream}; const MAX_PAGES: usize = 1_000; const MAX_PAGE_BYTES: usize = 16 * 1024 * 1024; const MAX_OPERATORS: usize = 100_000; +/// What all pages together may decode and keep: pages can share one stream, so the per-page limits +/// alone let a small file retain a plan for every page. Real documents stay far under these. +const MAX_TOTAL_BYTES: usize = 128 * 1024 * 1024; +const MAX_TOTAL_OPERATORS: usize = 2_000_000; const MAX_TEXT_OBJECTS: usize = 4_096; #[derive(Clone, Copy, Debug, PartialEq, Eq)] @@ -37,25 +44,27 @@ struct Plan { page: ObjRef, ops: Vec, spans: Vec<(usize, usize)>, + artifacts: HashMap, } fn unsupported(page: usize, reason: impl Into) -> AutoTagError { AutoTagError::Unsupported { page: page + 1, reason: reason.into() } } -fn content_bytes(doc: &Document, page: &pdfcraft_model::Page, index: usize) -> Result, AutoTagError> { - let Some(contents) = page.dict.get(b"Contents") else { return Ok(Vec::new()) }; +fn content_pieces(doc: &Document, page: &pdfcraft_model::Page, index: usize) -> Result { + let Some(contents) = page.dict.get(b"Contents") else { return Ok(Pieces::join::<&[u8]>(&[])) }; let resolved = doc.resolve(contents); - let pieces: Vec = match &*resolved { + let items: Vec = match &*resolved { Object::Array(items) => items.clone(), Object::Stream(_) => vec![contents.clone()], _ => return Err(unsupported(index, "contents are not a stream or array of streams")), }; - let mut all = Vec::new(); - for item in pieces { + let mut pieces = Vec::with_capacity(items.len()); + let mut total = 0usize; + for item in items { let value = doc.resolve(&item); let Object::Stream(stream) = &*value else { return Err(unsupported(index, "contents include a non-stream object")) }; - let remaining = MAX_PAGE_BYTES.saturating_sub(all.len().saturating_add(1)); + let remaining = MAX_PAGE_BYTES.saturating_sub(total.saturating_add(1)); if remaining == 0 { return Err(unsupported(index, "decoding the content takes more than 16 MiB")); } @@ -75,10 +84,18 @@ fn content_bytes(doc: &Document, page: &pdfcraft_model::Page, index: usize) -> R unsupported(index, format!("content cannot be decoded strictly: {why}")) } })?; - all.extend_from_slice(&decoded); - all.push(b' '); + total = total.saturating_add(decoded.len()).saturating_add(1); + pieces.push(decoded); } - Ok(all) + Ok(Pieces::join(&pieces)) +} + +fn page_xobjects(doc: &Document, page: &Dict) -> Dict { + page.get(b"Resources") + .map(|r| doc.resolve(r)) + .and_then(|r| r.as_dict().and_then(|r| r.get(b"XObject").map(|x| doc.resolve(x)))) + .and_then(|x| x.as_dict().cloned()) + .unwrap_or_default() } fn harmless_outside(op: &[u8]) -> bool { @@ -106,9 +123,35 @@ fn harmless_outside(op: &[u8]) -> bool { | b"sc" | b"SCN" | b"scn" + | b"m" + | b"l" + | b"c" + | b"v" + | b"y" + | b"h" + | b"re" + | b"W" + | b"W*" + | b"n" ) } +fn painted_outside(doc: &Document, xobjects: &Dict, op: &Op) -> bool { + match op.op.as_slice() { + b"f" | b"F" | b"f*" | b"B" | b"B*" | b"b" | b"b*" | b"S" | b"s" | b"sh" | b"BI" => true, + b"Do" => op + .name(0) + .and_then(|n| xobjects.get(n)) + .map(|o| doc.resolve(o)) + .and_then(|o| match &*o { + Object::Stream(s) => s.dict.name(b"Subtype").map(<[u8]>::to_vec), + _ => None, + }) + .is_some_and(|st| st == b"Image"), + _ => false, + } +} + fn harmless_inside(op: &[u8]) -> bool { matches!( op, @@ -138,17 +181,38 @@ fn harmless_inside(op: &[u8]) -> bool { ) } -/// The `(BT, ET)` operator index pairs of each text object that shows text, or why the page cannot be tagged. -fn text_spans(ops: &[Op], index: usize) -> Result, AutoTagError> { - let mut start = None; - let mut has_text = false; - let mut spans = Vec::new(); - for (at, op) in ops.iter().enumerate() { +/// `totals` is the decoded bytes and operators of the pages planned so far, bounded by +/// `MAX_TOTAL_BYTES` and `MAX_TOTAL_OPERATORS`. +fn plan_page(doc: &Document, page: &pdfcraft_model::Page, index: usize, totals: &mut (usize, usize)) -> Result { + if page.dict.contains(b"StructParents") { + return Err(unsupported(index, "page already has a structure parent key")); + } + if page.dict.get(b"Annots").is_some_and(|a| match &*doc.resolve(a) { + Object::Array(items) => !items.is_empty(), + _ => true, + }) { + return Err(unsupported(index, "annotations need separate structure tags")); + } + let pieces = content_pieces(doc, page, index)?; + totals.0 = totals.0.saturating_add(pieces.data().len()); + let parsed = pieces.parse(); + if parsed.skipped > 0 || parsed.ambiguous_images > 0 || parsed.ops.len() > MAX_OPERATORS { + return Err(unsupported(index, "content has malformed or too many operators")); + } + totals.1 = totals.1.saturating_add(parsed.ops.len()); + if totals.0 > MAX_TOTAL_BYTES || totals.1 > MAX_TOTAL_OPERATORS { + return Err(unsupported(index, "the document has too much content to auto-tag in one edit")); + } + let xobjects = page_xobjects(doc, &page.dict); + let (mut start, mut has_text, mut spans, mut artifacts) = (None, false, Vec::new(), HashMap::new()); + // Marked-content boundaries must enclose the whole path, not just its paint operator. + let mut path = None; + for (at, op) in parsed.ops.iter().enumerate() { + if path.is_some() && matches!(op.op.as_slice(), b"BT" | b"Do" | b"BI" | b"sh") { + return Err(unsupported(index, "painted content interrupts an unfinished path")); + } match op.op.as_slice() { - b"BT" if start.is_none() => { - start = Some(at); - has_text = false; - } + b"BT" if start.is_none() => (start, has_text) = (Some(at), false), b"ET" => { let Some(first) = start.take() else { return Err(unsupported(index, "unmatched ET operator")) }; if has_text { @@ -160,14 +224,31 @@ fn text_spans(ops: &[Op], index: usize) -> Result, AutoTagEr } b"Tj" | b"TJ" | b"'" | b"\"" if start.is_some() => has_text = true, name if start.is_some() && harmless_inside(name) => {} - name if start.is_none() && harmless_outside(name) => {} + name if start.is_none() && harmless_outside(name) => match name { + b"m" | b"re" => { + path.get_or_insert(at); + } + b"n" => path = None, + _ => {} + }, + _ if start.is_none() && painted_outside(doc, &xobjects, op) => { + let first = if matches!(op.op.as_slice(), b"f" | b"F" | b"f*" | b"B" | b"B*" | b"b" | b"b*" | b"S" | b"s") { + path.take().unwrap_or(at) + } else { + at + }; + artifacts.insert(first, at); + } _ => return Err(unsupported(index, format!("unsupported or painted operator {}", String::from_utf8_lossy(&op.op)))), } } if start.is_some() { return Err(unsupported(index, "unterminated BT text object")); } - Ok(spans) + if path.is_some() { + return Err(unsupported(index, "unterminated path object")); + } + Ok(Plan { page: page.obj, ops: parsed.ops, spans, artifacts }) } fn plan(doc: &Document) -> Result, AutoTagError> { @@ -186,24 +267,11 @@ fn plan(doc: &Document) -> Result, AutoTagError> { } let mut out = Vec::with_capacity(pages.len()); let mut total = 0usize; + let mut totals = (0usize, 0usize); for (index, page) in pages.iter().enumerate() { - if page.dict.contains(b"StructParents") { - return Err(unsupported(index, "page already has a structure parent key")); - } - if page.dict.get(b"Annots").is_some_and(|a| match &*doc.resolve(a) { - Object::Array(items) => !items.is_empty(), - _ => true, - }) { - return Err(unsupported(index, "annotations need separate structure tags")); - } - let bytes = content_bytes(doc, page, index)?; - let parsed = parse(&bytes); - if parsed.skipped > 0 || parsed.ops.len() > MAX_OPERATORS { - return Err(unsupported(index, "content has malformed or too many operators")); - } - let spans = text_spans(&parsed.ops, index)?; - total = total.saturating_add(spans.len()); - out.push(Plan { page: page.obj, ops: parsed.ops, spans }); + let planned = plan_page(doc, page, index, &mut totals)?; + total = total.saturating_add(planned.spans.len()); + out.push(planned); } if total == 0 { return Err(AutoTagError::NoText); @@ -211,62 +279,36 @@ fn plan(doc: &Document) -> Result, AutoTagError> { Ok(out) } -/// Tag every text object in an untagged, text-only PDF as a paragraph. The full structure tree, -/// parent tree and marked-content IDs are written together. Mixed content is refused before any -/// mutation. On success the caller can use tag-property edits to assign reviewed heading roles. -pub fn autotag_text(doc: &mut Document) -> Result { - let plans = plan(doc)?; - let mut next = doc.clone(); - let catalog = next.root().ok_or(AutoTagError::NoCatalog)?; - let root = next.add(Object::Null); - let document = next.add(Object::Null); - let mut children = Vec::new(); - let mut numbers = Vec::new(); - let mut tagged_pages = 0usize; - let mut paragraphs = 0usize; - for page in plans { - if page.spans.is_empty() { - continue; +fn mark_ops(ops: Vec, starts: &HashMap, ends: &HashSet, artifacts: &HashMap) -> Result, AutoTagError> { + let artifact_ends: HashSet = artifacts.values().copied().collect(); + let extra = starts.len().saturating_add(artifacts.len()).saturating_mul(2); + let mut marked = Vec::with_capacity(ops.len().saturating_add(extra)); + for (at, op) in ops.into_iter().enumerate() { + let is_artifact = artifacts.contains_key(&at); + if let Some(mcid) = starts.get(&at) { + let mut properties = Dict::new(); + properties.set(b"MCID".to_vec(), Object::Int(i64::try_from(*mcid).map_err(|_| AutoTagError::TooManyPages)?)); + marked.push(Op::new("BDC", vec![Object::name("P"), Object::Dict(properties)])); + } else if is_artifact { + marked.push(Op::new("BMC", vec![Object::name("Artifact")])); } - let key = i64::try_from(tagged_pages).map_err(|_| AutoTagError::TooManyPages)?; - let mut parents = Vec::with_capacity(page.spans.len()); - let mut starts = std::collections::HashMap::new(); - let mut ends = std::collections::HashSet::new(); - for (mcid, (first, last)) in page.spans.iter().enumerate() { - let mut element = Dict::new(); - element.set(b"Type".to_vec(), Object::name("StructElem")); - element.set(b"S".to_vec(), Object::name("P")); - element.set(b"P".to_vec(), Object::Ref(document)); - element.set(b"Pg".to_vec(), Object::Ref(page.page)); - element.set(b"K".to_vec(), Object::Int(i64::try_from(mcid).map_err(|_| AutoTagError::TooManyPages)?)); - let reference = next.add(element); - parents.push(Object::Ref(reference)); - children.push(Object::Ref(reference)); - starts.insert(*first, mcid); - ends.insert(*last); - paragraphs = paragraphs.saturating_add(1); + marked.push(op); + if ends.contains(&at) || artifact_ends.contains(&at) { + marked.push(Op::new("EMC", Vec::new())); } - let mut marked = Vec::with_capacity(page.ops.len().saturating_add(page.spans.len().saturating_mul(2))); - for (at, op) in page.ops.into_iter().enumerate() { - if let Some(mcid) = starts.get(&at) { - let mut properties = Dict::new(); - properties.set(b"MCID".to_vec(), Object::Int(i64::try_from(*mcid).map_err(|_| AutoTagError::TooManyPages)?)); - marked.push(Op::new("BDC", vec![Object::name("P"), Object::Dict(properties)])); - } - marked.push(op); - if ends.contains(&at) { - marked.push(Op::new("EMC", Vec::new())); - } - } - let stream = next.add(Stream::flate(Dict::new(), &serialize_ops(&marked))); - next.update_dict(page.page, |d| { - d.set(b"Contents".to_vec(), Object::Ref(stream)); - d.set(b"StructParents".to_vec(), Object::Int(key)); - })?; - numbers.push(Object::Int(key)); - numbers.push(Object::Array(parents)); - tagged_pages += 1; } + Ok(marked) +} + +fn write_structure( + next: &mut Document, + catalog: ObjRef, + root: ObjRef, + document: ObjRef, + children: Vec, + numbers: Vec, + tagged_pages: usize, +) -> Result<(), AutoTagError> { let mut tree = Dict::new(); tree.set(b"Nums".to_vec(), Object::Array(numbers)); let parent_tree = next.add(tree); @@ -289,6 +331,50 @@ pub fn autotag_text(doc: &mut Document) -> Result { d.set(b"StructTreeRoot".to_vec(), Object::Ref(root)); d.set(b"MarkInfo".to_vec(), Object::Dict(mark_info)); })?; + Ok(()) +} + +/// Tag every text object in an untagged PDF as a paragraph and mark painted paths, shadings and +/// images outside text objects as artifacts. The full structure tree, parent tree and +/// marked-content IDs are written together. Unsupported operators are refused before any +/// mutation. On success the caller can use tag-property edits to assign reviewed heading roles. +pub fn autotag_text(doc: &mut Document) -> Result { + let plans = plan(doc)?; + let mut next = doc.clone(); + let catalog = next.root().ok_or(AutoTagError::NoCatalog)?; + let (root, document) = (next.add(Object::Null), next.add(Object::Null)); + let (mut children, mut numbers, mut tagged_pages, mut paragraphs) = (Vec::new(), Vec::new(), 0usize, 0usize); + for page in plans { + if page.spans.is_empty() && page.artifacts.is_empty() { + continue; + } + let (mut starts, mut ends) = (HashMap::new(), HashSet::new()); + if !page.spans.is_empty() { + let key = i64::try_from(tagged_pages).map_err(|_| AutoTagError::TooManyPages)?; + let mut parents = Vec::with_capacity(page.spans.len()); + for (mcid, (first, last)) in page.spans.iter().enumerate() { + let mut element = Dict::new(); + element.set(b"Type".to_vec(), Object::name("StructElem")); + element.set(b"S".to_vec(), Object::name("P")); + element.set(b"P".to_vec(), Object::Ref(document)); + element.set(b"Pg".to_vec(), Object::Ref(page.page)); + element.set(b"K".to_vec(), Object::Int(i64::try_from(mcid).map_err(|_| AutoTagError::TooManyPages)?)); + let reference = next.add(element); + parents.push(Object::Ref(reference)); + children.push(Object::Ref(reference)); + starts.insert(*first, mcid); + ends.insert(*last); + paragraphs = paragraphs.saturating_add(1); + } + next.update_dict(page.page, |d| d.set(b"StructParents".to_vec(), Object::Int(key)))?; + numbers.extend([Object::Int(key), Object::Array(parents)]); + tagged_pages += 1; + } + let marked = mark_ops(page.ops, &starts, &ends, &page.artifacts)?; + let stream = next.add(Stream::flate(Dict::new(), &serialize_ops(&marked))); + next.update_dict(page.page, |d| d.set(b"Contents".to_vec(), Object::Ref(stream)))?; + } + write_structure(&mut next, catalog, root, document, children, numbers, tagged_pages)?; *doc = next; Ok(AutoTagResult { pages: tagged_pages, paragraphs }) } diff --git a/crates/a11y/src/autotag_fields.rs b/crates/a11y/src/autotag_fields.rs index 3203aca376..c93591c1d0 100644 --- a/crates/a11y/src/autotag_fields.rs +++ b/crates/a11y/src/autotag_fields.rs @@ -10,6 +10,23 @@ fn reference(object: Option<&Object>, what: &str) -> Result { object.and_then(Object::as_ref).ok_or_else(|| format!("the {what} must be an indirect reference")) } +fn root_document_ref(doc: &Document, k: Option<&Object>) -> Result { + let Some(k) = k else { + return Err("the structure tree root K must be an indirect reference".into()); + }; + if let Some(r) = k.as_ref() + && doc.get(r).as_dict().is_some() + { + return Ok(r); + } + if let Some([item]) = doc.resolve(k).as_array().map(Vec::as_slice) + && let Some(r) = item.as_ref() + { + return Ok(r); + } + Err("the structure tree root K must be an indirect reference".into()) +} + fn nums(doc: &Document, parent_tree: ObjRef) -> Result, String> { let tree = doc.get(parent_tree); let tree = tree.as_dict().ok_or("the structure parent tree is invalid")?; @@ -31,22 +48,32 @@ pub fn tag_form_fields(doc: &mut Document) -> Result { let root = doc.root().ok_or("the PDF has no catalog")?; let catalog_object = doc.get(root); let catalog = catalog_object.as_dict().ok_or("the PDF catalog is invalid")?; - if !catalog.contains(b"AcroForm") { + if !catalog.get(b"AcroForm").is_some_and(|v| !matches!(*doc.resolve(v), Object::Null)) { return Err("the PDF has no AcroForm fields".into()); } - let existing_root = catalog.get(b"StructTreeRoot").cloned(); + let existing_root = catalog.get(b"StructTreeRoot").filter(|v| !matches!(*doc.resolve(v), Object::Null)).cloned(); let mark_info = catalog.get(b"MarkInfo").map(|v| doc.resolve(v)).as_deref().and_then(Object::as_dict).cloned().unwrap_or_default(); let pages = pdfcraft_model::pages(doc); let mut widgets = Vec::new(); // A widget listed in more than one page's /Annots is tagged once (one key, one /Form). let mut seen = std::collections::HashSet::new(); + // A shared indirect /Annots array is scanned for the first page that lists it: its widgets + // are tagged once anyway, so scanning it again for every page only repeats the work. + let mut arrays = std::collections::HashSet::new(); for page in &pages { + if let Some(array) = page.dict.get(b"Annots").and_then(Object::as_ref) + && !arrays.insert(array) + { + continue; + } let annots = page.dict.get(b"Annots").map(|a| doc.resolve(a)).as_deref().and_then(Object::as_array).cloned().unwrap_or_default(); for annot in annots { let Some(reference) = annot.as_ref() else { continue }; let resolved = doc.get(reference); let Some(dict) = resolved.as_dict() else { continue }; - if dict.name(b"Subtype") == Some(b"Widget") && !dict.contains(b"StructParent") && seen.insert(reference) { + let is_widget = dict.get(b"Subtype").map(|s| doc.resolve(s)).as_deref().and_then(Object::as_name) == Some(b"Widget"); + let has_parent = dict.get(b"StructParent").map(|v| doc.resolve(v)).and_then(|v| v.as_int()).is_some_and(|k| k >= 0); + if is_widget && !has_parent && seen.insert(reference) { widgets.push((page.obj, reference)); } } @@ -54,16 +81,29 @@ pub fn tag_form_fields(doc: &mut Document) -> Result { if widgets.is_empty() { return Err("the AcroForm has no untagged page widgets".into()); } + let count = i64::try_from(widgets.len()).map_err(|_| "too many form widgets")?; - let (struct_root, document_ref, parent_tree, mut parent_nums, first_key, new_tree) = if let Some(existing_root) = existing_root { + let (struct_root, document_ref, mut existing_kids, parent_tree, mut parent_nums, first_key, new_tree) = if let Some(existing_root) = existing_root + { let struct_root = reference(Some(&existing_root), "structure tree root")?; let structure_object = doc.get(struct_root); let structure = structure_object.as_dict().ok_or("the structure tree root is invalid")?; - let document_ref = reference(structure.get(b"K"), "structure tree root K")?; - let document = doc.get(document_ref); - if document.as_dict().is_none_or(|d| d.name(b"S") != Some(b"Document")) { + let document_ref = root_document_ref(doc, structure.get(b"K"))?; + let document_obj = doc.get(document_ref); + let Some(document_dict) = document_obj.as_dict() else { + return Err("the structure tree root must point directly to a Document element".into()); + }; + if document_dict.get(b"S").map(|s| doc.resolve(s)).as_deref().and_then(Object::as_name) != Some(b"Document") { return Err("the structure tree root must point directly to a Document element".into()); } + let existing_kids = match document_dict.get(b"K") { + None => Vec::new(), + Some(k) => match &*doc.resolve(k) { + Object::Null => Vec::new(), + Object::Array(items) => items.clone(), + _ => vec![k.clone()], + }, + }; let parent_tree = reference(structure.get(b"ParentTree"), "structure parent tree")?; let parent_nums = nums(doc, parent_tree)?; if parent_nums.len() % 2 != 0 { @@ -71,14 +111,26 @@ pub fn tag_form_fields(doc: &mut Document) -> Result { } let mut greatest = -1_i64; for pair in parent_nums.as_chunks::<2>().0 { - let Some(key) = pair[0].as_int() else { return Err("the structure parent tree has a non-integer key".into()) }; + let Some(key) = doc.resolve(&pair[0]).as_int() else { + return Err("the structure parent tree has a non-integer key".into()); + }; if key < 0 { return Err("the structure parent tree has a negative key".into()); } greatest = greatest.max(key); } - let first_key = greatest.checked_add(1).ok_or("the structure parent tree has no available key")?; - (struct_root, document_ref, parent_tree, parent_nums, first_key, false) + let mut first_key = greatest.checked_add(1).ok_or("the structure parent tree has no available key")?; + if let Some(next_obj) = structure.get(b"ParentTreeNextKey") { + let resolved_next = doc.resolve(next_obj); + if !matches!(*resolved_next, Object::Null) { + let next_key = resolved_next.as_int().ok_or("the structure parent tree next key is invalid")?; + if next_key < 0 { + return Err("the structure parent tree next key is negative".into()); + } + first_key = first_key.max(next_key); + } + } + (struct_root, document_ref, existing_kids, parent_tree, parent_nums, first_key, false) } else { let mut root_dict = Dict::new(); root_dict.set(b"Type".to_vec(), Object::Name(b"StructTreeRoot".to_vec())); @@ -88,9 +140,10 @@ pub fn tag_form_fields(doc: &mut Document) -> Result { document.set(b"P".to_vec(), Object::Ref(struct_root)); let document_ref = doc.add(Object::Dict(document)); let parent_tree = doc.add(Object::Dict(Dict::new())); - (struct_root, document_ref, parent_tree, Vec::new(), 0, true) + (struct_root, document_ref, Vec::new(), parent_tree, Vec::new(), 0, true) }; - let mut kids = Vec::with_capacity(widgets.len()); + // ISO 32000-1 14.7.4.4: reserve the next unused key before adding any parent-tree entries. + let next_key = first_key.checked_add(count).ok_or("the structure parent tree has no available next key")?; for (offset, (page, widget)) in widgets.iter().copied().enumerate() { let key = first_key.checked_add(i64::try_from(offset).map_err(|_| "too many form widgets")?).ok_or("too many form widgets")?; let mut objr = Dict::new(); @@ -105,16 +158,10 @@ pub fn tag_form_fields(doc: &mut Document) -> Result { let form_ref = doc.add(Object::Dict(form)); doc.update_dict(widget, |d| d.set(b"StructParent".to_vec(), Object::Int(key))).map_err(|e| e.to_string())?; parent_nums.extend([Object::Int(key), Object::Ref(form_ref)]); - kids.push(Object::Ref(form_ref)); + existing_kids.push(Object::Ref(form_ref)); } doc.update_dict(document_ref, |d| { - let old = d.remove(b"K").unwrap_or(Object::Array(Vec::new())); - let mut all = match old { - Object::Array(items) => items, - item => vec![item], - }; - all.extend(kids); - d.set(b"K".to_vec(), Object::Array(all)); + d.set(b"K".to_vec(), Object::Array(existing_kids)); }) .map_err(|e| e.to_string())?; doc.update_dict(parent_tree, |d| d.set(b"Nums".to_vec(), Object::Array(parent_nums))).map_err(|e| e.to_string())?; @@ -122,6 +169,12 @@ pub fn tag_form_fields(doc: &mut Document) -> Result { doc.update_dict(struct_root, |d| { d.set(b"K".to_vec(), Object::Ref(document_ref)); d.set(b"ParentTree".to_vec(), Object::Ref(parent_tree)); + d.set(b"ParentTreeNextKey".to_vec(), Object::Int(next_key)); + }) + .map_err(|e| e.to_string())?; + } else { + doc.update_dict(struct_root, |d| { + d.set(b"ParentTreeNextKey".to_vec(), Object::Int(next_key)); }) .map_err(|e| e.to_string())?; } @@ -136,3 +189,87 @@ pub fn tag_form_fields(doc: &mut Document) -> Result { .map_err(|e| e.to_string())?; Ok(widgets.len()) } + +#[cfg(test)] +mod tests { + use std::sync::Arc; + + use super::*; + use crate::tags; + + fn pdf(bodies: &[String]) -> Document { + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (i, b) in bodies.iter().enumerate() { + offsets.push(out.len()); + out.extend_from_slice(format!("{} 0 obj\n{b}\nendobj\n", i + 1).as_bytes()); + } + let xref = out.len(); + out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", bodies.len() + 1).as_bytes()); + for o in offsets { + out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + out.extend_from_slice(format!("trailer << /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", bodies.len() + 1).as_bytes()); + Document::open(Arc::new(out)).unwrap() + } + + fn existing_tree(next: &str) -> Document { + pdf(&[ + "<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [4 0 R] >> /StructTreeRoot 5 0 R >>".into(), + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>".into(), + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Annots [4 0 R] >>".into(), + "<< /Type /Annot /Subtype /Widget /FT /Tx /T (name) /Rect [20 60 120 80] >>".into(), + format!("<< /Type /StructTreeRoot /K 6 0 R /ParentTree 7 0 R {next} >>"), + "<< /S /Document /P 5 0 R /K [] >>".into(), + "<< /Nums [7 []] >>".into(), + "null".into(), + ]) + } + + #[test] + fn extending_tree_creates_missing_next_key() { + for next in ["", "/ParentTreeNextKey null", "/ParentTreeNextKey 8 0 R"] { + let mut doc = existing_tree(next); + assert_eq!(tag_form_fields(&mut doc).unwrap(), 1); + assert_eq!(doc.get(ObjRef::new(4, 0)).as_dict().unwrap().int(b"StructParent"), Some(8)); + assert_eq!(doc.get(ObjRef::new(5, 0)).as_dict().unwrap().int(b"ParentTreeNextKey"), Some(9)); + } + } + + #[test] + fn exhausted_next_key_rejects_before_mutation() { + let mut doc = existing_tree(&format!("/ParentTreeNextKey {}", i64::MAX)); + let before = doc.get(ObjRef::new(4, 0)); + let objects = doc.object_numbers().len(); + assert!(tag_form_fields(&mut doc).is_err()); + assert_eq!(doc.get(ObjRef::new(4, 0)), before); + assert_eq!(doc.object_numbers().len(), objects); + } + + #[test] + fn extends_existing_tree_with_array_root_k_indirect_document_kids_and_indirect_nums_keys() { + let mut doc = pdf(&[ + "<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [5 0 R] >> /MarkInfo << /Marked true >> /StructTreeRoot 6 0 R >>".into(), + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>".into(), + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Annots [5 0 R] /StructParents 0 >>".into(), + "<< /Length 0 >>\nstream\n\nendstream".into(), + "<< /Type /Annot /Subtype /Widget /FT /Tx /T (name) /Rect [20 60 120 80] /StructParent null >>".into(), + "<< /Type /StructTreeRoot /K [7 0 R] /ParentTree 8 0 R /ParentTreeNextKey 5 >>".into(), + "<< /S /Document /P 6 0 R /K 10 0 R >>".into(), + "<< /Nums [11 0 R [9 0 R]] >>".into(), + "<< /S /P /P 7 0 R /Pg 3 0 R /K 0 >>".into(), + "[9 0 R]".into(), + "0".into(), + ]); + assert_eq!(tag_form_fields(&mut doc).unwrap(), 1); + // Existing paragraph (9 0 R) and new Form tag remain flat siblings under /Document (7 0 R). + let doc_k = doc.get(ObjRef::new(7, 0)).as_dict().unwrap().get(b"K").unwrap().as_array().unwrap().clone(); + assert_eq!(doc_k.len(), 2); + assert_eq!(doc_k[0].as_ref(), Some(ObjRef::new(9, 0))); + let listed: Vec<_> = tags(&doc).into_iter().map(|t| t.kind).collect(); + assert_eq!(listed, ["Document", "P", "Form"]); + // Widget got key 5 from /ParentTreeNextKey, and /ParentTreeNextKey advanced to 6. + assert_eq!(doc.get(ObjRef::new(5, 0)).as_dict().unwrap().int(b"StructParent"), Some(5)); + assert_eq!(doc.get(ObjRef::new(6, 0)).as_dict().unwrap().int(b"ParentTreeNextKey"), Some(6)); + } +} diff --git a/crates/a11y/src/content.rs b/crates/a11y/src/content.rs index d548c57f9d..faf1a87bad 100644 --- a/crates/a11y/src/content.rs +++ b/crates/a11y/src/content.rs @@ -73,7 +73,8 @@ impl Walk<'_> { // Marked-content stack: whether each level is tagged or an artifact. let mut stack: Vec = Vec::new(); let mut untagged = 0; - for op in pdfcraft_content::parse(data).ops { + // One operator at a time: the walk keeps counts, never the stream's operators. + for op in pdfcraft_content::ops(data) { let inside = covered || stack.last().copied().unwrap_or(false); match op.op.as_slice() { b"BMC" => stack.push(inside || op.name(0) == Some(b"Artifact")), @@ -261,11 +262,13 @@ fn annotation_rect(doc: &Document, annot: &Dict) -> Option<[f64; 4]> { pub(crate) fn page_rule(doc: &Document, rule: Rule, pages: &[Page], list: &[usize], scan: &Scan, tagged: bool) -> Vec { let mut out = Vec::new(); - let annots = |p: &Page| -> Vec<(ObjRef, Dict)> { + // The annotation objects are shared, never deep-copied: a reference repeated across a page's + // Annots array (or its pages) must not copy the dictionary each time. + let annots = |p: &Page| -> Vec<(ObjRef, std::sync::Arc)> { let a = p.dict.get(b"Annots").map(|a| doc.resolve(a)).and_then(|a| a.as_array().cloned()).unwrap_or_default(); a.iter() - .filter_map(|x| x.as_ref().and_then(|r| doc.get(r).as_dict().cloned().map(|d| (r, d)))) - .filter(|(_, d)| d.name(b"Subtype") != Some(b"Popup")) + .filter_map(|x| x.as_ref().map(|r| (r, doc.get(r)))) + .filter(|(_, o)| o.as_dict().is_some_and(|d| d.name(b"Subtype") != Some(b"Popup"))) .collect() }; match rule { @@ -283,7 +286,8 @@ pub(crate) fn page_rule(doc: &Document, rule: Rule, pages: &[Page], list: &[usiz Rule::TaggedAnnotations | Rule::TaggedMultimedia | Rule::TaggedFormFields => { for &p in list { let Some(page) = pages.get(p) else { continue }; - for (_, a) in annots(page) { + for (_, obj) in annots(page) { + let Some(a) = obj.as_dict() else { continue }; let st = a.name(b"Subtype").unwrap_or(b""); let applies = match rule { Rule::TaggedFormFields => st == b"Widget", @@ -298,11 +302,11 @@ pub(crate) fn page_rule(doc: &Document, rule: Rule, pages: &[Page], list: &[usiz let pattern = if st == b"Widget" { "Form field on page {p} isn't tagged" } else { - values.push(("{}", subtype_name(&a))); + values.push(("{}", subtype_name(a))); "{} annotation on page {p} isn't tagged" }; let mut finding = Finding::worded(Some(p), pattern, values); - finding.bbox = annotation_rect(doc, &a); + finding.bbox = annotation_rect(doc, a); out.push(finding); } } @@ -344,7 +348,8 @@ pub(crate) fn page_rule(doc: &Document, rule: Rule, pages: &[Page], list: &[usiz let mut seen = HashSet::new(); for &p in list { let Some(page) = pages.get(p) else { continue }; - for (r, a) in annots(page) { + for (r, obj) in annots(page) { + let Some(a) = obj.as_dict() else { continue }; if a.name(b"Subtype") != Some(b"Widget") { continue; } @@ -361,7 +366,7 @@ pub(crate) fn page_rule(doc: &Document, rule: Rule, pages: &[Page], list: &[usiz "Field \"{}\" on page {p} has no description (tooltip)", vec![("{}", full_name(doc, field)), ("{p}", (p + 1).to_string())], ); - finding.bbox = annotation_rect(doc, &a); + finding.bbox = annotation_rect(doc, a); out.push(finding); } } diff --git a/crates/a11y/src/headings.rs b/crates/a11y/src/headings.rs index 0fa31ccbb1..235ba4b452 100644 --- a/crates/a11y/src/headings.rs +++ b/crates/a11y/src/headings.rs @@ -37,9 +37,16 @@ struct Walker<'a> { visits: usize, /// Per page: the text each MCID shows (read once per page). texts: HashMap>, + /// What is left of [`MAX_TITLE_SOURCE_BYTES`]. + title_left: usize, out: Vec, } +/// Bytes of `/ActualText`, `/Alt` and `/T` strings the headings of one document decode in all. +/// Titles are cut to [`MAX_TITLE`] characters, but many headings can name one large string, and +/// each would otherwise be decoded in full first. +const MAX_TITLE_SOURCE_BYTES: usize = 32 << 20; + /// The headings, in document order. Headings without text or a page are left out. pub fn headings(doc: &Document) -> Vec { let Some(root) = tree_root(doc) else { return Vec::new() }; @@ -52,6 +59,7 @@ pub fn headings(doc: &Document) -> Vec { seen: HashSet::new(), visits: 0, texts: HashMap::new(), + title_left: MAX_TITLE_SOURCE_BYTES, out: Vec::new(), }; if let Some(k) = root.get(b"K") { @@ -131,17 +139,34 @@ impl Walker<'_> { self.marks(kids, pg, depth + 1, &mut marks); } let Some(page) = pg.or_else(|| marks.iter().find_map(|m| m.0)).and_then(|p| self.page_index.get(&p).copied()) else { return }; - let doc = self.doc; - let attr = |key: &[u8]| d.get(key).map(|o| doc.resolve(o)).and_then(|o| o.as_string().map(|s| s.to_text())).filter(|t| !t.trim().is_empty()); - let title = match attr(b"ActualText") { + let title = match self.attr(d, b"ActualText") { Some(t) => Some(t), - None => self.marked_text(&marks, page).or_else(|| attr(b"Alt")).or_else(|| attr(b"T")), + None => match self.marked_text(&marks, page) { + Some(t) => Some(t), + None => self.attr(d, b"Alt").or_else(|| self.attr(d, b"T")), + }, }; let Some(title) = title else { return }; let title: String = title.split_whitespace().collect::>().join(" ").chars().take(MAX_TITLE).collect(); self.out.push(Heading { obj, level, title, page }); } + /// The non-blank text of the string `key` holds, charged against the title allowance by the + /// longer of its raw and decoded size (a string that does not fit is not decoded). + fn attr(&mut self, d: &Dict, key: &[u8]) -> Option { + let o = self.doc.resolve(d.get(key)?); + let s = o.as_string()?; + // A text string decodes to at most three bytes per raw byte, so one whose worst case does + // not fit the remaining allowance is never decoded: the raw check alone let a string at + // the raw limit decode to three times it before the allowance was reduced. + if s.bytes.len().saturating_mul(3) > self.title_left { + return None; + } + let text = s.to_text(); + self.title_left = self.title_left.saturating_sub(text.len().max(s.bytes.len())); + Some(text).filter(|t| !t.trim().is_empty()) + } + /// The text the marked content shows (`page`: where marks without a page of their own are). fn marked_text(&mut self, marks: &[(Option, i64)], page: usize) -> Option { let mut parts = Vec::new(); diff --git a/crates/a11y/src/layout_table.rs b/crates/a11y/src/layout_table.rs new file mode 100644 index 0000000000..6b2bab2d09 --- /dev/null +++ b/crates/a11y/src/layout_table.rs @@ -0,0 +1,359 @@ +//! A layout table as plain blocks: a table used only to place content (label and value pairs, a +//! letterhead grid) is read as a data table, with headers and cell navigation it doesn't have. One +//! edit turns the table, its row groups and rows into Div elements and its cells into P elements +//! (Div when a cell holds tags of its own), keeping the content, its order and everything inside +//! the cells, and drops the Table attributes (summary, scope, spans) they no longer have (#4105). + +use std::collections::HashSet; + +use pdfcraft_cos::{CosError, Dict, Document, ObjRef, Object}; + +use crate::alt; + +/// The most structure elements one table may have: past it the edit is refused. +const MAX_ELEMENTS: usize = 100_000; +/// The deepest nesting of direct (inline) elements followed. +const MAX_DEPTH: usize = 256; + +#[derive(Debug, thiserror::Error, PartialEq)] +pub enum LayoutTableError { + #[error("object {0} is not a reachable indirect structure tag")] + NotATag(ObjRef), + #[error("the tag is not a table")] + NotATable, + #[error("the table has too many tags, or tags nested too deeply")] + TooLarge, + #[error(transparent)] + Cos(#[from] CosError), +} + +/// What a table part becomes; `None` for anything else (which keeps its type and contents). +fn block_type(ty: &[u8], has_tags: bool, root: bool) -> Option<&'static str> { + match ty { + b"Table" if root => Some("Div"), + b"THead" | b"TBody" | b"TFoot" | b"TR" => Some("Div"), + b"TH" | b"TD" | b"Caption" => Some(if has_tags { "Div" } else { "P" }), + _ => None, + } +} + +struct Retag<'a> { + doc: &'a Document, + role_map: Dict, + /// Indirect elements under the table still to look at. + queue: Vec, + /// How many elements changed. + changed: usize, +} + +impl Retag<'_> { + fn standard(&self, d: &Dict) -> Option> { + Some(alt::standard(self.doc, &self.role_map, d.name(b"S")?)) + } + + /// Whether children `k` include structure elements (indirect or direct, also in an indirect + /// array of children). + fn has_tags(&self, k: &Object, depth: usize) -> bool { + if depth > MAX_DEPTH { + return false; + } + match k { + Object::Array(items) => items.iter().any(|i| self.has_tags(i, depth + 1)), + Object::Ref(r) => match &*self.doc.get(*r) { + Object::Dict(d) => d.contains(b"S"), + array @ Object::Array(_) => self.has_tags(array, depth + 1), + _ => false, + }, + Object::Dict(d) => d.contains(b"S"), + _ => false, + } + } + + /// Element `d` as a block when it is a part of the table (`root`: the table itself), with its + /// direct children converted the same way and its indirect ones queued; `None` otherwise. + fn convert(&mut self, d: &Dict, root: bool, depth: usize) -> Result, LayoutTableError> { + if depth > MAX_DEPTH { + return Err(LayoutTableError::TooLarge); + } + let Some(ty) = self.standard(d) else { return Ok(None) }; + // As written: a single child is often a reference to it, which must stay one. + let kids = d.get(b"K").cloned(); + let has_tags = kids.as_ref().is_some_and(|k| self.has_tags(k, 0)); + let Some(block) = block_type(&ty, has_tags, root) else { return Ok(None) }; + let mut out = d.clone(); + out.set(b"S".to_vec(), Object::name(block)); + self.drop_table_attributes(&mut out); + if let Some(kids) = kids { + let kids = self.convert_kids(&kids, depth + 1)?; + out.set(b"K".to_vec(), kids); + } + self.changed = self.changed.saturating_add(1); + Ok(Some(out)) + } + + /// Children `k` with direct table parts converted and indirect elements queued (references + /// stay references; an indirect array of children is written out only when a direct table + /// part in it changed). + fn convert_kids(&mut self, k: &Object, depth: usize) -> Result { + if depth > MAX_DEPTH { + return Err(LayoutTableError::TooLarge); + } + Ok(match k { + Object::Array(items) => Object::Array(items.iter().map(|i| self.convert_kids(i, depth)).collect::>()?), + Object::Ref(r) => match &*self.doc.get(*r) { + Object::Dict(d) if d.contains(b"S") => { + self.queue.push(*r); + k.clone() + } + array @ Object::Array(_) => { + let converted = self.convert_kids(array, depth + 1)?; + if converted == *array { k.clone() } else { converted } + } + _ => k.clone(), + }, + Object::Dict(d) if d.contains(b"S") => match self.convert(d, false, depth)? { + Some(block) => Object::Dict(block), + None => k.clone(), + }, + _ => k.clone(), + }) + } + + /// Remove the Table attribute objects (`/O /Table`) from `d`'s `/A`, keeping the others and + /// their revision numbers; an `/A` left empty goes. + fn drop_table_attributes(&self, d: &mut Dict) { + let Some(a) = d.get(b"A").cloned() else { return }; + let is_table = |o: &Object| self.doc.resolve(o).as_dict().is_some_and(|x| x.name(b"O") == Some(b"Table")); + let resolved = self.doc.resolve(&a); + let Object::Array(items) = &*resolved else { + if is_table(&a) { + d.remove(b"A"); + } + return; + }; + let mut kept = Vec::with_capacity(items.len()); + let mut dropping = false; + for item in items { + match item { + // A revision number goes with the attribute object before it. + Object::Int(_) if dropping => {} + _ => { + dropping = is_table(item); + if !dropping { + kept.push(item.clone()); + } + } + } + } + if kept.is_empty() { + d.remove(b"A"); + } else if kept.len() != items.len() { + d.set(b"A".to_vec(), Object::Array(kept)); + } + } +} + +/// Turn layout table `table` (a reachable indirect tag whose type is Table after the role map) into +/// Div and P elements as the module documentation says; returns how many elements changed. Tables +/// nested in its cells stay tables. +pub fn layout_table_to_blocks(doc: &mut Document, table: ObjRef) -> Result { + let Some(element) = alt::elements(doc).into_iter().find(|e| e.object == table) else { + return Err(LayoutTableError::NotATag(table)); + }; + let role_map = alt::tree_root(doc).and_then(|r| r.get(b"RoleMap").map(|o| doc.resolve(o))).and_then(|o| o.as_dict().cloned()).unwrap_or_default(); + let mut retag = Retag { doc: &*doc, role_map, queue: Vec::new(), changed: 0 }; + if retag.standard(&element.dict).as_deref() != Some(b"Table".as_slice()) { + return Err(LayoutTableError::NotATable); + } + // Read everything first, then write: a refused table changes nothing. + let mut updates: Vec<(ObjRef, Dict)> = Vec::new(); + let mut seen = HashSet::new(); + if let Some(block) = retag.convert(&element.dict, true, 0)? { + seen.insert(table); + updates.push((table, block)); + } + while let Some(r) = retag.queue.pop() { + if !seen.insert(r) { + continue; + } + if seen.len() > MAX_ELEMENTS { + return Err(LayoutTableError::TooLarge); + } + let Some(d) = retag.doc.get(r).as_dict().cloned() else { continue }; + if let Some(block) = retag.convert(&d, false, 0)? { + updates.push((r, block)); + } + } + let changed = retag.changed; + for (r, block) in updates { + doc.update_dict(r, |d| *d = block)?; + } + Ok(changed) +} + +#[cfg(test)] +mod tests { + use std::sync::Arc; + + use pdfcraft_cos::{Document, ObjRef, Object}; + + use super::*; + + /// A tagged document: the catalog (1), the structure tree root (2, `root` its other entries) + /// over the Document element (3), then `objects` from 4 on. + fn doc(root: &str, objects: &[&str]) -> Document { + let mut bodies = vec!["<< /Type /Catalog /StructTreeRoot 2 0 R >>".to_string(), format!("<< /Type /StructTreeRoot /K 3 0 R {root} >>")]; + bodies.extend(objects.iter().map(|s| s.to_string())); + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (i, b) in bodies.iter().enumerate() { + offsets.push(out.len()); + out.extend_from_slice(format!("{} 0 obj\n{b}\nendobj\n", i + 1).as_bytes()); + } + let xref = out.len(); + out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", bodies.len() + 1).as_bytes()); + for o in offsets { + out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + out.extend_from_slice(format!("trailer << /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", bodies.len() + 1).as_bytes()); + Document::open(Arc::new(out)).unwrap() + } + + fn r(n: u32) -> ObjRef { + ObjRef::new(n, 0) + } + + fn dict(doc: &Document, n: u32) -> Dict { + doc.get(r(n)).as_dict().cloned().unwrap() + } + + fn ty(doc: &Document, n: u32) -> Vec { + dict(doc, n).name(b"S").unwrap().to_vec() + } + + #[test] + fn a_label_and_value_table_becomes_divs_and_paragraphs() { + let mut d = doc( + "", + &[ + "<< /S /Document /K [4 0 R] >>", + // 4: the table, with a layout attribute and a summary (revision 1). + "<< /S /Table /K [5 0 R 8 0 R] /A [ << /O /Layout /Placement /Block >> << /O /Table /Summary (s) >> 1 ] >>", + "<< /S /TR /K [6 0 R 7 0 R] >>", + "<< /S /TH /K 0 /A << /O /Table /Scope /Row >> >>", + "<< /S /TD /K 1 >>", + "<< /S /TR /K [9 0 R 10 0 R] >>", + "<< /S /TD /K 2 >>", + "<< /S /TD /K 3 /A [ << /O /Table /ColSpan 1 >> << /O /Layout /TextAlign /End >> ] >>", + ], + ); + let before: Vec = crate::tags(&d).iter().map(|t| t.object).collect(); + assert_eq!(layout_table_to_blocks(&mut d, r(4)), Ok(7)); + for (n, want) in [(4, "Div"), (5, "Div"), (6, "P"), (7, "P"), (8, "Div"), (9, "P"), (10, "P")] { + assert_eq!(ty(&d, n), want.as_bytes(), "object {n}"); + } + // The content and the order stay; Table attributes go, layout ones stay. + assert_eq!(crate::tags(&d).iter().map(|t| t.object).collect::>(), before); + assert_eq!(dict(&d, 6).get(b"K"), Some(&Object::Int(0))); + assert_eq!(dict(&d, 6).get(b"A"), None); + let Some(Object::Array(a)) = dict(&d, 4).get(b"A").cloned() else { panic!("{:?}", dict(&d, 4)) }; + assert_eq!(a.len(), 1, "the summary and its revision number go: {a:?}"); + assert_eq!(a[0].as_dict().and_then(|x| x.name(b"O")), Some(&b"Layout"[..])); + let Some(Object::Array(a)) = dict(&d, 10).get(b"A").cloned() else { panic!() }; + assert_eq!(a.len(), 1); + } + + #[test] + fn what_the_cells_hold_stays_and_a_table_in_a_cell_stays_a_table() { + let mut d = doc( + "", + &[ + "<< /S /Document /K [4 0 R] >>", + "<< /S /Table /K [5 0 R] >>", + "<< /S /TR /K [6 0 R 7 0 R] >>", + // 6: a cell holding a paragraph; 7: a cell holding a table of its own. + "<< /S /TD /K [8 0 R] >>", + "<< /S /TD /K [9 0 R] >>", + "<< /S /P /K 0 >>", + "<< /S /Table /K [10 0 R] >>", + "<< /S /TR /K [11 0 R] >>", + "<< /S /TD /K 1 >>", + ], + ); + layout_table_to_blocks(&mut d, r(4)).unwrap(); + for (n, want) in [(4, "Div"), (5, "Div"), (6, "Div"), (7, "Div"), (8, "P"), (9, "Table"), (10, "TR"), (11, "TD")] { + assert_eq!(ty(&d, n), want.as_bytes(), "object {n}"); + } + } + + #[test] + fn direct_rows_and_cells_are_converted_in_place() { + let mut d = doc( + "/RoleMap << /Grid /Table /Row /TR >>", + &[ + "<< /S /Document /K [4 0 R] >>", + "<< /S /Grid /K [ << /S /Row /K [ << /S /TD /K 0 >> << /S /TD /K [5 0 R] >> ] >> ] >>", + "<< /S /Span /K 1 >>", + ], + ); + assert_eq!(layout_table_to_blocks(&mut d, r(4)), Ok(4)); + let table = dict(&d, 4); + assert_eq!(table.name(b"S"), Some(&b"Div"[..]), "the role map makes /Grid a table"); + let Some(Object::Array(rows)) = table.get(b"K").cloned() else { panic!() }; + let row = rows[0].as_dict().unwrap(); + assert_eq!(row.name(b"S"), Some(&b"Div"[..])); + let Some(Object::Array(cells)) = row.get(b"K").cloned() else { panic!() }; + assert_eq!(cells[0].as_dict().and_then(|c| c.name(b"S")), Some(&b"P"[..])); + assert_eq!(cells[1].as_dict().and_then(|c| c.name(b"S")), Some(&b"Div"[..]), "it holds a Span tag"); + assert_eq!(ty(&d, 5), b"Span"); + } + + #[test] + fn only_a_reachable_table_is_changed() { + let mut d = doc("", &["<< /S /Document /K [4 0 R] >>", "<< /S /P /K 0 >>", "<< /S /Table >>"]); + assert_eq!(layout_table_to_blocks(&mut d, r(4)), Err(LayoutTableError::NotATable)); + assert_eq!(layout_table_to_blocks(&mut d, r(5)), Err(LayoutTableError::NotATag(r(5))), "not in the structure tree"); + assert_eq!(layout_table_to_blocks(&mut d, ObjRef::new(4, 1)), Err(LayoutTableError::NotATag(ObjRef::new(4, 1)))); + assert_eq!(ty(&d, 5), b"Table"); + } + + #[test] + fn a_single_child_written_as_a_reference_stays_its_own_object() { + // /K 5 0 R: the table's one row, the row's one cell; 8: a row whose cells are an indirect + // array (object 9) holding a direct cell and a reference. + let mut d = doc( + "", + &[ + "<< /S /Document /K [4 0 R 7 0 R] >>", + "<< /S /Table /K 5 0 R >>", + "<< /S /TR /K 6 0 R >>", + "<< /S /TD /K 0 >>", + "<< /S /Table /K 8 0 R >>", + "<< /S /TR /K 9 0 R >>", + "[ << /S /TD /K 1 >> 10 0 R ]", + "<< /S /TD /K 2 >>", + ], + ); + assert_eq!(layout_table_to_blocks(&mut d, r(4)), Ok(3)); + assert_eq!(dict(&d, 4).get(b"K"), Some(&Object::Ref(r(5))), "the row stays a reference"); + assert_eq!(dict(&d, 5).get(b"K"), Some(&Object::Ref(r(6)))); + for (n, want) in [(4, "Div"), (5, "Div"), (6, "P")] { + assert_eq!(ty(&d, n), want.as_bytes(), "object {n}"); + } + assert!(crate::tags(&d).iter().any(|t| t.object == r(6)), "the cell is still listed"); + assert_eq!(layout_table_to_blocks(&mut d, r(7)), Ok(4)); + let Some(Object::Array(cells)) = dict(&d, 8).get(b"K").cloned() else { panic!("{:?}", dict(&d, 8)) }; + assert_eq!(cells[0].as_dict().and_then(|c| c.name(b"S")), Some(&b"P"[..])); + assert_eq!(cells[1], Object::Ref(r(10))); + assert_eq!(ty(&d, 10), b"P"); + } + + #[test] + fn rows_that_point_at_each_other_end() { + // A row listed twice and a cell that lists its own row: each is converted once. + let mut d = + doc("", &["<< /S /Document /K [4 0 R] >>", "<< /S /Table /K [5 0 R 5 0 R] >>", "<< /S /TR /K [6 0 R] >>", "<< /S /TD /K [5 0 R] >>"]); + assert_eq!(layout_table_to_blocks(&mut d, r(4)), Ok(3)); + assert_eq!((ty(&d, 5), ty(&d, 6)), (b"Div".to_vec(), b"Div".to_vec())); + } +} diff --git a/crates/a11y/src/lib.rs b/crates/a11y/src/lib.rs index 280f4e845f..50e632dec8 100644 --- a/crates/a11y/src/lib.rs +++ b/crates/a11y/src/lib.rs @@ -17,6 +17,7 @@ mod autotag; mod autotag_fields; mod content; mod headings; +mod layout_table; mod report; mod structure; mod tags; @@ -41,6 +42,7 @@ pub use alt::{AltError, ArtifactError, Figure, figures, mark_decorative, mark_ta pub use autotag::{AutoTagError, AutoTagResult, autotag_text}; pub use autotag_fields::tag_form_fields; pub use headings::{Heading, headings}; +pub use layout_table::{LayoutTableError, layout_table_to_blocks}; pub use report::{REPORT_WORDS, report_html}; pub use tags::{TagContentItem, TagError, TagInfo, TagPlacement, TagProperty, move_tag, set_tag_property, tag_content, tags}; diff --git a/crates/a11y/src/tags.rs b/crates/a11y/src/tags.rs index ba9d639018..ab38f8aece 100644 --- a/crates/a11y/src/tags.rs +++ b/crates/a11y/src/tags.rs @@ -101,6 +101,8 @@ pub enum TagError { NotSiblings, #[error("the parent must list each tag exactly once in its /K array")] AmbiguousChildren, + #[error("the tag has too much or too deeply nested marked content")] + ContentLimit, #[error("the tag is already at the requested position")] AlreadyThere, #[error("{0} applies only to {1} tags")] @@ -118,7 +120,13 @@ pub fn move_tag(doc: &mut Document, object: ObjRef, anchor: ObjRef, placement: T return Err(TagError::NotSiblings); } let reachable = alt::elements(doc); - if !reachable.iter().any(|e| e.object == object) || !reachable.iter().any(|e| e.object == anchor) { + let Some(obj_elem) = reachable.iter().find(|e| e.object == object) else { + return Err(TagError::NotSiblings); + }; + let Some(anc_elem) = reachable.iter().find(|e| e.object == anchor) else { + return Err(TagError::NotSiblings); + }; + if obj_elem.parent != anc_elem.parent { return Err(TagError::NotSiblings); } let parent_of = |r| doc.get(r).as_dict().and_then(|d| d.get(b"P").and_then(Object::as_ref)); @@ -126,8 +134,23 @@ pub fn move_tag(doc: &mut Document, object: ObjRef, anchor: ObjRef, placement: T if parent_of(anchor) != Some(parent) { return Err(TagError::NotSiblings); } + let valid_parent = match obj_elem.parent { + Some(expected) => parent == expected, + None => doc + .root() + .and_then(|r| doc.get(r).as_dict().and_then(|c| c.get(b"StructTreeRoot").and_then(Object::as_ref))) + .is_some_and(|root_ref| parent == root_ref), + }; + if !valid_parent { + return Err(TagError::NotSiblings); + } let owner = doc.get(parent); - let Some(children) = owner.as_dict().and_then(|d| d.get(b"K").and_then(Object::as_array)) else { + let mut k_obj = owner.as_dict().and_then(|d| d.get(b"K")).cloned(); + for _ in 0..16 { + let Some(Object::Ref(r)) = k_obj else { break }; + k_obj = Some((*doc.get(r)).clone()); + } + let Some(children) = k_obj.and_then(|k| k.as_array().cloned()) else { return Err(TagError::AmbiguousChildren); }; let one_position = |child| { @@ -135,7 +158,7 @@ pub fn move_tag(doc: &mut Document, object: ObjRef, anchor: ObjRef, placement: T matches.next().and_then(|(index, _)| matches.next().is_none().then_some(index)) }; let (Some(from), Some(to)) = (one_position(object), one_position(anchor)) else { return Err(TagError::AmbiguousChildren) }; - let mut reordered = children.clone(); + let mut reordered = children; let moved = reordered.remove(from); let target = if from < to { to - 1 } else { to }; let destination = target + usize::from(placement == TagPlacement::After); @@ -151,6 +174,10 @@ fn text(doc: &Document, d: &pdfcraft_cos::Dict, key: &[u8]) -> Option { d.get(key).map(|o| doc.resolve(o)).and_then(|o| o.as_string().map(PdfString::to_text)) } +fn tag_type_name(doc: &Document, d: &Dict) -> Option> { + d.get(b"S").map(|s| doc.resolve(s)).and_then(|s| s.as_name().map(<[u8]>::to_vec)) +} + /// List reachable, indirect tags in structure-tree order. pub fn tags(doc: &Document) -> Vec { let pages = pdfcraft_model::pages(doc); @@ -162,7 +189,7 @@ pub fn tags(doc: &Document) -> Vec { object: e.object, parent: e.parent, depth: e.depth, - kind: String::from_utf8_lossy(e.dict.name(b"S")?).into_owned(), + kind: String::from_utf8_lossy(&tag_type_name(doc, &e.dict)?).into_owned(), title: text(doc, &e.dict, b"T"), actual_text: text(doc, &e.dict, b"ActualText"), alt: text(doc, &e.dict, b"Alt"), @@ -176,13 +203,16 @@ pub fn tags(doc: &Document) -> Vec { } /// List marked-content references directly owned by an indirect tag. +/// Returns `ContentLimit` rather than a partial list beyond 65,536 marks or 512 child/reference +/// levels. Child structure elements do not count as direct marks. pub fn tag_content(doc: &Document, object: ObjRef) -> Result, TagError> { let Some(element) = alt::elements(doc).into_iter().find(|e| e.object == object) else { return Err(TagError::NotATag(object)) }; let pages = pdfcraft_model::pages(doc); let page_index: std::collections::HashMap = pages.iter().enumerate().map(|(i, p)| (p.obj, i)).collect(); let mut marks = Vec::new(); + let mut seen = std::collections::HashSet::new(); if let Some(kids) = element.dict.get(b"K") { - direct_marks(doc, kids, element.page, &mut marks); + direct_marks(doc, kids, element.page, &mut seen, &mut marks)?; } // Each page's content is parsed once, however many of its MCIDs the tag owns. let mut texts: std::collections::HashMap> = std::collections::HashMap::new(); @@ -206,25 +236,63 @@ pub fn tag_content(doc: &Document, object: ObjRef) -> Result .collect()) } -fn direct_marks(doc: &Document, item: &Object, page: Option, out: &mut Vec<(Option, i64)>) { - match item { - Object::Int(mcid) => out.push((page, *mcid)), - Object::Array(items) => items.iter().for_each(|item| direct_marks(doc, item, page, out)), - Object::Dict(dict) if dict.name(b"Type") == Some(b"MCR") => { - if let Some(mcid) = dict.get(b"MCID").and_then(Object::as_int) { - out.push((dict.get(b"Pg").and_then(Object::as_ref).or(page), mcid)); - } +/// Bound both graph work and the queued array entries. Following references never uses the +/// call stack; the depth limit also rejects pathologically nested direct content explicitly. +fn direct_marks( + doc: &Document, + item: &Object, + page: Option, + seen: &mut std::collections::HashSet, + out: &mut Vec<(Option, i64)>, +) -> Result<(), TagError> { + const MAX_DEPTH: usize = 512; + const MAX_MARKS: usize = 65_536; + // Account for the root `/K` container separately so an array of 65,536 entries passes. + let mut remaining = MAX_MARKS + 1; + let mut stack = vec![(std::sync::Arc::new(item.clone()), 0usize)]; + while let Some((item, depth)) = stack.pop() { + remaining = remaining.checked_sub(1).ok_or(TagError::ContentLimit)?; + if depth > MAX_DEPTH { + return Err(TagError::ContentLimit); } - Object::Ref(reference) => { - if let Some(dict) = doc.get(*reference).as_dict() - && dict.name(b"Type") == Some(b"MCR") - && let Some(mcid) = dict.get(b"MCID").and_then(Object::as_int) - { - out.push((dict.get(b"Pg").and_then(Object::as_ref).or(page), mcid)); + match &*item { + Object::Int(mcid) => { + if out.len() >= MAX_MARKS { + return Err(TagError::ContentLimit); + } + out.push((page, *mcid)); } + Object::Array(items) => { + if items.len() > remaining.saturating_sub(stack.len()) { + return Err(TagError::ContentLimit); + } + stack.extend(items.iter().rev().map(|item| (std::sync::Arc::new(item.clone()), depth + 1))); + } + Object::Dict(dict) if dict.get(b"Type").map(|t| doc.resolve(t)).as_deref().and_then(Object::as_name) == Some(b"MCR") => { + if let Some(mcid) = dict.get(b"MCID").map(|m| doc.resolve(m)).and_then(|m| m.as_int()) { + if out.len() >= MAX_MARKS { + return Err(TagError::ContentLimit); + } + out.push((dict.get(b"Pg").and_then(Object::as_ref).or(page), mcid)); + } + } + Object::Ref(reference) if seen.insert(*reference) => { + let target = doc.get(*reference); + // Resolved non-MCR dictionaries are child structure elements, not direct content, + // so do not push them onto the stack or charge them a second time. + let is_non_mcr_dict = matches!( + &*target, + Object::Dict(dict) + if dict.get(b"Type").map(|t| doc.resolve(t)).as_deref().and_then(Object::as_name) != Some(b"MCR") + ); + if !is_non_mcr_dict { + stack.push((target, depth + 1)); + } + } + _ => {} } - _ => {} } + Ok(()) } /// The attribute objects of a tag's `/A` entry (one object, or an array of them each optionally @@ -236,8 +304,8 @@ fn attribute_entries(doc: &Document, d: &Dict) -> (Vec<(Object, Option)> let Object::Array(items) = &*resolved else { return (vec![(a.clone(), None)], false) }; let mut out: Vec<(Object, Option)> = Vec::new(); for item in items { - match (item, out.last_mut()) { - (Object::Int(_), Some((_, revision @ None))) => *revision = Some(item.clone()), + match (doc.resolve(item).as_int(), out.last_mut()) { + (Some(_), Some((_, revision @ None))) => *revision = Some(item.clone()), _ => out.push((item.clone(), None)), } } @@ -245,13 +313,18 @@ fn attribute_entries(doc: &Document, d: &Dict) -> (Vec<(Object, Option)> } /// Whether attribute object `o` (resolved) is a Table attribute object. -fn is_table_attributes(o: &Object) -> bool { - o.as_dict().is_some_and(|d| d.name(b"O") == Some(b"Table")) +fn is_table_attributes(doc: &Document, o: &Object) -> bool { + o.as_dict().and_then(|d| d.get(b"O")).map(|v| doc.resolve(v)).as_deref().and_then(Object::as_name) == Some(b"Table") } /// The value of `key` in the first Table attribute object of tag `d` that has it. fn table_attribute(doc: &Document, d: &Dict, key: &[u8]) -> Option { - attribute_entries(doc, d).0.iter().map(|(o, _)| doc.resolve(o)).filter(|o| is_table_attributes(o)).find_map(|o| o.as_dict()?.get(key).cloned()) + attribute_entries(doc, d) + .0 + .iter() + .map(|(o, _)| doc.resolve(o)) + .filter(|o| is_table_attributes(doc, o)) + .find_map(|o| o.as_dict()?.get(key).map(|v| (*doc.resolve(v)).clone()).filter(|v| !matches!(v, Object::Null))) } /// Set `key` in the Table attribute object of `object`'s `/A` entry that holds it (else the first @@ -266,11 +339,12 @@ fn set_table_attribute(doc: &mut Document, object: ObjRef, key: &[u8], value: Op let mut entries: Vec<(Object, Option, Option)> = entries .into_iter() .map(|(o, r)| { - let table = Some(doc.resolve(&o)).filter(|a| is_table_attributes(a)).and_then(|a| a.as_dict().cloned()); + let table = Some(doc.resolve(&o)).filter(|a| is_table_attributes(doc, a)).and_then(|a| a.as_dict().cloned()); (o, r, table) }) .collect(); - let holding = |e: &(Object, Option, Option)| e.2.as_ref().is_some_and(|d| d.contains(key)); + let holding = + |e: &(Object, Option, Option)| e.2.as_ref().and_then(|d| d.get(key)).is_some_and(|v| !matches!(*doc.resolve(v), Object::Null)); match value { Some(v) => match entries.iter().position(holding).or_else(|| entries.iter().position(|e| e.2.is_some())) { Some(i) => { @@ -331,7 +405,7 @@ pub fn set_tag_property(doc: &mut Document, object: ObjRef, property: TagPropert if let Some(owner) = property.table_attribute_of() { let role_map = alt::tree_root(doc).and_then(|r| r.get(b"RoleMap").map(|o| doc.resolve(o))).and_then(|o| o.as_dict().cloned()).unwrap_or_default(); - let ty = element.dict.name(b"S").map(|s| alt::standard(doc, &role_map, s)).unwrap_or_default(); + let ty = tag_type_name(doc, &element.dict).map(|s| alt::standard(doc, &role_map, &s)).unwrap_or_default(); if ty != owner.as_bytes() { let name = if property == TagProperty::Summary { "summary" } else { "scope" }; return Err(TagError::WrongTagType(name, owner)); @@ -357,3 +431,168 @@ pub fn set_tag_property(doc: &mut Document, object: ObjRef, property: TagPropert })?; Ok(()) } + +#[cfg(test)] +mod tests { + use std::sync::Arc; + + use super::*; + + fn pdf(bodies: &[String]) -> Document { + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (i, b) in bodies.iter().enumerate() { + offsets.push(out.len()); + out.extend_from_slice(format!("{} 0 obj\n{b}\nendobj\n", i + 1).as_bytes()); + } + let xref = out.len(); + out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", bodies.len() + 1).as_bytes()); + for o in offsets { + out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + out.extend_from_slice(format!("trailer << /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", bodies.len() + 1).as_bytes()); + Document::open(Arc::new(out)).unwrap() + } + + #[test] + fn tag_content_and_table_attributes_resolve_indirect_entries() { + let content = "/P << /MCID 0 >> BDC BT /F1 12 Tf (Hello) Tj ET EMC /P << /MCID 1 >> BDC BT /F1 12 Tf (World) Tj ET EMC"; + let doc = pdf(&[ + "<< /Type /Catalog /Pages 2 0 R /StructTreeRoot 5 0 R >>".into(), + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>".into(), + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Contents 4 0 R /Resources << /Font << /F1 << /Type /Font /Subtype /Type1 /BaseFont /Helvetica >> >> >> >>".into(), + format!("<< /Length {} >>\nstream\n{content}\nendstream", content.len()), + "<< /Type /StructTreeRoot /K 6 0 R >>".into(), + "<< /S /Document /P 5 0 R /K [7 0 R 8 0 R] >>".into(), + "<< /S /Table /P 6 0 R /Pg 3 0 R /A << /O 11 0 R /Summary 12 0 R >> /K 9 0 R >>".into(), + "<< /S /TH /P 6 0 R /Pg 3 0 R /A [ << /O 11 0 R /Scope 13 0 R >> 14 0 R ] /K 10 0 R >>".into(), + "[15 0 R << /Type /MCR /MCID 16 0 R >>]".into(), + "15 0 R".into(), + "/Table".into(), + "(Indirect summary)".into(), + "/Column".into(), + "1".into(), + "0".into(), + "1".into(), + ]); + let listed = tags(&doc); + let table = listed.iter().find(|t| t.object == ObjRef::new(7, 0)).unwrap(); + let th = listed.iter().find(|t| t.object == ObjRef::new(8, 0)).unwrap(); + assert_eq!(table.summary.as_deref(), Some("Indirect summary")); + assert_eq!(th.scope.as_deref(), Some("Column")); + + let table_marks = tag_content(&doc, ObjRef::new(7, 0)).unwrap(); + assert_eq!( + table_marks, + [TagContentItem { page: Some(0), mcid: 0, text: "Hello".into() }, TagContentItem { page: Some(0), mcid: 1, text: "World".into() },] + ); + let th_marks = tag_content(&doc, ObjRef::new(8, 0)).unwrap(); + assert_eq!(th_marks, [TagContentItem { page: Some(0), mcid: 0, text: "Hello".into() }]); + } + + #[test] + fn move_tag_resolves_indirect_array_children_and_cycles() { + let mut doc = pdf(&[ + "<< /Type /Catalog /StructTreeRoot 2 0 R >>".into(), + "<< /Type /StructTreeRoot /K 3 0 R >>".into(), + "<< /S /Document /P 2 0 R /K 7 0 R >>".into(), + "<< /S /P /P 3 0 R >>".into(), + "<< /S /P /P 3 0 R >>".into(), + "[4 0 R 6 0 R 5 0 R]".into(), + "6 0 R".into(), + ]); + let first = ObjRef::new(4, 7); + let second = ObjRef::new(5, 9); + doc.set(first, (*doc.get(ObjRef::new(4, 0))).clone()); + doc.set(second, (*doc.get(ObjRef::new(5, 0))).clone()); + doc.set(ObjRef::new(6, 0), Object::Array(vec![Object::Ref(first), Object::Ref(ObjRef::new(6, 0)), Object::Ref(second)])); + assert_eq!(tags(&doc).iter().map(|t| t.object).collect::>(), [ObjRef::new(3, 0), first, second]); + move_tag(&mut doc, second, first, TagPlacement::Before).unwrap(); + assert_eq!(tags(&doc).iter().map(|t| t.object).collect::>(), [ObjRef::new(3, 0), second, first]); + } + + #[test] + fn tag_content_rejects_excessive_indirect_depth() { + let mut doc = + pdf(&["<< /Type /Catalog /StructTreeRoot 2 0 R >>".into(), "<< /Type /StructTreeRoot /K 3 0 R >>".into(), "<< /S /P /P 2 0 R >>".into()]); + let mut kid = doc.add(Object::Int(42)); + for _ in 0..128 { + kid = doc.add(Object::Ref(kid)); + } + doc.update_dict(ObjRef::new(3, 0), |d| d.set(b"K".to_vec(), Object::Ref(kid))).unwrap(); + assert_eq!(tag_content(&doc, ObjRef::new(3, 0)).unwrap().iter().map(|m| m.mcid).collect::>(), [42]); + for _ in 0..1024 { + kid = doc.add(Object::Ref(kid)); + } + doc.update_dict(ObjRef::new(3, 0), |d| d.set(b"K".to_vec(), Object::Ref(kid))).unwrap(); + assert_eq!(tag_content(&doc, ObjRef::new(3, 0)), Err(TagError::ContentLimit)); + } + + #[test] + fn tag_content_rejects_excessive_mark_count() { + let mut doc = + pdf(&["<< /Type /Catalog /StructTreeRoot 2 0 R >>".into(), "<< /Type /StructTreeRoot /K 3 0 R >>".into(), "<< /S /P /P 2 0 R >>".into()]); + doc.update_dict(ObjRef::new(3, 0), |d| d.set(b"K".to_vec(), Object::Array(vec![Object::Int(1); 65_537]))).unwrap(); + assert_eq!(tag_content(&doc, ObjRef::new(3, 0)), Err(TagError::ContentLimit)); + } + + #[test] + fn tag_content_accepts_full_mark_limit() { + let mut doc = + pdf(&["<< /Type /Catalog /StructTreeRoot 2 0 R >>".into(), "<< /Type /StructTreeRoot /K 3 0 R >>".into(), "<< /S /P /P 2 0 R >>".into()]); + doc.update_dict(ObjRef::new(3, 0), |d| d.set(b"K".to_vec(), Object::Array(vec![Object::Int(1); 65_536]))).unwrap(); + assert_eq!(tag_content(&doc, ObjRef::new(3, 0)).unwrap().len(), 65_536); + } + + #[test] + fn tag_content_skips_many_child_elements_without_spending_mark_budget() { + let mut doc = + pdf(&["<< /Type /Catalog /StructTreeRoot 2 0 R >>".into(), "<< /Type /StructTreeRoot /K 3 0 R >>".into(), "<< /S /P /P 2 0 R >>".into()]); + let mut kids = Vec::new(); + for _ in 0..33_000 { + let mut child = Dict::new(); + child.set(b"S".to_vec(), Object::name("P")); + kids.push(Object::Ref(doc.add(Object::Dict(child)))); + } + kids.push(Object::Int(42)); + doc.update_dict(ObjRef::new(3, 0), |d| d.set(b"K".to_vec(), Object::Array(kids))).unwrap(); + assert_eq!(tag_content(&doc, ObjRef::new(3, 0)).unwrap().iter().map(|m| m.mcid).collect::>(), [42]); + } + + #[test] + fn tag_content_keeps_order_and_stops_indirect_cycles() { + let doc = pdf(&[ + "<< /Type /Catalog /StructTreeRoot 2 0 R >>".into(), + "<< /Type /StructTreeRoot /K 3 0 R >>".into(), + "<< /S /P /P 2 0 R /K 4 0 R >>".into(), + "[1 5 0 R 4 0 R 3]".into(), + "<< /Type /MCR /MCID 2 >>".into(), + ]); + assert_eq!(tag_content(&doc, ObjRef::new(3, 0)).unwrap().iter().map(|m| m.mcid).collect::>(), [1, 2, 3]); + } + + #[test] + fn move_tag_rejects_non_sibling_tags_sharing_stale_parent_ref() { + // 8 0 R is a child of 6 0 R, while 9 0 R is a child of 7 0 R, even though both point /P at 6 0 R. + let mut doc = pdf(&[ + "<< /Type /Catalog /StructTreeRoot 2 0 R >>".into(), + "<< /Type /StructTreeRoot /K 3 0 R >>".into(), + "<< /S /Document /P 2 0 R /K [6 0 R 7 0 R] >>".into(), + "<< /S /P >>".into(), + "<< /S /P >>".into(), + "<< /S /Sect /P 3 0 R /K [8 0 R 9 0 R] >>".into(), + "<< /S /Sect /P 3 0 R /K [9 0 R] >>".into(), + "<< /S /P /P 6 0 R >>".into(), + "<< /S /P /P 6 0 R >>".into(), + ]); + // Give 6 0 R only [8 0 R] in the reachable tree, and put [8 0 R 9 0 R] on an unreachable object 4 0 R. + doc.update_dict(ObjRef::new(6, 0), |d| d.set(b"K".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(8, 0))]))).unwrap(); + doc.update_dict(ObjRef::new(4, 0), |d| { + d.set(b"K".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(8, 0)), Object::Ref(ObjRef::new(9, 0))])); + }) + .unwrap(); + doc.update_dict(ObjRef::new(8, 0), |d| d.set(b"P".to_vec(), Object::Ref(ObjRef::new(4, 0)))).unwrap(); + doc.update_dict(ObjRef::new(9, 0), |d| d.set(b"P".to_vec(), Object::Ref(ObjRef::new(4, 0)))).unwrap(); + assert_eq!(move_tag(&mut doc, ObjRef::new(9, 0), ObjRef::new(8, 0), TagPlacement::Before), Err(TagError::NotSiblings)); + } +} diff --git a/crates/a11y/src/tests.rs b/crates/a11y/src/tests.rs index d8566eddf1..890eb33e9f 100644 --- a/crates/a11y/src/tests.rs +++ b/crates/a11y/src/tests.rs @@ -142,17 +142,120 @@ fn autotag_text_says_why_it_cannot_read_a_page() { } #[test] -fn autotag_text_refuses_paint_and_does_not_modify_the_document() { +fn autotag_text_marks_vector_graphics_and_images_as_artifacts() { + let mut doc = pdf( + &[ + "<< /Type /Catalog /Pages 2 0 R >>".into(), + "<< /Type /Pages /Kids [3 0 R 7 0 R] /Count 2 >>".into(), + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> /XObject << /Im1 6 0 R >> >> >>".into(), + stream("0 0 200 200 m 200 0 l 200 200 l 0 200 l h f 10 10 50 50 re B BT /F1 12 Tf 20 100 Td (Hello) Tj ET q 20 0 0 20 80 80 cm /Im1 Do Q"), + FONT.into(), + "<< /Type /XObject /Subtype /Image /Width 1 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8 /Length 1 >>\nstream\n\u{0}\nendstream".into(), + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Contents 8 0 R >>".into(), + stream("0 0 100 100 re S"), + ], + "", + ); + let result = autotag_text(&mut doc).unwrap(); + assert_eq!(result, AutoTagResult { pages: 1, paragraphs: 1 }); + let report = check(&doc, &Options { rules: [Rule::TaggedPdf, Rule::TaggedContent].into(), pages: None }); + assert_eq!(status(&report, Rule::TaggedPdf), Status::Passed); + assert_eq!(status(&report, Rule::TaggedContent), Status::Passed); + let bytes = pdfcraft_cos::write_incremental(&doc, &pdfcraft_cos::SaveOptions::default()).unwrap(); + let reopened = Document::open(Arc::new(bytes)).unwrap(); + let reopened_report = check(&reopened, &Options { rules: [Rule::TaggedPdf, Rule::TaggedContent].into(), pages: None }); + assert_eq!(status(&reopened_report, Rule::TaggedPdf), Status::Passed); + assert_eq!(status(&reopened_report, Rule::TaggedContent), Status::Passed); +} + +#[test] +fn autotag_artifacts_wrap_complete_paths_and_leave_unpainted_clips_outside() { + let content = "0 0 m 10 0 l 10 10 l h f 0 0 20 20 re W B 0 0 50 50 re W* n /Shade sh BT /F1 12 Tf (Text) Tj ET 3 3 5 5 re S"; let mut doc = pdf( &[ "<< /Type /Catalog /Pages 2 0 R >>".into(), "<< /Type /Pages /Kids [3 0 R] /Count 1 >>".into(), "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >>".into(), - stream("BT /F1 12 Tf 20 100 Td (Hello) Tj ET 0 0 50 50 re f"), + stream(content), FONT.into(), ], "", ); + autotag_text(&mut doc).unwrap(); + let bytes = pdfcraft_cos::write_incremental(&doc, &pdfcraft_cos::SaveOptions::default()).unwrap(); + let doc = Document::open(Arc::new(bytes)).unwrap(); + let page = pdfcraft_model::pages(&doc).remove(0); + let content = doc.resolve(page.dict.get(b"Contents").unwrap()); + let Object::Stream(content) = &*content else { panic!("page stream") }; + let ops = pdfcraft_content::parse(&content.decoded_strict().unwrap()).ops; + let names: Vec<&[u8]> = ops.iter().map(|op| op.op.as_slice()).collect(); + assert_eq!( + names, + vec![ + b"BMC".as_slice(), + b"m", + b"l", + b"l", + b"h", + b"f", + b"EMC", + b"BMC", + b"re", + b"W", + b"B", + b"EMC", + b"re", + b"W*", + b"n", + b"BMC", + b"sh", + b"EMC", + b"BDC", + b"BT", + b"Tf", + b"Tj", + b"ET", + b"EMC", + b"BMC", + b"re", + b"S", + b"EMC", + ] + ); +} + +#[test] +fn autotag_rejects_text_interleaved_with_an_unfinished_path_atomically() { + for tail in ["10 10 l S", "n"] { + let mut doc = pdf( + &[ + "<< /Type /Catalog /Pages 2 0 R >>".into(), + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>".into(), + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >>".into(), + stream(&format!("0 0 m BT /F1 12 Tf (Text) Tj ET {tail}")), + FONT.into(), + ], + "", + ); + let before = pdfcraft_cos::write_incremental(&doc, &pdfcraft_cos::SaveOptions::default()).unwrap(); + assert!(matches!(autotag_text(&mut doc), Err(AutoTagError::Unsupported { .. }))); + assert_eq!(pdfcraft_cos::write_incremental(&doc, &pdfcraft_cos::SaveOptions::default()).unwrap(), before); + } +} + +#[test] +fn autotag_text_refuses_form_xobjects_and_does_not_modify_the_document() { + let mut doc = pdf( + &[ + "<< /Type /Catalog /Pages 2 0 R >>".into(), + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>".into(), + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> /XObject << /Fm1 6 0 R >> >> >>".into(), + stream("BT /F1 12 Tf 20 100 Td (Hello) Tj ET /Fm1 Do"), + FONT.into(), + "<< /Type /XObject /Subtype /Form /BBox [0 0 50 50] /Length 12 >>\nstream\n0 0 10 10 re f\nendstream".into(), + ], + "", + ); assert!(matches!(autotag_text(&mut doc), Err(AutoTagError::Unsupported { .. }))); assert!(!doc.get(doc.root().unwrap()).as_dict().unwrap().contains(b"StructTreeRoot")); assert_eq!(autotag_text(&mut pdf(&good(), "")), Err(AutoTagError::AlreadyTagged)); @@ -1559,6 +1662,24 @@ fn q_inside_a_text_object_restores_the_text_matrices() { assert_eq!(titles, ["Chapter One", "Chapter Two"]); } +#[test] +fn headings_naming_one_large_string_stop_decoding_it_at_the_allowance() { + let count = 100; + let kids = (0..count).map(|i| format!("{} 0 R", 8 + i)).collect::>().join(" "); + let mut objects = vec![ + "<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 5 0 R >>".to_string(), + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_string(), + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Contents 4 0 R >>".to_string(), + stream("BT ET"), + "<< /Type /StructTreeRoot /K 6 0 R >>".to_string(), + format!("<< /S /Document /P 5 0 R /K [{kids}] >>"), + format!("({})", "a".repeat(512 << 10)), + ]; + objects.extend((0..count).map(|_| "<< /S /H1 /P 6 0 R /Pg 3 0 R /ActualText 7 0 R >>".to_string())); + let found = headings(&pdf(&objects, "")); + assert!(!found.is_empty() && found.len() < count, "{}", found.len()); +} + #[test] fn an_untagged_document_has_no_headings() { let mut objs = headings_doc(); @@ -1745,3 +1866,54 @@ fn marking_a_tag_artifact_refuses_content_past_the_decode_budget() { assert_eq!(mark_tag_artifact(&mut doc, ObjRef::new(7, 0)), Err(ArtifactError::TooLarge)); assert_eq!(doc.get(ObjRef::new(3, 0)).as_dict().unwrap().get(b"Contents"), Some(&contents), "the page keeps its content"); } + +#[test] +fn autotag_text_refuses_pages_that_share_content_past_the_total_operator_allowance() { + // Each page is just under the per-page limit; sharing one stream makes the total 21 times it. + let content = format!("BT (a) Tj ET {}", "q Q ".repeat(49_990)); + let pages = 21; + let kids: Vec = (0..pages).map(|i| format!("{} 0 R", 4 + i)).collect(); + let mut bodies = vec![ + "<< /Type /Catalog /Pages 2 0 R >>".to_string(), + format!("<< /Type /Pages /Kids [{}] /Count {pages} /MediaBox [0 0 200 200] >>", kids.join(" ")), + stream(&content), + ]; + bodies.extend((0..pages).map(|_| "<< /Type /Page /Parent 2 0 R /Contents 3 0 R >>".to_string())); + let mut doc = pdf(&bodies, ""); + let Err(AutoTagError::Unsupported { reason, .. }) = autotag_text(&mut doc) else { panic!("expected a refusal") }; + assert!(reason.contains("too much content"), "{reason}"); +} + +#[test] +fn autotag_form_fields_tags_widgets_of_a_shared_annots_array_once() { + let mut doc = pdf( + &[ + "<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [5 0 R] >> >>".into(), + "<< /Type /Pages /Kids [3 0 R 6 0 R] /Count 2 >>".into(), + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Annots 7 0 R >>".into(), + "<< /Length 0 >>\nstream\n\nendstream".into(), + "<< /Type /Annot /Subtype /Widget /FT /Tx /T (name) /Rect [20 60 120 80] >>".into(), + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Annots 7 0 R >>".into(), + "[5 0 R 5 0 R]".into(), + ], + "", + ); + assert_eq!(tag_form_fields(&mut doc).unwrap(), 1); +} + +#[test] +fn checker_reports_each_untagged_annotation_without_copying_shared_dictionaries() { + let doc = pdf( + &[ + "<< /Type /Catalog /Pages 2 0 R >>".into(), + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>".into(), + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Annots [4 0 R 4 0 R 5 0 R] >>".into(), + "<< /Type /Annot /Subtype /Text /Rect [20 60 120 80] >>".into(), + "<< /Type /Annot /Subtype /Popup /Rect [20 60 120 80] >>".into(), + ], + "", + ); + let pages = pdfcraft_model::pages(&doc); + let found = content::page_rule(&doc, Rule::TaggedAnnotations, &pages, &[0], &Default::default(), false); + assert_eq!(found.len(), 2, "the popup is skipped, each listed text annotation reported"); +} diff --git a/crates/annot/README.md b/crates/annot/README.md index 999baec20a..3c258a2ab7 100644 --- a/crates/annot/README.md +++ b/crates/annot/README.md @@ -6,13 +6,19 @@ Comments (annotations), ISO 32000-2 §12.5. Layer L3, depends on `pdfcraft-cos`, - **Builders** for the comment types Acrobat's commenting tools create: sticky note (`Text` + `Popup`), highlight, underline, strikethrough, squiggly, rectangle (`Square`), oval (`Circle`), - line and arrow (`Line` with `/LE`), freehand drawing (`Ink`) and text box (`FreeText`). + line and arrow (`Line` with `/LE`), freehand drawing (`Ink`), text box and callout + (`FreeText`), polygon (including cloudy borders), polyline, caret, rubber and custom stamp, + and file attachment. - **Appearance streams** (`appearance::build`) generated from the annotation dictionary, so new and restyled comments look the same in every viewer. Highlights multiply (`/BM /Multiply`). A line's leader lines (`/LL`, `/LLE`, `/LLO`) and caption (`/Cap`, `/CP`, `/CO`) are drawn as ISO 32000-2 Table 178 places them, the caption in black Helvetica 10. Note icons are PdfCraft's own drawings. Text boxes use standard Helvetica (WinAnsi), with - line breaking by approximate Helvetica proportions (no font program is bundled). Text it can't + line breaking by Helvetica's standard-14 metrics. Thai in a text box, callout or Fill & Sign + text is shaped and drawn from the craft-fonts Thai face with Add text's writer + (`pdfcraft_fonts::shaped_text`: Type 3 glyphs with a ToUnicode map), the box sized to the ink + of its stacked marks and descenders; a build without a Thai face refuses it + (`AnnotError::Undrawable`) instead of drawing `?`. Other text it can't draw (in a text box, callout, Fill & Sign text, line caption, measurement label or dynamic stamp's by-line; `appearance::undrawable_text`) is kept as typed and drawn as `?`, and the comment is marked with the first such character (`pdfcraft_fonts::UNDRAWN_MARK`) so the engine reports the edit and @@ -21,7 +27,9 @@ Comments (annotations), ISO 32000-2 §12.5. Layer L3, depends on `pdfcraft-cos`, comment another program drew with a font that has those characters, whose appearance is kept rather than redrawn with `?` (new text for it is taken). - **Edits:** reply (`/IRT`), set review status (a `/State` reply, as Acrobat's "Set status" - does), change text, move, resize, restyle, delete (with pop-up and replies). + does), change text, move, resize, restyle, delete (with pop-up and replies), and open, close + or move the pop-up window (`set_popup`: `/Popup` `/Open` and `/Rect`, created beside a comment + that has none; a note's own `/Open` follows). `summaries` report each comment's pop-up. ## API sketch @@ -43,8 +51,8 @@ deterministic; the engine supplies them. - Unknown keys are kept. Inline annotation dictionaries are promoted to indirect objects only when an edit needs a reference to them. -- `set_style` refuses (and changes nothing) when it cannot redraw the comment, e.g. stamps, - cloudy borders or callouts, instead of leaving a stale appearance. +- `set_style` refuses when it cannot redraw the comment, e.g. a stamp + whose appearance cannot be regenerated, instead of leaving a stale appearance. - Replies get the parent's `/Rect` and an empty appearance: they appear in comment lists but never paint a second icon. - Resizing a stamp changes its `/Rect` and preserves its original appearance and resources; @@ -59,7 +67,9 @@ deterministic; the engine supplies them. (vendored hayro-interpret patch 37), and `Summary::no_rotate` tells the UI where the icon is drawn. -## Not yet +## Rich text and remaining limits -Callouts, polygons and polylines, clouds, carets (insert/replace text), stamps, file -attachments, rich text (`/RC`, `/DS`), and the shared named-appearance cache Acrobat writes. +Text-box text and colour edits regenerate `/RC` (XHTML) and `/DS` from the plain +`/Contents`, `/DA` and `/Q`, so rich-text editors show the same text and style. These edits +replace imported rich-text formatting. The shared named-appearance cache Acrobat writes +is not implemented. diff --git a/crates/annot/src/appearance.rs b/crates/annot/src/appearance.rs index fcb3cb22b5..e16803484a 100644 --- a/crates/annot/src/appearance.rs +++ b/crates/annot/src/appearance.rs @@ -3,16 +3,51 @@ //! [`build`] draws a normal appearance (`/AP /N`) from the annotation dictionary and the //! objects it points at, so the same code serves new comments and restyled ones. Drawings are in page space with //! `/BBox = /Rect` (identity matrix), except note icons, which are drawn in a 20 × 20 box. -//! It returns `None` for anything it cannot draw faithfully (cloudy borders, unknown line -//! endings), so callers never replace an appearance with a worse one. +//! It returns `None` for anything it cannot draw faithfully (unknown line endings, a cloudy +//! callout), so callers never replace an appearance with a worse one. It resolves `/QuadPoints` +//! itself; for the other values, callers that hold the document follow references first +//! (`draw::with_direct_values`). //! -//! The note icons are PdfCraft's own drawings (AGENTS.md §1). Text boxes use the standard -//! Helvetica font with WinAnsi encoding; line breaking uses [`text_width`], an approximation of -//! Helvetica's proportions by character class (no font program or metrics file is bundled). +//! The note icons are PdfCraft's own drawings (AGENTS.md §1). Text boxes use a standard text +//! font with WinAnsi encoding: Helvetica, or the face their `/DA` or `/DS` asks for +//! ([`crate::text_style`]), and draw Thai from the craft-fonts Thai face as Type 3 glyphs +//! (`pdfcraft_fonts::shaped_text`, the writer Add text uses); lines break at that face's +//! standard-14 widths and the Thai face's shaped advances (no other font program is bundled). use pdfcraft_cos::{Dict, Object, PdfString, Stream}; -use pdfcraft_fonts::{first_undrawable, literal, win_ansi}; -pub use pdfcraft_fonts::{helvetica_width as text_width, wrap}; +use pdfcraft_fonts::shaped_text; +use pdfcraft_fonts::{first_undrawable, literal, win_ansi, wrap_with}; + +/// The width of `text` at `size` as appearances draw it: Helvetica, with Thai shaped in the Thai +/// face that text boxes draw it with. +pub fn text_width(text: &str, size: f64) -> f64 { + text_width_in(pdfcraft_fonts::Std14::Helvetica, text, size) +} + +/// [`text_width`] in the standard `face` a text box draws its other text in. +pub fn text_width_in(face: pdfcraft_fonts::Std14, text: &str, size: f64) -> f64 { + shaped_text::thai_width(text, size, |s, size| face.text_width(s, size)) +} + +/// `text` wrapped `width` wide as [`text_width`] measures it; a Thai mark stays on its letter. +pub fn wrap(text: &str, size: f64, width: f64) -> Vec { + wrap_with(text, size, width, text_width) +} + +/// How far a text box's `lines` reach, in em: above the first baseline (0.9 of the 1.2 em line) +/// and below the last (0.3), or further where Thai stacked marks or descenders reach, so the box +/// is sized to their ink and doesn't clip them. +fn reach(lines: &[String]) -> [f64; 2] { + let ink = |line: Option<&String>| line.and_then(|l| shaped_text::thai_reach(l)); + [ink(lines.first()).map_or(0.9, |r| r[1].max(0.9)), ink(lines.last()).map_or(0.3, |r| (-r[0]).max(0.3))] +} + +/// The height a text box's `lines` take at `size`, inside its padding: 1.2 em a line, and more +/// for Thai that reaches further ([`reach`]). +pub fn text_lines_height(lines: &[String], size: f64) -> f64 { + let [up, down] = reach(lines); + lines.len().max(1) as f64 * size * 1.2 + (up - 0.9 + down - 0.3) * size +} use crate::{Document, NOTE_SIZE, Rgb, n}; @@ -26,7 +61,9 @@ pub fn arrow_size(w: f64) -> f64 { 6.0 + 3.0 * w.max(0.0) } -/// Nominal radius of a cloudy border's bumps for a line of width `w` (intensity 1). +/// Nominal radius of a cloudy border's bumps for a line of width `w` (intensity 1). ISO 32000-2 +/// Table 169 leaves the size open; this is the size PdfCraft's polygon clouds already had, so +/// squares, circles and text boxes match them. pub fn cloud_radius(w: f64) -> f64 { 6.0 + 1.5 * w.max(0.0) } @@ -120,6 +157,29 @@ pub fn smooth_segments(pts: &[(f64, f64)]) -> Vec<[(f64, f64); 3]> { .collect() } +/// The points where the cubic Bézier segment from `p0` (control points `c1`, `c2`) to `p3` turns +/// back in x or in y, strictly between its ends: with its ends, they bound the segment. +fn cubic_extremes(p0: (f64, f64), c1: (f64, f64), c2: (f64, f64), p3: (f64, f64)) -> Vec<(f64, f64)> { + let at = |t: f64| { + let u = 1.0 - t; + let w = [u * u * u, 3.0 * u * u * t, 3.0 * u * t * t, t * t * t]; + (w[0] * p0.0 + w[1] * c1.0 + w[2] * c2.0 + w[3] * p3.0, w[0] * p0.1 + w[1] * c1.1 + w[2] * c2.1 + w[3] * p3.1) + }; + let mut out = Vec::new(); + for (q0, q1, q2, q3) in [(p0.0, c1.0, c2.0, p3.0), (p0.1, c1.1, c2.1, p3.1)] { + // The derivative over 3: a t² + 2 b t + c. + let (a, b, c) = (-q0 + 3.0 * q1 - 3.0 * q2 + q3, q0 - 2.0 * q1 + q2, q1 - q0); + let roots = if a.abs() < 1e-12 { + if b.abs() < 1e-12 { vec![] } else { vec![-c / (2.0 * b)] } + } else { + let disc = b * b - a * c; + if disc < 0.0 { vec![] } else { vec![(-b + disc.sqrt()) / a, (-b - disc.sqrt()) / a] } + }; + out.extend(roots.into_iter().filter(|t| *t > 0.0 && *t < 1.0).map(at)); + } + out +} + /// `c` operators for [`smooth_segments`] (the path already starts at the first point). fn smooth_curve(pts: &[(f64, f64)]) -> String { smooth_segments(pts).iter().map(|[a, b, e]| format!("{} {} {} {} {} {} c\n", n(a.0), n(a.1), n(b.0), n(b.1), n(e.0), n(e.1))).collect() @@ -128,6 +188,80 @@ fn smooth_curve(pts: &[(f64, f64)]) -> String { /// Bumps one cloudy outline draws at most (plus one per edge); past it the bumps grow. pub(crate) const MAX_CLOUD_BUMPS: usize = 20_000; +/// A cloudy border's intensity (ISO 32000-2 §12.5.4, Table 169: `/BE /I`, "in the range 0 to 2. +/// Default value: 0"), or `None` for no cloud: no `/BE`, a style other than `/C`, or intensity 0, +/// which draws the border plain. +pub(crate) fn cloud_intensity(d: &Dict) -> Option { + let be = d.get(b"BE").and_then(|b| b.as_dict())?; + if be.name(b"S") != Some(b"C") { + return None; + } + let i = be.get(b"I").and_then(|i| i.as_f64()).unwrap_or(0.0); + (i.is_finite() && i > 0.0).then(|| i.min(2.0)) +} + +/// The bump radius for a cloudy border of width `w` at `intensity` (0–2]. +pub(crate) fn cloud_bump(w: f64, intensity: f64) -> f64 { + cloud_radius(w) * intensity.clamp(0.5, 2.0) +} + +/// How far past its outline a cloud with bumps of radius `bump` around an outline `perimeter` +/// long can reach. `cloud_path` rounds each edge to whole bumps, so one may span up to three +/// radii and bulge 1.5 radii; it also grows the radius to keep within its bump budget. +pub(crate) fn cloud_reach(perimeter: f64, bump: f64) -> f64 { + 1.5 * bump.max(perimeter / (2.0 * MAX_CLOUD_BUMPS as f64)) +} + +/// `/RD` when it is four finite, non-negative numbers (Tables 170 and 180). +fn rd_of(d: &Dict) -> Option> { + nums(d, b"RD").filter(|r| r.len() == 4 && r.iter().all(|x| x.is_finite() && *x >= 0.0)) +} + +/// `rect` less `rd`, whose four numbers are the left, top, right and bottom differences. +fn less_rd(rect: [f64; 4], rd: &[f64]) -> [f64; 4] { + let at = |i: usize| rd.get(i).copied().unwrap_or(0.0); + [rect[0] + at(0), rect[1] + at(3), rect[2] - at(2), rect[3] - at(1)] +} + +/// The box a plain (not cloudy) square or circle is drawn in: `/Rect` less a usable `/RD` (Table +/// 180: the differences between `/Rect` and "the actual boundaries of the underlying square or +/// circle"), or `None` when there is no `/RD`, it isn't four finite non-negative numbers, or it +/// leaves less than a point either way, so the caller draws in `/Rect` as without one. +fn rect_less_rd(d: &Dict, rect: [f64; 4]) -> Option<[f64; 4]> { + let inner = less_rd(rect, &rd_of(d)?); + (inner[2] - inner[0] >= 1.0 && inner[3] - inner[1] >= 1.0).then_some(inner) +} + +/// The shape a cloudy square, circle or text box draws its bumps around: `/Rect` less `/RD` +/// (Table 180: the differences between "the left, top, right, and bottom coordinates of Rect and +/// those of the square or circle, respectively"), but at least the bumps' greatest reach and half +/// the line from each side, so they stay inside `/Rect`; without a usable `/RD`, that margin. +fn cloud_inner(d: &Dict, rect: [f64; 4], bump: f64, w: f64) -> Option<[f64; 4]> { + // The cloud's reach and half the line keep it inside `/Rect`; an `/RD` leaving more is + // followed, one leaving less is not. + let perimeter = 2.0 * ((rect[2] - rect[0]) + (rect[3] - rect[1])); + let m = cloud_reach(perimeter, bump) + w / 2.0; + let safe = [rect[0] + m, rect[1] + m, rect[2] - m, rect[3] - m]; + let inner = match rd_of(d) { + Some(rd) => { + let r = less_rd(rect, &rd); + [r[0].max(safe[0]), r[1].max(safe[1]), r[2].min(safe[2]), r[3].min(safe[3])] + } + None => safe, + }; + (inner[2] - inner[0] >= 1.0 && inner[3] - inner[1] >= 1.0).then_some(inner) +} + +/// Points around the ellipse inscribed in `r`, counter-clockwise, one per bump of radius `bump` +/// (each edge of the outline becomes one bump), for a cloudy circle. +fn ellipse_points(r: [f64; 4], bump: f64) -> Vec<(f64, f64)> { + let (cx, cy, rx, ry) = ((r[0] + r[2]) / 2.0, (r[1] + r[3]) / 2.0, (r[2] - r[0]) / 2.0, (r[3] - r[1]) / 2.0); + // Ramanujan's approximation of the perimeter; the count stays within the bump budget. + let perimeter = std::f64::consts::PI * (3.0 * (rx + ry) - ((3.0 * rx + ry) * (rx + 3.0 * ry)).max(0.0).sqrt()); + let count = (perimeter / (2.0 * bump.max(0.5))).round().clamp(8.0, MAX_CLOUD_BUMPS as f64) as u32; + (0..count).map(|i| f64::from(i) * std::f64::consts::TAU / f64::from(count)).map(|a| (cx + rx * a.cos(), cy + ry * a.sin())).collect() +} + /// A closed cloudy outline through `pts` (§12.5.4 `/BE /S /C`): each edge becomes a row of /// half-circle bumps of about `r`, bulging outwards. pub fn cloud_path(pts: &[(f64, f64)], r: f64) -> String { @@ -178,11 +312,24 @@ fn quads(context: &AppearanceContext<'_>) -> Option> { (!q.is_empty()).then_some(q) } -/// `/C`-style colour arrays: none (transparent), gray, RGB or CMYK. -fn color(d: &Dict, key: &[u8]) -> Option> { +/// `/C`-style colour arrays (ISO 32000-2 Table 166): none (transparent), gray, RGB or CMYK. +/// The entries should all be numbers; a four-entry array with a non-number in it, such as +/// `[1 0 0 null]`, is still read as CMYK with that entry 0, as PDFium, MuPDF and Poppler read it. +/// A reference is not such an entry: it stands for a value this builder can't see (§7.3.10), so +/// the colour is unreadable, as the other colour and number readers here treat one. +pub(crate) fn color(d: &Dict, key: &[u8]) -> Option> { let c = match d.get(key) { None => return Some(None), - Some(o) => o.as_array()?.iter().map(|o| o.as_f64()).collect::>>()?, + Some(o) => match o.as_array()? { + a if a.len() == 4 => a + .iter() + .map(|o| match o { + Object::Ref(_) => None, + o => Some(o.as_f64().unwrap_or(0.0)), + }) + .collect::>>()?, + a => a.iter().map(|o| o.as_f64()).collect::>>()?, + }, }; Some(match c.as_slice() { [] => None, @@ -269,13 +416,17 @@ pub fn da_border(d: &Dict) -> Option { col } -/// `/DA` of a text box: (text colour, font size). Defaults to black 12 pt. +/// `/DA` of a text box: (text colour, font size). Defaults to black 12 pt: a `Tf` without a +/// usable size (missing, zero, negative or not a number) keeps 12 pt, and sizes past 400 pt are +/// capped. A text box has no automatic size, so a zero size, which means auto-size in a form +/// field's `/DA` (ISO 32000-2 §12.7.4.3), is the default here. An operand that isn't a finite +/// number (`NaN`, `inf`, which Rust's parser would otherwise accept) is ignored with its operator. pub fn parse_da(d: &Dict) -> (Rgb, f64) { let da = d.get(b"DA").and_then(|o| o.as_string()).map(|s| String::from_utf8_lossy(&s.bytes).into_owned()).unwrap_or_default(); let toks: Vec<&str> = da.split_whitespace().collect(); let mut col = [0.0; 3]; let mut size = 12.0; - let num = |i: usize| toks.get(i).and_then(|t| t.parse::().ok()); + let num = |i: usize| toks.get(i).and_then(|t| t.parse::().ok()).filter(|v| v.is_finite()); for (i, t) in toks.iter().enumerate() { match *t { "rg" if i >= 3 => { @@ -392,16 +543,20 @@ pub(crate) fn inset(d: &Dict) -> Option<[f64; 4]> { /// The first text [`build`] would draw in the standard fonts (Helvetica, WinAnsiEncoding) that /// they can't, which would come out as `?`: what it is (for the message) and the character. The -/// text of a text box, callout or Fill & Sign text, a line's caption, a measurement's label, a +/// text of a text box, callout or Fill & Sign text (whose Thai the Thai face draws, where this +/// build has one), a line's caption, a measurement's label, a /// dynamic stamp's "By …" line, and a redaction mark's overlay text (drawn when it is applied). `None` when all of /// it can be drawn. pub fn undrawable_text(d: &Dict) -> Option<(&'static str, char)> { let text = |key: &[u8]| d.get(key).and_then(Object::as_string).map(PdfString::to_text); let (what, text) = match d.name(b"Subtype")? { - b"FreeText" => ("the comment's text", text(b"Contents")?), + // A text box draws Thai too. + b"FreeText" => return Some(("the comment's text", shaped_text::first_undrawable(&text(b"Contents")?)?)), b"Line" if !d.contains(b"PCMeasureValue") => ("the line's caption", line_caption(d)?), b"Line" | b"PolyLine" | b"Polygon" => ("the measurement label", text(b"PCMeasureValue")?), - b"Stamp" if matches!(d.get(b"PCStamp"), Some(Object::Bool(true))) => ("the stamp's \"By\" line", text(b"PCByLine")?), + b"Stamp" if matches!(d.get(b"PCStamp"), Some(Object::Bool(true))) || d.contains(b"PCPicture") => { + ("the stamp's \"By\" line", text(b"PCByLine")?) + } // Drawn in Helvetica when the redaction is applied: refused when it is marked. b"Redact" => ("the redaction's overlay text", text(b"OverlayText")?), _ => return None, @@ -411,14 +566,20 @@ pub fn undrawable_text(d: &Dict) -> Option<(&'static str, char)> { /// `/Font` resources naming `/Helv`: standard Helvetica with WinAnsiEncoding. pub(crate) fn helvetica_resource() -> Object { + Object::Dict(font_resources(pdfcraft_fonts::Std14::Helvetica)) +} + +/// `/Font` resources naming one standard face with WinAnsiEncoding, under the resource name +/// [`crate::text_style::resource_name`] gives it (`/Helv` for Helvetica). +fn font_resources(face: pdfcraft_fonts::Std14) -> Dict { let mut font = Dict::new(); font.set(b"Type".to_vec(), Object::name("Font")); font.set(b"Subtype".to_vec(), Object::name("Type1")); - font.set(b"BaseFont".to_vec(), Object::name("Helvetica")); + font.set(b"BaseFont".to_vec(), Object::name(face.base_font())); font.set(b"Encoding".to_vec(), Object::name("WinAnsiEncoding")); let mut fonts = Dict::new(); - fonts.set(b"Helv".to_vec(), Object::Dict(font)); - Object::Dict(fonts) + fonts.set(crate::text_style::resource_name(face).as_bytes().to_vec(), Object::Dict(font)); + fonts } /// Where a caret and its paragraph symbol go within `rect`: the box the caret is drawn in, and for @@ -464,7 +625,8 @@ struct AppearanceContext<'a> { dict: &'a Dict, subtype: &'a [u8], rect: [f64; 4], - cloudy: bool, + /// A cloudy border's intensity ([`cloud_intensity`]), or `None` for a plain one. + cloud: Option, /// The line's opacity (`parts.line`). opacity: f64, parts: PartOpacity, @@ -498,13 +660,22 @@ impl AppearanceDrawing { /// Text the standard fonts can't draw comes out as `?`: [`undrawable_text`] says which, and /// [`crate::set_appearance`] marks the annotation with it (`pdfcraft_fonts::UNDRAWN_MARK`). pub fn build(doc: &Document, d: &Dict) -> Option { - let context = appearance_context(doc, d)?; + build_on_page(doc, d, None) +} + +/// [`build`] for an annotation on a page whose visible region is `page` (`Page::crop` in +/// pdfcraft-model): a text markup, line, polyline, polygon or ink annotation whose points reach +/// past its `/Rect` is drawn through them as far as the page shows ([`point_reach`]), and its +/// appearance's `/BBox` says how far ([`widened_rect`]). Without a page it is drawn within `/Rect`. +pub(crate) fn build_on_page(doc: &Document, d: &Dict, page: Option<[f64; 4]>) -> Option { + let context = appearance_context(doc, d, page)?; let drawing = AppearanceDrawing::new(&context); match context.subtype { b"Text" => return draw_icon(&context, "Note", [1.0, 0.82, 0.0], drawing), b"FileAttachment" => return draw_icon(&context, "PushPin", [0.0, 0.47, 0.84], drawing), b"Stamp" => return draw_stamp(&context, drawing), - b"FreeText" => return draw_free_text(&context, drawing), + // The face a text box's `/DA` font names in the form's `/DR`, when it names one. + b"FreeText" => return draw_free_text(&context, drawing, crate::text_style::dr_face(doc, d)), _ => {} } let mut drawing = drawing; @@ -512,18 +683,20 @@ pub fn build(doc: &Document, d: &Dict) -> Option { finish_appearance(&context, drawing) } -fn appearance_context<'a>(doc: &'a Document, d: &'a Dict) -> Option> { +fn appearance_context<'a>(doc: &'a Document, d: &'a Dict, page: Option<[f64; 4]>) -> Option> { // Imported measurement appearances may include leaders, captions and formatting that // this builder cannot reproduce. Preserve them rather than silently losing detail. if d.contains(b"Measure") && !d.contains(b"PCMeasureValue") { return None; } let subtype = d.name(b"Subtype")?; - let rect = nums(d, b"Rect").filter(|r| r.len() == 4)?; - let rect = [rect[0].min(rect[2]), rect[1].min(rect[3]), rect[0].max(rect[2]), rect[1].max(rect[3])]; - // Cloudy borders (§12.5.4) are drawn for polygons only. - let cloudy = d.get(b"BE").and_then(|b| b.as_dict()).is_some_and(|b| b.name(b"S") == Some(b"C")); - if cloudy && subtype != b"Polygon" { + // A number too long for a double reads as infinite, and two near its limit are an infinite + // width apart: nothing is drawn rather than "inf" written into the appearance (review on #4195). + let rect = normalised_rect(d).filter(|r| r.iter().all(|x| x.is_finite()) && (r[2] - r[0]).is_finite() && (r[3] - r[1]).is_finite())?; + let rect = page.and_then(|page| point_reach(doc, d, rect, page)).unwrap_or(rect); + // Cloudy borders (§12.5.4, Table 169) apply to squares, circles, polygons and text boxes. + let cloud = cloud_intensity(d); + if cloud.is_some() && !matches!(subtype, b"Square" | b"Circle" | b"Polygon" | b"FreeText") { return None; } Some(AppearanceContext { @@ -531,7 +704,7 @@ fn appearance_context<'a>(doc: &'a Document, d: &'a Dict) -> Option(doc: &'a Document, d: &'a Dict) -> Option Option<[f64; 4]> { + let [x0, y0, x1, y1] = <[f64; 4]>::try_from(nums(d, b"Rect")?).ok()?; + Some([x0.min(x1), y0.min(y1), x0.max(x1), y0.max(y1)]) +} + +/// The subtypes drawn through points of their own rather than within `/Rect`: text markup over +/// its `/QuadPoints` (ISO 32000-2 §12.5.6.10), lines along `/L` (§12.5.6.7), polygons and +/// polylines through `/Vertices` (§12.5.6.9) and ink through `/InkList` (§12.5.6.13). +pub(crate) fn is_drawn_from_points(d: &Dict) -> bool { + matches!(d.name(b"Subtype"), Some(b"Highlight" | b"Underline" | b"StrikeOut" | b"Squiggly" | b"Line" | b"PolyLine" | b"Polygon" | b"Ink")) +} + +/// At most this many quadrilaterals widen a text markup's box ([`point_reach`]). Each takes in a +/// word or a group of contiguous words (ISO 32000-2 Table 182), usually a line, so even written +/// word by word a markup over a whole dense page stays well below it; a larger one is drawn +/// within its `/Rect`, as before the box was widened. +pub(crate) const MAX_MARKUP_QUADS: usize = 10_000; + +/// At most this many points widen a line's, polyline's, polygon's or ink annotation's box +/// ([`point_reach`]): as many as [`MAX_MARKUP_QUADS`] quadrilaterals have. A larger one is drawn +/// within its `/Rect`. +pub(crate) const MAX_REACH_POINTS: usize = 4 * MAX_MARKUP_QUADS; + +/// The points `d` is drawn through, as `x y` pairs, and how far its strokes reach past them: +/// half the border width, and for a line ending (`/LE`) or a cloudy border (§12.5.4) the size +/// they are drawn at. `None` for other subtypes, more points than the caps allow, and points that +/// aren't all finite numbers. +fn drawn_points(doc: &Document, d: &Dict) -> Option<(Vec, f64)> { + let w = border_width(d); + let has_ending = || d.get(b"LE").and_then(Object::as_array).is_some_and(|e| e.iter().any(|e| e.as_name().is_some_and(|e| e != b"None"))); + let (points, margin) = match d.name(b"Subtype")? { + subtype @ (b"Highlight" | b"Underline" | b"StrikeOut" | b"Squiggly") => { + // The quads `draw_markup` draws (`quads`: resolved, malformed ones skipped), counted + // before the numbers are read. + let array = doc.resolve(d.get(b"QuadPoints")?); + let entries = array.as_array()?; + if entries.len() / 8 > MAX_MARKUP_QUADS { + return None; + } + let quads = pdfcraft_geom::quad_points::usable_quads(entries.iter().map(|x| doc.resolve(x).as_f64())); + if quads.is_empty() { + return None; + } + // How far each quad's stroke reaches past it: half the line's width, and a squiggle's + // peaks above a quad lower than they are (`draw_markup`). + let margin = quads.iter().map(|q| markup_overreach(subtype, (q[0] - q[4]).hypot(q[1] - q[5]))).fold(0.0, f64::max); + (quads.into_iter().flatten().collect(), margin) + } + b"Ink" => { + let strokes = d.get(b"InkList")?.as_array()?; + let count = strokes.iter().try_fold(0usize, |sum, s| sum.checked_add(s.as_array()?.len()))?; + if count / 2 > MAX_REACH_POINTS { + return None; + } + let mut points = Vec::with_capacity(count.saturating_mul(3)); + for s in strokes { + let stroke: Vec = s.as_array()?.iter().map(Object::as_f64).collect::>()?; + // A stroke's odd last number is no point (`draw_ink` skips it too). + let pts: Vec<(f64, f64)> = stroke.as_chunks::<2>().0.iter().map(|p| (p[0], p[1])).collect(); + points.extend(pts.iter().flat_map(|p| [p.0, p.1])); + // The smooth curve through three or more points (`smooth_segments`) can bulge + // past them: its turning points count too. + if pts.len() > 2 { + let mut start = pts.first().copied().unwrap_or_default(); + for [c1, c2, end] in smooth_segments(&pts) { + points.extend(cubic_extremes(start, c1, c2, end).iter().flat_map(|p| [p.0, p.1])); + start = end; + } + } + } + (points, w / 2.0) + } + subtype @ (b"PolyLine" | b"Polygon") => { + let count = d.get(b"Vertices")?.as_array()?.len(); + if count < 4 || count % 2 != 0 || count / 2 > MAX_REACH_POINTS { + return None; + } + let vertices = nums(d, b"Vertices")?; + // A cloud's bumps (`draw_polygon`) reach as far past the outline as `cloud_reach` says. + let reach = match (subtype, cloud_intensity(d)) { + (b"Polygon", Some(intensity)) => { + let pts: Vec<(f64, f64)> = vertices.as_chunks::<2>().0.iter().map(|p| (p[0], p[1])).collect(); + let perimeter: f64 = pts.iter().zip(pts.iter().cycle().skip(1)).map(|(a, b)| (b.0 - a.0).hypot(b.1 - a.1)).sum(); + cloud_reach(if perimeter.is_finite() { perimeter } else { 0.0 }, cloud_bump(w, intensity)) + } + (b"PolyLine", _) if has_ending() => arrow_size(w), + _ => 0.0, + }; + (vertices, w / 2.0 + reach) + } + b"Line" => { + let [x0, y0, x1, y1] = <[f64; 4]>::try_from(nums(d, b"L")?).ok()?; + if ![x0, y0, x1, y1].iter().all(|v| v.is_finite()) { + return None; + } + // Leader lines move the line off `/L` (§12.5.6.7, Table 178). + let layout = line_layout(d, [(x0, y0), (x1, y1)], w); + let mut points = vec![x0, y0, x1, y1]; + for (x, y) in layout.line.iter().chain(layout.leaders.iter().flatten()).chain(layout.tick.iter().flatten()) { + points.extend([*x, *y]); + } + // The caption's box: its width along the text matrix, from the descender to the + // cap height and a little over (Helvetica, as `draw_line` sets it). + if let Some((text, m)) = &layout.caption { + let width = text_width(text, CAPTION_SIZE); + for (x, y) in [(0.0, -0.25), (width, -0.25), (0.0, 1.0), (width, 1.0)].map(|(x, y)| (x, y * CAPTION_SIZE)) { + points.extend([m[0] * x + m[2] * y + m[4], m[1] * x + m[3] * y + m[5]]); + } + } + (points, w / 2.0 + if has_ending() { arrow_size(w) } else { 0.0 }) + } + _ => return None, + }; + (points.iter().all(|v| v.is_finite()) && margin.is_finite()).then_some((points, margin)) +} + +/// An annotation's `/Rect` (normalised, `rect`) grown to take in the points it is drawn through +/// ([`drawn_points`]) where they reach past it, as far as the page's visible region `page` goes; +/// `None` when they don't, aren't usable or number more than the caps allow, and for subtypes +/// drawn within `/Rect` ([`is_drawn_from_points`]). The points are what the annotation draws (ISO +/// 32000-2 Table 182 `/QuadPoints`, Table 178 `/L`, Table 181 `/Vertices`, Table 185 `/InkList`), +/// and the appearance is mapped onto `/Rect` (§12.5.5), so a `/Rect` that leaves them out would +/// cut off what is drawn: a highlight or ink stroke whose `/Rect` lies beside it drew nothing. +/// Once a point is outside, the box takes in the strokes' width and line endings too. Nothing +/// outside the crop box is shown or printed (§14.11.2), so the box grows no further than it; it +/// never shrinks `rect`, and a `/Rect` that holds every point is kept as it is. +fn point_reach(doc: &Document, d: &Dict, rect: [f64; 4], page: [f64; 4]) -> Option<[f64; 4]> { + if !rect.iter().all(|v| v.is_finite()) { + return None; + } + let (points, margin) = drawn_points(doc, d)?; + let mut reach = [f64::INFINITY, f64::INFINITY, f64::NEG_INFINITY, f64::NEG_INFINITY]; + for p in points.as_chunks::<2>().0 { + reach = [reach[0].min(p[0]), reach[1].min(p[1]), reach[2].max(p[0]), reach[3].max(p[1])]; + } + let clip = |b: [f64; 4]| [b[0].max(page[0]), b[1].max(page[1]), b[2].min(page[2]), b[3].min(page[3])]; + let shown = clip(reach); + // Strokes wholly off the page widen nothing. Points `/Rect` already holds don't either, and + // points off the page whose strokes reach onto it only as far as `/Rect` holds. + let stroked = clip([reach[0] - margin, reach[1] - margin, reach[2] + margin, reach[3] + margin]); + if stroked[0] > stroked[2] || stroked[1] > stroked[3] { + return None; + } + let inside = |b: [f64; 4]| rect[0] <= b[0] && rect[1] <= b[1] && b[2] <= rect[2] && b[3] <= rect[3]; + let on_page = shown[0] <= shown[2] && shown[1] <= shown[3]; + if inside(if on_page { shown } else { stroked }) { + return None; + } + let reach = stroked; + let out = [rect[0].min(reach[0]), rect[1].min(reach[1]), rect[2].max(reach[2]), rect[3].max(reach[3])]; + // A `/Rect` of its own can still be too wide for a width or height (`-1e308` to `1e308`). + let usable = (out[2] - out[0]).is_finite() && (out[3] - out[1]).is_finite(); + (usable && out != rect).then_some(out) +} + +/// The `/Rect` an annotation takes with its built appearance `stream`: the appearance's `/BBox` +/// where [`point_reach`] widened it past `/Rect`, as the appearance is mapped onto `/Rect` +/// (§12.5.5); `None` (keep `/Rect`) otherwise and for subtypes drawn within `/Rect`. +pub(crate) fn widened_rect(d: &Dict, stream: &Stream) -> Option<[f64; 4]> { + if !is_drawn_from_points(d) { + return None; + } + let bbox = <[f64; 4]>::try_from(nums(&stream.dict, b"BBox")?).ok()?; + (bbox.iter().all(|v| v.is_finite()) && Some(bbox) != normalised_rect(d)).then_some(bbox) +} + fn draw_body(context: &AppearanceContext<'_>, drawing: &mut AppearanceDrawing) -> Option<()> { match context.subtype { b"Highlight" | b"Underline" | b"StrikeOut" | b"Squiggly" => draw_markup(context, drawing), @@ -574,6 +915,30 @@ fn finish_appearance(context: &AppearanceContext<'_>, mut drawing: AppearanceDra Some(form(context.rect, &out, drawing.resources)) } +/// The line width of an underline or strike-out over a quad `h` high. +fn markup_line_width(h: f64) -> f64 { + (h * 0.07).clamp(0.5, 3.0) +} + +/// A squiggle's line width and amplitude over a quad `h` high: its peaks stand twice the +/// amplitude above the quad's bottom edge. +fn squiggle(h: f64) -> (f64, f64) { + ((h * 0.05).clamp(0.5, 2.0), (h * 0.06).max(0.75)) +} + +/// How far a text markup of `subtype` drawn over a quad `h` high reaches past the quad: half its +/// line's width, and for a squiggle, how far its peaks stand above a quad lower than they are. +fn markup_overreach(subtype: &[u8], h: f64) -> f64 { + match subtype { + b"Underline" | b"StrikeOut" => markup_line_width(h) / 2.0, + b"Squiggly" => { + let (lw, amp) = squiggle(h); + lw / 2.0 + (amp * 2.0 - h).max(0.0) + } + _ => 0.0, + } +} + fn draw_markup(context: &AppearanceContext<'_>, drawing: &mut AppearanceDrawing) -> Option<()> { let subtype = context.subtype; let stroke = context.stroke; @@ -592,15 +957,14 @@ fn draw_markup(context: &AppearanceContext<'_>, drawing: &mut AppearanceDrawing) c.push_str(&format!("{} {} m {} {} l {} {} l {} {} l h f\n", n(p1.0), n(p1.1), n(p2.0), n(p2.1), n(p4.0), n(p4.1), n(p3.0), n(p3.1))); } b"Underline" | b"StrikeOut" => { - let lw = (h * 0.07).clamp(0.5, 3.0); + let lw = markup_line_width(h); let t = if subtype == b"Underline" { 0.08 } else { 0.42 }; let (a, b) = (at(p3, p1, t), at(p4, p2, t)); c.push_str(&format!("{}{} w\n{} {} m {} {} l S\n", rg_stroke(col), n(lw), n(a.0), n(a.1), n(b.0), n(b.1))); } _ => { // Squiggly: a zigzag along the bottom of the quad. - let lw = (h * 0.05).clamp(0.5, 2.0); - let amp = (h * 0.06).max(0.75); + let (lw, amp) = squiggle(h); let len = (p4.0 - p3.0).hypot(p4.1 - p3.1); let step = (h / 4.0).max(1.5); let steps = ((len / step).ceil() as usize).clamp(1, 10_000); @@ -621,11 +985,31 @@ fn draw_markup(context: &AppearanceContext<'_>, drawing: &mut AppearanceDrawing) } fn draw_redaction(context: &AppearanceContext<'_>, drawing: &mut AppearanceDrawing) -> Option<()> { + let d = context.dict; let stroke = context.stroke; let c = &mut drawing.content; // While marked: the outline of each area (the fill comes when applied). - let q = quads(context)?; - let col = stroke.unwrap_or([0.89, 0.13, 0.13]); + // ISO 32000-2 gives a mark no outline colour (§12.5.6.23, Table 195). An array `/OC` of + // three numbers is the one some writers give it, and is drawn (the observation is in + // `parity/acrobat-features.toml`); `/OC` as ISO 32000-2 defines it (Table 166) is an + // optional-content dictionary, which this leaves alone. + let outline = nums(d, b"OC").and_then(|c| match c.as_slice() { + [r, g, b] if c.iter().all(|x| x.is_finite()) => Some([r.clamp(0.0, 1.0), g.clamp(0.0, 1.0), b.clamp(0.0, 1.0)]), + _ => None, + }); + // Quadrilaterals are outlined in the `/OC` colour, else in `/C` as they always were. + let (q, col) = match d.get(b"QuadPoints") { + Some(_) => (quads(context)?, outline.or(stroke)), + // Without `/QuadPoints`, `/Rect` is the area (Table 195): outlined just inside its + // edges when the mark names an outline colour, else not drawn while marked. + None => { + let outline = outline?; + let rect = context.rect; + let [x0, y0, x1, y1] = [rect[0] + 0.5, rect[1] + 0.5, rect[2] - 0.5, rect[3] - 0.5]; + (vec![[x0, y1, x1, y1, x0, y0, x1, y0]], Some(outline)) + } + }; + let col = col.unwrap_or([0.89, 0.13, 0.13]); c.push_str(&format!("{}1 w\n", rg_stroke(col))); for quad in q.iter() { c.push_str(&format!( @@ -654,15 +1038,27 @@ fn draw_box(context: &AppearanceContext<'_>, drawing: &mut AppearanceDrawing) -> if stroke.is_none() && fill.is_none() { return Some(()); } + // A plain square or circle lies within /Rect less /RD (Table 180), its stroke centred + // half the width inside that box. + let shape = rect_less_rd(d, rect).unwrap_or(rect); let inset = w / 2.0; - let [x0, y0, x1, y1] = [rect[0] + inset, rect[1] + inset, rect[2] - inset, rect[3] - inset]; + let [x0, y0, x1, y1] = [shape[0] + inset, shape[1] + inset, shape[2] - inset, shape[3] - inset]; if let Some(f) = fill { c.push_str(&rg(f)); } if let Some(s) = stroke { c.push_str(&format!("{}{} w\n{}", rg_stroke(s), n(w), dash(d))); } - if subtype == b"Square" { + if let Some(intensity) = context.cloud { + let bump = cloud_bump(w, intensity); + let inner = cloud_inner(d, rect, bump, w)?; + let pts = if subtype == b"Square" { + vec![(inner[0], inner[1]), (inner[2], inner[1]), (inner[2], inner[3]), (inner[0], inner[3])] + } else { + ellipse_points(inner, bump) + }; + c.push_str(&format!("1 j\n{}", cloud_path(&pts, bump))); + } else if subtype == b"Square" { c.push_str(&format!("{} {} {} {} re\n", n(x0), n(y0), n(x1 - x0), n(y1 - y0))); } else { c.push_str(&ellipse(x0, y0, x1, y1)); @@ -723,7 +1119,6 @@ fn draw_polygon(context: &AppearanceContext<'_>, drawing: &mut AppearanceDrawing let subtype = context.subtype; let stroke = context.stroke; let w = context.width; - let cloudy = context.cloudy; let c = &mut drawing.content; let v = nums(d, b"Vertices").filter(|v| v.len() >= 4 && v.len() % 2 == 0 && v.iter().all(|x| x.is_finite()))?; let pts: Vec<(f64, f64)> = v.as_chunks::<2>().0.iter().map(|p| (p[0], p[1])).collect(); @@ -751,9 +1146,8 @@ fn draw_polygon(context: &AppearanceContext<'_>, drawing: &mut AppearanceDrawing if let Some(s) = col { c.push_str(&format!("{}{} w 1 J 1 j\n{}", rg_stroke(s), n(w), dash(d))); } - if cloudy { - let intensity = d.get(b"BE").and_then(|b| b.as_dict()).and_then(|b| b.get(b"I")).and_then(|i| i.as_f64()).unwrap_or(1.0); - c.push_str(&cloud_path(&pts, cloud_radius(w) * intensity.clamp(0.5, 2.0))); + if let Some(intensity) = context.cloud { + c.push_str(&cloud_path(&pts, cloud_bump(w, intensity))); } else { for (i, p) in pts.iter().enumerate() { c.push_str(&format!("{} {} {}\n", n(p.0), n(p.1), if i == 0 { "m" } else { "l" })); @@ -870,7 +1264,8 @@ fn draw_stamp(context: &AppearanceContext<'_>, mut drawing: AppearanceDrawing) - return Some(form(rect, c.as_bytes(), res)); } // A custom stamp: its picture (an image, or a form mapped to /PCPictureSize) fills - // the rectangle. An image signature added to a turned page (`/PCPictureRotate`) gets + // the rectangle. An image signature or a stamp with a line added to a turned page + // (`/PCPictureRotate`) gets // the standard counter-rotation as its form `/Matrix`: viewers fit the turned // bounding box to `/Rect` (ISO 32000-2 §12.5.5), so it reads upright as displayed. if let Some(pic) = d.get(b"PCPicture").and_then(Object::as_ref) { @@ -879,14 +1274,61 @@ fn draw_stamp(context: &AppearanceContext<'_>, mut drawing: AppearanceDrawing) - [w, h] if *w > 0.0 && *h > 0.0 => Some((*w, *h)), _ => None, }); - let place = picture_place(rect, image, size)?; + // A text line (#1741) takes the bottom of the rectangle, and the picture keeps its + // shape, centred above it. Without one the picture fills the rectangle. + let line = d.get(b"PCByLine").and_then(Object::as_string).map(PdfString::to_text).filter(|l| !l.trim().is_empty()); + let turn = picture_turn(d); + // A lined stamp on a quarter-turned page is laid out in its displayed box (the + // rectangle with its sides swapped) and turned into `/Rect` by the `/Matrix` below. + // Laid out in user space, the viewer's fit of the turned box to `/Rect` would scale + // the two displayed axes differently and squash the picture and its line. + let bbox = if line.is_some() && matches!(turn, 90 | 270) { [0.0, 0.0, rect[3] - rect[1], rect[2] - rect[0]] } else { rect }; + let [x0, y0, x1, y1] = bbox; + let band = if line.is_some() { (y1 - y0) * crate::CUSTOM_LINE_BAND } else { 0.0 }; + let (w, h) = (x1 - x0, y1 - y0 - band); + let place = match (&line, size) { + (Some(_), Some((sw, sh))) => { + let k = (w / sw).min(h / sh).max(0.0); + let (pw, ph) = (sw * k, sh * k); + let (px, py) = (x0 + (w - pw) / 2.0, y0 + band + (h - ph) / 2.0); + if image { + format!("{} 0 0 {} {} {} cm", n(pw), n(ph), n(px), n(py)) + } else { + format!("{} 0 0 {} {} {} cm", n(k), n(k), n(px), n(py)) + } + } + // Without a line the band is empty and this is the rectangle itself. + _ => picture_place([x0, y0 + band, x1, y1], image, size)?, + }; c.push_str(&format!("q {place} /Pic Do Q\n")); + // Bytes from here: the line is WinAnsi, not UTF-8. + let mut content = std::mem::take(c).into_bytes(); + if let Some(l) = &line { + // Centred in the band, at its usual size unless the stamp was made narrower. + let mut size = band * crate::CUSTOM_LINE_TEXT; + let tw = text_width(l, size); + if tw > w - 4.0 && tw > 0.0 { + size *= ((w - 4.0) / tw).max(0.0); + } + let tw = text_width(l, size); + let ty = y0 + (band - size * 0.72) / 2.0; + content.extend(format!("BT\n0.2 0.2 0.2 rg\n/Helv {} Tf\n1 0 0 1 {} {} Tm ", n(size), n(x0 + (w - tw) / 2.0), n(ty)).bytes()); + content.extend(literal(&win_ansi(l))); + content.extend_from_slice(b" Tj\nET\n"); + let mut f = Dict::new(); + f.set(b"Type".to_vec(), Object::name("Font")); + f.set(b"Subtype".to_vec(), Object::name("Type1")); + f.set(b"BaseFont".to_vec(), Object::name("Helvetica")); + f.set(b"Encoding".to_vec(), Object::name("WinAnsiEncoding")); + let mut fonts = Dict::new(); + fonts.set(b"Helv".to_vec(), Object::Dict(f)); + res.set(b"Font".to_vec(), Object::Dict(fonts)); + } let mut xo = Dict::new(); xo.set(b"Pic".to_vec(), Object::Ref(pic)); res.set(b"XObject".to_vec(), Object::Dict(xo)); - let mut stream = form(rect, c.as_bytes(), res); - let turn = picture_turn(d); - if image && turn != 0 { + let mut stream = form(bbox, &content, res); + if (image || line.is_some()) && turn != 0 { // The page's view matrix without its offset: display axes in user space. let m = pdfcraft_model::view_matrix_for(turn, [0.0; 4]); let matrix = [m[0], m[1], m[2], m[3], 0.0, 0.0]; @@ -999,7 +1441,7 @@ fn shaped_border( Some(rect) } -fn draw_free_text(context: &AppearanceContext<'_>, mut drawing: AppearanceDrawing) -> Option { +fn draw_free_text(context: &AppearanceContext<'_>, mut drawing: AppearanceDrawing, dr_face: Option) -> Option { let d = context.dict; let parts = context.parts; let rect = context.rect; @@ -1007,11 +1449,12 @@ fn draw_free_text(context: &AppearanceContext<'_>, mut drawing: AppearanceDrawin let mut c = std::mem::take(&mut drawing.content); let mut res = std::mem::take(&mut drawing.resources); let _tail = std::mem::take(&mut drawing.tail); - let (text_color, size) = parse_da(d); + let crate::text_style::TextStyle { face, size, color: text_color } = crate::text_style::text_style(d, dr_face); // The box's border and a callout's leader line: their own colour, else the text's. let line_color = da_border(d).unwrap_or(text_color); let bg = stroke; - let bw = if d.contains(b"BS") || d.contains(b"Border") { border_width(d) } else { 0.0 }; + // Without `/BS` or `/Border`, the `/Border` default [0 0 1]: a 1-pt border (Table 166). + let bw = border_width(d); // A callout: the leader line (arrowhead at its first point), and the text box inside // `/Rect` by `/RD`. // Text added to a turned page is drawn upright in its displayed box, which the form @@ -1031,7 +1474,11 @@ fn draw_free_text(context: &AppearanceContext<'_>, mut drawing: AppearanceDrawin let mut rect = rect; if d.contains(b"CL") { let cl = nums(d, b"CL").filter(|l| l.len() == 4 || l.len() == 6)?; - let rd = nums(d, b"RD").filter(|r| r.len() == 4 && r.iter().all(|x| *x >= 0.0))?; + // `/RD` is optional (Table 177; null is absent, §7.3.9): without it the text box is `/Rect`. + let rd = match d.get(b"RD") { + None | Some(Object::Null) => vec![0.0; 4], + Some(_) => nums(d, b"RD").filter(|r| r.len() == 4 && r.iter().all(|x| *x >= 0.0))?, + }; rect = [full[0] + rd[0], full[1] + rd[1], full[2] - rd[2], full[3] - rd[3]]; if rect[2] - rect[0] < 1.0 || rect[3] - rect[1] < 1.0 { return None; @@ -1061,7 +1508,20 @@ fn draw_free_text(context: &AppearanceContext<'_>, mut drawing: AppearanceDrawin // `/RD` and keeps its place: background and border take the shape, the text is laid out // in the text box as in a rectangle. let shape = crate::BorderShape::of(d); - if shape != crate::BorderShape::Rectangle { + // A cloudy box (not a callout): its background and border follow the cloud around + // the text, which keeps to the cloud's inner rectangle. + if let Some(intensity) = context.cloud.filter(|_| !d.contains(b"CL")) { + let cw = if bw > 0.0 { bw } else { 1.0 }; + let bump = cloud_bump(cw, intensity); + let inner = cloud_inner(d, rect, bump, cw)?; + let pts = [(inner[0], inner[1]), (inner[2], inner[1]), (inner[2], inner[3]), (inner[0], inner[3])]; + c.push_str(&format!("{}{}{} w 1 j\n{}", rg_stroke(line_color), rg(bg.unwrap_or([1.0; 3])), n(cw), dash(d))); + c.push_str(&cloud_path(&pts, bump)); + c.push_str(if bg.is_some() { "B\n[] 0 d\n" } else { "S\n[] 0 d\n" }); + rect = inner; + } else if context.cloud.is_some() { + return None; + } else if shape != crate::BorderShape::Rectangle { rect = shaped_border(d, shape, full, turn, bw, bg, line_color, &mut c)?; } else { if let Some(bg) = bg { @@ -1092,37 +1552,42 @@ fn draw_free_text(context: &AppearanceContext<'_>, mut drawing: AppearanceDrawin c.push_str("/GS1 gs\n"); } c.push_str(&format!( - "{} {} {} {} re W n\nBT\n/Helv {} Tf\n{}", + "{} {} {} {} re W n\nBT\n/{} {} Tf\n{}", n(rect[0]), n(rect[1]), n(rect[2] - rect[0]), n(rect[3] - rect[1]), + crate::text_style::resource_name(face), n(size), rg(text_color) )); let mut out = c.into_bytes(); - let lines = wrap(&text, size, width); - let mut y = rect[3] - pad - size * 0.9; + let lines = pdfcraft_fonts::wrap_with(&text, size, width, |s, size| text_width_in(face, s, size)); + let mut y = rect[3] - pad - size * reach(&lines)[0]; if round { - y -= ((rect[3] - rect[1] - 2.0 * pad - lines.len() as f64 * size * 1.2) / 2.0).max(0.0); + y -= ((rect[3] - rect[1] - 2.0 * pad - text_lines_height(&lines, size)) / 2.0).max(0.0); } - for line in lines { + // Thai is shaped and drawn from the Thai face (Type 3 fonts in the form's resources, whose + // streams `apply_appearance` stores in the document); the rest, and Thai it can't draw (as + // `?`, which `undrawable_text` reports), in the box's standard face. + let mut fonts = font_resources(face); + let (pieces, shaped) = shaped_text::appearance_lines(&lines, &mut fonts); + let mut current = None; + for pieces in &pieces { if y < rect[1] - size { break; } - let lw = text_width(&line, size); + let lw = shaped_text::pieces_width(pieces, size, 0.0, |s, size| face.text_width(s, size)); let x = match q { 1 => rect[0] + pad + (width - lw) / 2.0, 2 => rect[2] - pad - lw, _ => rect[0] + pad, }; - out.extend(format!("1 0 0 1 {} {} Tm ", n(x), n(y)).bytes()); - out.extend(literal(&win_ansi(&line))); - out.extend_from_slice(b" Tj\n"); + shaped.show_line(&mut out, pieces, x, y, size, crate::text_style::resource_name(face), win_ansi, |s| face.text_width(s, size), &mut current); y -= size * 1.2; } out.extend_from_slice(b"ET\n"); - res.set(b"Font".to_vec(), helvetica_resource()); + res.set(b"Font".to_vec(), Object::Dict(fonts)); let mut stream = form(full, &out, res); if let Some(m) = matrix { stream.dict.set(b"Matrix".to_vec(), Object::Array(m.iter().copied().map(Object::Real).collect())); diff --git a/crates/annot/src/create.rs b/crates/annot/src/create.rs index 6376655ce9..e592b5d934 100644 --- a/crates/annot/src/create.rs +++ b/crates/annot/src/create.rs @@ -1,6 +1,6 @@ use super::draw::draw_appearance; use super::lookup::{annot_dict, annot_ref, annots, fill_sign_shape, page_ref, set_annots}; -use super::modify::{free_text_da, n, rich_text, set_part_keys, shape_text_box}; +use super::modify::{free_text_da, n, refuse_undrawable_thai, rich_text, set_part_keys, shape_text_box}; use super::types::{ FLAG_NO_ROTATE, FLAG_NO_ZOOM, FLAG_PRINT, LineEnding, Markup, Meta, NOTE_SIZE, NewAnnotation, OverlayLook, Rgb, Shape, StampGroup, Style, }; @@ -103,12 +103,8 @@ pub(crate) fn rect_for(shape: &Shape, style: &Style) -> Result<[f64; 4], AnnotEr } grow(ink_bounds(strokes).unwrap_or_default(), half + 1.0) } - Shape::Polygon { vertices, cloud } => { - let b = bounds(vertices.iter().copied()) - .filter(|b| vertices.len() >= 3 && vertices.iter().all(|p| finite(p)) && b[2] - b[0] >= 1.0 && b[3] - b[1] >= 1.0); - let b = b.ok_or_else(|| bad("polygon (it needs three points)"))?; - grow(b, half + 1.0 + if *cloud { 1.5 * appearance::cloud_radius(style.width) } else { 0.0 }) - } + // A new cloud has intensity 1 (`/BE /I 1`). + Shape::Polygon { vertices, cloud } => polygon_rect(vertices, style.width, cloud.then_some(1.0))?, Shape::PolyLine { vertices, start, end } => { let b = bounds(vertices.iter().copied()) .filter(|b| vertices.len() >= 2 && vertices.iter().all(|p| finite(p)) && (b[2] - b[0]).max(b[3] - b[1]) >= 1.0); @@ -135,6 +131,19 @@ pub(crate) fn rect_for(shape: &Shape, style: &Style) -> Result<[f64; 4], AnnotEr Ok(r) } +/// Validate a polygon's corners and compute its `/Rect`: room for half the line, and with a +/// cloud of `cloud` intensity, for the farthest its bumps reach (`appearance::cloud_reach`). +pub(crate) fn polygon_rect(vertices: &[[f64; 2]], width: f64, cloud: Option) -> Result<[f64; 4], AnnotError> { + let b = bounds(vertices.iter().copied()) + .filter(|b| vertices.len() >= 3 && vertices.iter().all(|p| finite(p)) && b[2] - b[0] >= 1.0 && b[3] - b[1] >= 1.0); + let b = b.ok_or_else(|| AnnotError::Invalid("invalid polygon (it needs three points)".into()))?; + let reach = cloud.map_or(0.0, |intensity| { + let perimeter: f64 = vertices.iter().zip(vertices.iter().cycle().skip(1)).map(|(a, b)| (b[0] - a[0]).hypot(b[1] - a[1])).sum(); + appearance::cloud_reach(perimeter, appearance::cloud_bump(width, intensity)) + }); + Ok(grow(b, width.max(0.0) / 2.0 + 1.0 + reach)) +} + pub(crate) fn base_dict(subtype: &str, rect: [f64; 4], page: ObjRef, contents: &str, author: &str, meta: &Meta) -> Dict { let mut d = Dict::new(); d.set(b"Type".to_vec(), Object::name("Annot")); @@ -195,14 +204,110 @@ fn subject(shape: &Shape) -> &'static str { pub fn callout_attach(rect: [f64; 4], knee: [f64; 2]) -> [f64; 2] { let [x0, y0, x1, y1] = normalize(rect); let (cx, cy) = ((x0 + x1) / 2.0, (y0 + y1) / 2.0); - if knee[0] < x0 { - [x0, cy] - } else if knee[0] > x1 { - [x1, cy] - } else if knee[1] > y1 { - [cx, y1] + match callout_side(rect, knee) { + Side::Left => [x0, cy], + Side::Right => [x1, cy], + Side::Top => [cx, y1], + Side::Bottom => [cx, y0], + } +} + +/// A side of a callout's text box. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Side { + Left, + Right, + Top, + Bottom, +} + +impl Side { + /// Whether a knee line on this side is horizontal (it meets the left or right side). Acrobat + /// lets a knee line be resized along its own direction only (Acrobat DC Help, 11/28/2019, + /// p. 357). + pub fn horizontal(self) -> bool { + matches!(self, Side::Left | Side::Right) + } +} + +/// The side of the text box `rect` that faces `knee`: the one [`callout_attach`] attaches to. +/// +/// Every position has one side (review on #4094): +/// - left or right of the box (`x < x0`, `x > x1`): that side, whatever `y` is, so a knee off a +/// corner diagonally, or on the line of the top or bottom edge past the box, is beside it; +/// - otherwise above or below it (`y > y1`, `y < y0`): that side, so a knee on the line of the +/// left or right edge past the box is above or below it; +/// - on the box's outline, or within `CALLOUT_SLACK` (0.01 pt) inside it, since the page hands +/// positions back as `f32` values: the side whose edge it lies on, the left or right edge +/// first, so a corner takes the left or right side as a knee beside the box does. The line +/// stays out of the box (it was sent across the box to the bottom side); +/// - inside the box, or with a coordinate that isn't a number: the bottom side. +pub fn callout_side(rect: [f64; 4], knee: [f64; 2]) -> Side { + let [x0, y0, x1, y1] = normalize(rect); + let [x, y] = knee; + // Before any comparison: one with NaN is false, so the other coordinate alone would decide. + if x.is_nan() || y.is_nan() { + return Side::Bottom; + } + if x < x0 { + Side::Left + } else if x > x1 { + Side::Right + } else if y > y1 { + Side::Top + } else if y < y0 { + Side::Bottom + } else if x - x0 < CALLOUT_SLACK { + Side::Left + } else if x1 - x < CALLOUT_SLACK { + Side::Right + } else if y1 - y < CALLOUT_SLACK { + Side::Top } else { - [cx, y0] + Side::Bottom + } +} + +/// A callout's knee line when Acrobat lays it out: 12 pt, out from the middle of the side. Seen in +/// Acrobat Pro DC, a 2021 Continuous release: a new callout's knee line, and one laid out again +/// after its knee was dragged into the box or the side it meets was moved, are all 12 pt, and the +/// knee can't come nearer the box than that. +pub const CALLOUT_KNEE_LINE: f64 = 12.0; + +/// Where a knee goes on `side` of the text box `rect`: [`CALLOUT_KNEE_LINE`] out from the middle +/// of that side, so the knee line is level (left, right) or upright (top, bottom). +pub fn callout_knee_on(rect: [f64; 4], side: Side) -> [f64; 2] { + let [x0, y0, x1, y1] = normalize(rect); + let (cx, cy) = ((x0 + x1) / 2.0, (y0 + y1) / 2.0); + match side { + Side::Left => [x0 - CALLOUT_KNEE_LINE, cy], + Side::Right => [x1 + CALLOUT_KNEE_LINE, cy], + Side::Top => [cx, y1 + CALLOUT_KNEE_LINE], + Side::Bottom => [cx, y0 - CALLOUT_KNEE_LINE], + } +} + +/// A new callout's knee: on the side of its text box `rect` that faces the arrow `point` +/// ([`callout_knee_on`]). +pub fn callout_knee(rect: [f64; 4], point: [f64; 2]) -> [f64; 2] { + callout_knee_on(rect, callout_side(rect, point)) +} + +/// How far apart, in points, two positions of a callout's box or line may be and still count as the +/// same. The page hands a box back as `f32` values, which moves a coordinate such as 323.05 by a few +/// hundred-thousandths of a point (review on #4094). +pub(crate) const CALLOUT_SLACK: f64 = 0.01; + +/// Whether `end` lies on `side` of the text box `rect`: the leader line can stay where it meets it. +pub(crate) fn on_side(rect: [f64; 4], side: Side, end: [f64; 2]) -> bool { + let [x0, y0, x1, y1] = normalize(rect); + let near = |a: f64, b: f64| (a - b).abs() < CALLOUT_SLACK; + let within = |v: f64, lo: f64, hi: f64| v > lo - CALLOUT_SLACK && v < hi + CALLOUT_SLACK; + match side { + Side::Left => near(end[0], x0) && within(end[1], y0, y1), + Side::Right => near(end[0], x1) && within(end[1], y0, y1), + Side::Top => near(end[1], y1) && within(end[0], x0, x1), + Side::Bottom => near(end[1], y0) && within(end[0], x0, x1), } } @@ -287,15 +392,20 @@ pub fn add_annotation(doc: &mut Document, new: &NewAnnotation, meta: &Meta) -> R let list = contours.iter().filter(|c| c.len() > 2).map(|c| num_array(&c.iter().flat_map(|p| [p[0], p[1]]).collect::>())).collect(); d.set(b"PCOutline".to_vec(), Object::Array(list)); } - Shape::CustomStamp { name, picture, image, size, .. } => { + Shape::CustomStamp { name, picture, image, size, line, .. } => { let clean: String = name.chars().filter(char::is_ascii_alphanumeric).collect(); d.set(b"Name".to_vec(), Object::name(&format!("PCCustom{clean}"))); d.set(b"PCPicture".to_vec(), Object::Ref(*picture)); d.set(b"PCPictureImage".to_vec(), Object::Bool(*image)); d.set(b"PCPictureSize".to_vec(), num_array(&[size.0, size.1])); + let lined = line.as_deref().map(str::trim).filter(|l| !l.is_empty()); + if let Some(l) = lined { + d.set(b"PCByLine".to_vec(), PdfString::text(l)); + } // An image signature's rectangle is in user space; on a turned page its appearance is - // turned back (see `appearance::build`), so it reads upright as displayed. - if *image && matches!(clean.as_str(), "Signature" | "Initials") { + // turned back (see `appearance::build`), so it reads upright as displayed. A stamp + // with a line is laid out in the displayed box, so it needs the turn too. + if lined.is_some() || (*image && matches!(clean.as_str(), "Signature" | "Initials")) { let turn = pdfcraft_model::pages(doc).get(new.page).map_or(0, |p| p.rotation(doc)); if turn != 0 { d.set(b"PCPictureRotate".to_vec(), Object::Int(turn)); @@ -398,6 +508,7 @@ pub fn add_annotation(doc: &mut Document, new: &NewAnnotation, meta: &Meta) -> R if matches!(turn, 90 | 180 | 270) { d.set(b"PCTextRotate".to_vec(), Object::Int(turn)); } + let rect = fit_thai_ink(rect, turn, &new.contents, size, 2.0 + appearance::border_width(&d)); // An oval or a circle goes around the text box, its text wrapped as displayed: // after `/PCTextRotate`, which swaps the box's sides on a quarter-turned page. shape_text_box(&mut d, rect, style.border_shape, style.width); @@ -417,6 +528,7 @@ pub fn add_annotation(doc: &mut Document, new: &NewAnnotation, meta: &Meta) -> R { return Err(AnnotError::Undrawable(pdfcraft_fonts::undrawable_message(what, c))); } + refuse_undrawable_thai(d.name(b"Subtype") == Some(b"FreeText"), &new.contents)?; let r = doc.add(Object::Dict(d.clone())); draw_appearance(doc, r)?; let mut list = annots(doc, page); @@ -427,6 +539,30 @@ pub fn add_annotation(doc: &mut Document, new: &NewAnnotation, meta: &Meta) -> R Ok(index) } +/// Text box `tb` (user space; its text upright on a page turned by `turn`), grown down as displayed +/// until the ink of `text`'s Thai (stacked marks above its first line, descenders below its last) +/// fits inside the padding `pad`, as the appearance lays it out at `size`. Other text, and a box +/// tall enough already, keep the box they were given. +fn fit_thai_ink(tb: [f64; 4], turn: i64, text: &str, size: f64, pad: f64) -> [f64; 4] { + if !text.chars().any(pdfcraft_fonts::shaped_text::is_thai) { + return tb; + } + let [x0, y0, x1, y1] = normalize(tb); + let (w, h) = if matches!(turn, 90 | 270) { (y1 - y0, x1 - x0) } else { (x1 - x0, y1 - y0) }; + let need = appearance::text_lines_height(&appearance::wrap(text, size, (w - 2.0 * pad).max(1.0)), size) + 2.0 * pad + 2.0; + let by = need - h; + if !(by.is_finite() && by > 0.0) { + return tb; + } + // Displayed down is user -y unturned, +x on a page turned 90°, +y at 180° and -x at 270°. + match turn { + 90 => [x0, y0, x1 + by, y1], + 180 => [x0, y0, x1, y1 + by], + 270 => [x0 - by, y0, x1, y1], + _ => [x0, y0 - by, x1, y1], + } +} + fn validate_style(style: &Style) -> Result<(), AnnotError> { if !finite(&style.color) || style.border_color.is_some_and(|c| !finite(&c)) || !style.opacity.is_finite() || !style.width.is_finite() { return Err(AnnotError::Invalid("invalid style".into())); @@ -465,6 +601,22 @@ fn add_popup(doc: &mut Document, parent: ObjRef, rect: Option<[f64; 4]>, annots: Ok(()) } +/// The share of a custom stamp's height its text line takes (the rest is the picture). +pub const CUSTOM_LINE_BAND: f64 = 0.22; + +/// A custom stamp line's text height as a share of its band. +pub const CUSTOM_LINE_TEXT: f64 = 0.62; + +/// The natural size of a custom stamp whose picture is `picture` points (#1741): with `line`, +/// the band under the picture is added, and the stamp is as wide as the line at its usual size +/// when that is wider than the picture. +pub fn custom_stamp_size(picture: (f64, f64), line: Option<&str>) -> (f64, f64) { + let Some(line) = line else { return picture }; + let h = picture.1 / (1.0 - CUSTOM_LINE_BAND); + let text = appearance::text_width(line, h * CUSTOM_LINE_BAND * CUSTOM_LINE_TEXT) + 8.0; + (picture.0.max(text), h) +} + /// Counterrotate a newly placed picture stamp's appearance into displayed-page axes. /// The picture may be an image or a PDF page. The normal appearance's transformed bounding /// box is fitted to `/Rect` by PDF viewers. diff --git a/crates/annot/src/draw.rs b/crates/annot/src/draw.rs index eac1f09fea..c0e82eb151 100644 --- a/crates/annot/src/draw.rs +++ b/crates/annot/src/draw.rs @@ -33,14 +33,70 @@ pub(crate) fn redraw_or_restore(doc: &mut Document, r: ObjRef, before: Dict) -> pub(crate) fn draw_appearance(doc: &mut Document, r: ObjRef) -> Result<(), AnnotError> { let d = annot_dict(doc, r); let subtype = String::from_utf8_lossy(d.name(b"Subtype").unwrap_or_default()).into_owned(); - let Some(stream) = build_appearance(doc, &d) else { return Err(AnnotError::Unsupported(subtype)) }; - apply_appearance(doc, r, &d, stream) + let page = appearance::is_drawn_from_points(&d).then(|| page_box_listing(doc, r)).flatten(); + let direct = with_direct_values(doc, &d); + let Some(stream) = build_appearance(doc, &d, &direct, page) else { return Err(AnnotError::Unsupported(subtype)) }; + let built = Built::new(stream, &direct); + apply_appearance(doc, r, &d, built) } -/// Draw the appearance for an annotation dictionary: the expensive half of [`set_appearance`]. -/// Pure from the dictionary and the objects it points at, so [`AppearanceCache`] can reuse it. -fn build_appearance(doc: &Document, d: &Dict) -> Option { - let mut stream = appearance::build(doc, d)?; +/// Whether [`set_appearance`] can draw the annotation `d`: read as the display reads it, with +/// references followed and the same fallback. Edits check this before changing anything. +pub(crate) fn can_draw(doc: &Document, d: &Dict) -> bool { + build_appearance(doc, d, &with_direct_values(doc, d), None).is_some() +} + +/// The visible region (`Page::crop`) of the page that lists `r` among its annotations: the page +/// its `/P` names when that page lists it, read up its own `/Parent` chain, else the first page +/// of the tree that lists it. An annotation is listed by one page only (ISO 32000-2 §12.5.2), so +/// both find the same page, and the first reads one page instead of walking the whole tree for +/// every comment redrawn. +fn page_box_listing(doc: &Document, r: ObjRef) -> Option<[f64; 4]> { + let named = annot_dict(doc, r).reference(b"P").filter(|p| annots(doc, *p).iter().any(|a| a.as_ref() == Some(r))); + if let Some(page) = named { + return Some(page_with_inherited_boxes(doc, page).crop(doc)); + } + pdfcraft_model::pages(doc).into_iter().find(|p| annots(doc, p.obj).iter().any(|a| a.as_ref() == Some(r))).map(|p| p.crop(doc)) +} + +/// The page `page` with the `/MediaBox` and `/CropBox` it inherits from its ancestors filled in +/// (ISO 32000-2 §7.7.3.4), enough for [`pdfcraft_model::Page::crop`]. Follows at most 64 +/// `/Parent` links and stops at a loop. +fn page_with_inherited_boxes(doc: &Document, page: ObjRef) -> pdfcraft_model::Page { + let mut dict = doc.get(page).as_dict().cloned().unwrap_or_default(); + let mut seen = std::collections::HashSet::from([page]); + let mut up = dict.reference(b"Parent"); + while let Some(node) = up { + if seen.len() > 64 || !seen.insert(node) { + break; + } + let obj = doc.get(node); + let Some(d) = obj.as_dict() else { break }; + for key in [&b"MediaBox"[..], b"CropBox"] { + if !dict.contains(key) + && let Some(v) = d.get(key) + { + dict.set(key.to_vec(), v.clone()); + } + } + up = d.reference(b"Parent"); + } + pdfcraft_model::Page { obj: page, dict } +} + +/// Draw the appearance for an annotation dictionary on a page whose visible region is `page` +/// (used by annotations drawn through points of their own, [`appearance::build_on_page`]): the +/// expensive half of [`set_appearance`]. `direct` is `d` with references followed +/// ([`with_direct_values`]). Pure from the dictionary, the objects it points at and the page, so +/// [`AppearanceCache`] can reuse it. +fn build_appearance(doc: &Document, d: &Dict, direct: &Dict, page: Option<[f64; 4]>) -> Option { + // References followed first. Where that turns up something the builder doesn't draw (a + // cloudy `/BE` on a square), the drawing made without them stays, rather than none. + let mut stream = match appearance::build_on_page(doc, direct, page) { + Some(stream) => stream, + None if !std::ptr::eq(direct, d) => appearance::build_on_page(doc, d, page)?, + None => return None, + }; // Picture-stamp restyling must retain the placement's page-axis correction. if d.get(b"PCPicture").is_some() { let matrix = d.get(b"AP").and_then(|ap| { @@ -56,10 +112,172 @@ fn build_appearance(doc: &Document, d: &Dict) -> Option { Some(stream) } +/// Entries that link the annotation to other objects rather than describe how it looks: left as +/// references by [`with_direct_values`]. +const LINK_KEYS: [&[u8]; 5] = [b"P", b"Popup", b"IRT", b"Parent", b"AP"]; + +/// How deep inside an entry [`with_direct_values`] follows references: `/InkList` is an array of +/// arrays, `/Border` may end in a dash array. +const INLINE_DEPTH: usize = 4; + +/// Values [`with_direct_values`] copies out of referenced arrays and dictionaries, or walks +/// before finding one too large to copy, for one annotation, so a file can't have a large object +/// copied or scanned for every entry that points at it. +pub(crate) const INLINE_BUDGET: usize = 1 << 16; + +/// Values copied or walked for all the annotations one display pass draws +/// ([`with_missing_appearances_cached`]), so a large object that many annotations point at is +/// not copied, or scanned and found too large, for each of them without bound. Past it, +/// references are left as they are. +pub(crate) const PASS_BUDGET: usize = 1 << 22; + +/// `d` with the references among the values [`appearance::build`] reads followed. Any value may be +/// given by reference (ISO 32000-2 §7.3.10): a border style dictionary (`/BS 5 0 R`, Table 168), a +/// colour, `/RD`, `/CL`, a `/W` inside `/BS`. The builder reads only direct values (apart from +/// `/QuadPoints`, which it resolves itself), so it draws from this copy. The entries in +/// [`LINK_KEYS`] and streams stay references; past [`INLINE_DEPTH`] levels or [`INLINE_BUDGET`] +/// copied values a reference is left as it is, which the builder treats as unreadable. `d` itself +/// when nothing is a reference. +pub(crate) fn with_direct_values<'d>(doc: &Document, d: &'d Dict) -> std::borrow::Cow<'d, Dict> { + with_direct_values_within(doc, d, &mut INLINE_BUDGET.clone()) +} + +/// [`with_direct_values`], copying at most [`INLINE_BUDGET`] values and no more than `pass` has +/// left, which is reduced by what was copied. +pub(crate) fn with_direct_values_within<'d>(doc: &Document, d: &'d Dict, pass: &mut usize) -> std::borrow::Cow<'d, Dict> { + let allowed = INLINE_BUDGET.min(*pass); + let mut budget = allowed; + let mut out: Option = None; + for (key, value) in d.iter() { + if LINK_KEYS.contains(&key.as_slice()) { + continue; + } + if let Some(direct) = direct_value(doc, value, INLINE_DEPTH, &mut budget) { + out.get_or_insert_with(|| d.clone()).set(key.clone(), direct); + } + } + *pass = pass.saturating_sub(allowed.saturating_sub(budget)); + out.map_or(std::borrow::Cow::Borrowed(d), std::borrow::Cow::Owned) +} + +/// The number of values in `o`, counting into its arrays and dictionaries (not through +/// references, which are charged when followed), or, past `limit`, `Err` with the values it +/// visited or queued before finding that out (at least one), so a rejected walk is charged too. +/// Walks at most `limit` values and holds no more than `limit` waiting. +fn weight(o: &Object, limit: usize) -> Result { + let mut count = 0usize; + let mut stack = vec![o]; + while let Some(o) = stack.pop() { + let spent = count.saturating_add(stack.len()).max(1); + count = match count.checked_add(1).filter(|c| *c <= limit) { + Some(c) => c, + None => return Err(spent), + }; + let inner = match o { + Object::Array(items) => items.len(), + Object::Dict(entries) => entries.len(), + _ => continue, + }; + // Every value waiting is counted later, so more than `limit` in all is over it already. + if count.saturating_add(stack.len()).saturating_add(inner) > limit { + return Err(spent.saturating_add(1)); + } + match o { + Object::Array(items) => stack.extend(items.iter()), + Object::Dict(entries) => stack.extend(entries.iter().map(|(_, v)| v)), + _ => {} + } + } + Ok(count) +} + +/// `o` with references followed `depth` levels into arrays and dictionaries, or `None` when there +/// is nothing to follow (or it may not be: a stream, an over-budget object, a chain of references). +fn direct_value(doc: &Document, o: &Object, depth: usize, budget: &mut usize) -> Option { + let held; + let (base, followed) = match o { + Object::Ref(_) => { + // A spent budget follows nothing more, not even to find out what is there. + if *budget == 0 { + return None; + } + held = doc.resolve(o); + if matches!(&*held, Object::Stream(_) | Object::Ref(_)) { + return None; + } + // Everything copied counts, inner arrays and dictionaries too, and so does the walk + // that finds an object too large: many references to one would otherwise each walk + // it again for free. + match weight(&held, *budget) { + Ok(n) => *budget = budget.checked_sub(n)?, + Err(spent) => { + *budget = budget.saturating_sub(spent); + return None; + } + } + (&*held, true) + } + other => (other, false), + }; + let Some(next) = depth.checked_sub(1) else { return followed.then(|| base.clone()) }; + match base { + Object::Array(items) => { + let mut changed: Option> = None; + for (i, item) in items.iter().enumerate() { + if let Some(direct) = direct_value(doc, item, next, budget) { + let copy = changed.get_or_insert_with(|| items.clone()); + if let Some(slot) = copy.get_mut(i) { + *slot = direct; + } + } + } + changed.map(Object::Array).or_else(|| followed.then(|| base.clone())) + } + Object::Dict(entries) => { + let mut changed: Option = None; + for (key, value) in entries.iter() { + if let Some(direct) = direct_value(doc, value, next, budget) { + changed.get_or_insert_with(|| entries.clone()).set(key.clone(), direct); + } + } + changed.map(Object::Dict).or_else(|| followed.then(|| base.clone())) + } + _ => followed.then(|| base.clone()), + } +} + +/// A built appearance and what storing it needs from the dictionary it was drawn from, read +/// with references followed, so a cached one is stored without following them again. +#[derive(Clone, Debug)] +struct Built { + stream: Stream, + /// The first character the standard fonts can't draw (it came out as `?`). + missing: Option, + /// A note's or attachment's `/Rect`, as written. + rect: Option>, + /// Points (/QuadPoints, /L, /Vertices, /InkList) past /Rect: its appearance covers them, so + /// /Rect does too. + markup_rect: Option<[f64; 4]>, +} + +impl Built { + fn new(stream: Stream, direct: &Dict) -> Self { + let missing = appearance::undrawable_text(direct).map(|(_, c)| c); + let rect = matches!(direct.name(b"Subtype"), Some(b"Text" | b"FileAttachment")) + .then(|| direct.get(b"Rect")?.as_array()?.iter().map(|o| o.as_f64()).collect::>>()) + .flatten(); + let markup_rect = appearance::widened_rect(direct, &stream); + Built { stream, missing, rect, markup_rect } + } +} + /// Store a built appearance on the annotation: the cheap half of [`set_appearance`]. -fn apply_appearance(doc: &mut Document, r: ObjRef, d: &Dict, stream: Stream) -> Result<(), AnnotError> { - // Characters the standard fonts can't draw came out as `?`: mark the annotation with the first. - let missing = appearance::undrawable_text(d).map(|(_, c)| c); +fn apply_appearance(doc: &mut Document, r: ObjRef, d: &Dict, built: Built) -> Result<(), AnnotError> { + // `missing`: characters the standard fonts can't draw came out as `?`; the annotation is + // marked with the first. + let Built { mut stream, missing, rect, markup_rect } = built; + // A text box's Thai glyphs are streams of their own. + pdfcraft_fonts::shaped_text::store_streams(doc, &mut stream.dict); let ap = doc.add(Object::Stream(stream)); // A copy (a shared /AP is left alone) that keeps unknown entries; the old down and rollover // appearances would show the previous look on press or hover, so they go with the old /N. @@ -70,17 +288,13 @@ fn apply_appearance(doc: &mut Document, r: ObjRef, d: &Dict, stream: Stream) -> // A note icon has a fixed size (`NOTE_SIZE`); viewers draw it at the upper-left corner of `/Rect`. // The appearance's box is mapped onto `/Rect`, so any other `/Rect` (a large one, or one with // no width, as LaTeX `pdfcomment` writes) would stretch the icon or shrink it to nothing. - let icon_rect = matches!(d.name(b"Subtype"), Some(b"Text" | b"FileAttachment")) - .then(|| d.get(b"Rect")?.as_array()?.iter().map(|o| o.as_f64()).collect::>>()) - .flatten() - .filter(|q| q.len() == 4 && finite(q)) - .map(|q| { - let (x, top) = (q[0].min(q[2]), q[1].max(q[3])); - [x, top - NOTE_SIZE, x + NOTE_SIZE, top] - }); + let icon_rect = rect.filter(|q| q.len() == 4 && finite(q)).map(|q| { + let (x, top) = (q[0].min(q[2]), q[1].max(q[3])); + [x, top - NOTE_SIZE, x + NOTE_SIZE, top] + }); doc.update_dict(r, |d| { d.set(b"AP".to_vec(), Object::Dict(apd)); - if let Some(rect) = icon_rect { + if let Some(rect) = icon_rect.or(markup_rect) { d.set(b"Rect".to_vec(), num_array(&rect)); } d.remove(b"AS"); @@ -101,7 +315,9 @@ fn apply_appearance(doc: &mut Document, r: ObjRef, d: &Dict, stream: Stream) -> /// refreshes) and keep it for content-only ones. #[derive(Clone, Debug, Default)] pub struct AppearanceCache { - entries: std::collections::HashMap, + /// Each appearance with the page region it was drawn for (see [`appearance::build_on_page`]): + /// a changed crop box draws it again. + entries: std::collections::HashMap, Built)>, bytes: usize, hits: u64, builds: u64, @@ -116,20 +332,21 @@ impl AppearanceCache { *self = Self::default(); } - fn get(&mut self, r: ObjRef) -> Option { - let stream = self.entries.get(&r)?.clone(); + fn get(&mut self, r: ObjRef, crop: Option<[f64; 4]>) -> Option { + let built = self.entries.get(&r).filter(|(drawn_for, _)| *drawn_for == crop)?.1.clone(); self.hits += 1; - Some(stream) + Some(built) } - fn insert(&mut self, r: ObjRef, stream: &Stream) { + fn insert(&mut self, r: ObjRef, crop: Option<[f64; 4]>, built: &Built) { + let stream = &built.stream; self.builds += 1; if self.entries.len() >= 1_000_000 { self.clear(); } if self.bytes.saturating_add(stream.raw.len()) <= Self::MAX_BYTES { self.bytes += stream.raw.len(); - self.entries.insert(r, stream.clone()); + self.entries.insert(r, (crop, built.clone())); } // Else over budget: keep serving the entries already cached instead of growing. } @@ -163,6 +380,9 @@ pub fn with_missing_appearances_cached(doc: &Document, cache: &mut AppearanceCac // reads `doc`, which never gets the appearance, so every repeat used to add another stream to // the copy, and the display bytes grew with the number of listings. let mut seen = std::collections::HashSet::new(); + // Pages' visible regions, read once and only when an annotation drawn through its points is. + let mut crops: Option> = None; + let mut budget = PASS_BUDGET; for page in page_refs(doc).ok()? { for r in annots(doc, page).iter().filter_map(Object::as_ref) { if !seen.insert(r) { @@ -176,9 +396,12 @@ pub fn with_missing_appearances_cached(doc: &Document, cache: &mut AppearanceCac if has_normal { continue; } - if let Some(stream) = cache.get(r) { + let crop = appearance::is_drawn_from_points(&d) + .then(|| crops.get_or_insert_with(|| pdfcraft_model::pages(doc).iter().map(|p| (p.obj, p.crop(doc))).collect()).get(&page).copied()) + .flatten(); + if let Some(built) = cache.get(r, crop) { // Fails only when `r` isn't a dictionary, which the cached build ruled out. - apply_appearance(copy.get_or_insert_with(|| doc.clone()), r, &d, stream).ok(); + apply_appearance(copy.get_or_insert_with(|| doc.clone()), r, &d, built).ok(); continue; } // Another program's stamp gets a drawing for display only (ISO 32000-2 Table 184). Its @@ -188,10 +411,12 @@ pub fn with_missing_appearances_cached(doc: &Document, cache: &mut AppearanceCac bare.remove(b"AP"); appearance::display_stamp(&bare) }; - let Some(stream) = build_appearance(doc, &d).or_else(other_stamp) else { continue }; - cache.insert(r, &stream); + let direct = with_direct_values_within(doc, &d, &mut budget); + let Some(stream) = build_appearance(doc, &d, &direct, crop).or_else(other_stamp) else { continue }; + let built = Built::new(stream, &direct); + cache.insert(r, crop, &built); // Fails only when `r` isn't a dictionary, which the build just ruled out. - apply_appearance(copy.get_or_insert_with(|| doc.clone()), r, &d, stream).ok(); + apply_appearance(copy.get_or_insert_with(|| doc.clone()), r, &d, built).ok(); } } copy diff --git a/crates/annot/src/info.rs b/crates/annot/src/info.rs index 8af76cb2ed..99c3264006 100644 --- a/crates/annot/src/info.rs +++ b/crates/annot/src/info.rs @@ -1,7 +1,8 @@ use super::create::finite; use super::draw::set_appearance; use super::lookup::{annot_dict, annot_ref, annotation_flags, annots, drawn_upright, is_fill_sign, page_ref, page_refs}; -use super::modify::{fillable, touch, unlocked}; +use super::modify::{callout_line, callout_text_box, fillable, touch, unlocked}; +use super::text_style; use super::types::{BorderShape, FLAG_LOCKED, LineEnding, Meta, NoteIcon, Rgb}; use super::{AnnotError, Dict, Document, ObjRef, Object, PdfString, appearance}; @@ -23,7 +24,8 @@ pub struct Summary { pub in_reply_to: Option, /// The page and `/Annots` position `/IRT` points at, by reference (`/NM` need not be unique). pub parent: Option<(usize, usize)>, - /// `/RT /Group`: grouped with the `/IRT` annotation (Replace Text's caret), not a reply to it. + /// `/RT /Group`: with an `/IRT` reference, a subordinate of the group whose primary that is + /// ([`is_group_subordinate`]), not a reply to it. pub grouped: bool, pub rect: [f32; 4], pub color: Option<[f32; 3]>, @@ -35,9 +37,66 @@ pub struct Summary { pub fill_sign: bool, /// The NoRotate flag (`/F` bit 5), as `pdfcraft_render::Annotation::no_rotate`. pub no_rotate: bool, - /// A line's two ends (`/L`), or the corners of connected lines, a polygon or a cloud - /// (`/Vertices`), in user space; empty for other comments. + /// A line's two ends (`/L`), the corners of connected lines, a polygon or a cloud + /// (`/Vertices`), or a callout's arrow point and knee (`/CL` without its end, which follows + /// the text box), in user space; empty for other comments. pub points: Vec<[f32; 2]>, + /// A callout's text box (`/Rect` inset by `/RD`), in user space; `None` for other comments. + pub text_box: Option<[f32; 4]>, + /// The comment's pop-up window (`/Popup`), when it has one. + pub popup: Option, +} + +impl Summary { + /// A subordinate of a group (`/RT /Group` with an `/IRT` to a listed comment, ISO 32000-2 + /// §12.5.6.2): part of its primary's comment, not a reply. As + /// `pdfcraft_render::Annotation::is_group_member`, which also drops a looping `/IRT`. + pub fn is_group_member(&self) -> bool { + self.grouped && self.parent.is_some() + } +} + +/// A comment's pop-up window (ISO 32000-2 §12.5.6.14): where it sits on the page and whether it +/// is shown open. +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct PopupInfo { + /// `/Rect` of the `/Popup` annotation, in user space. + pub rect: [f32; 4], + /// `/Open` of the `/Popup` annotation. + pub open: bool, +} + +/// The pop-up the comment `annot` (the object `own`, when it is one of its own) names in +/// `/Popup`, when it is the comment's: a pop-up annotation whose `/Parent`, if it has one, is a +/// reference to the comment. A `/Popup` naming another annotation, or a pop-up of another +/// comment, isn't. ISO 32000-2:2020 §12.5.6.14, Table 186: `/Parent` is "(Optional; shall be +/// an indirect reference)". +pub(crate) fn own_popup(doc: &Document, own: Option, annot: &Dict) -> Option { + let pr = annot.reference(b"Popup")?; + let is_own = doc.get(pr).as_dict().is_some_and(|pd| { + pd.name(b"Subtype") == Some(b"Popup") + && pd + .get(b"Parent") + .is_none_or(|parent| parent.as_ref().is_some_and(|parent| own.is_some_and(|own| final_ref(doc, parent) == final_ref(doc, own)))) + }); + is_own.then_some(pr) +} + +/// The comment's own pop-up (see [`own_popup`]), as [`PopupInfo`]: `None` when it has none or it +/// is unreadable. +pub(crate) fn popup_info(doc: &Document, own: Option, annot: &Dict) -> Option { + // Another annotation's rectangle and open state aren't this comment's (review on #2137). + let r = own_popup(doc, own, annot)?; + let p = doc.get(r); + let p = p.as_dict()?; + let rect: Vec = doc.resolve(p.get(b"Rect")?).as_array()?.iter().filter_map(|v| doc.resolve(v).as_f64().map(|x| x as f32)).collect(); + // Four numbers that are finite as `f32` (a huge one becomes infinite), corners in any order. + let [x0, y0, x1, y1] = rect[..] else { return None }; + if !rect.iter().all(|v| v.is_finite()) { + return None; + } + let open = p.get(b"Open").map(|o| doc.resolve(o)).is_some_and(|o| matches!(&*o, Object::Bool(true))); + Some(PopupInfo { rect: [x0.min(x1), y0.min(y1), x0.max(x1), y0.max(y1)], open }) } pub(crate) fn text_value(doc: &Document, d: &Dict, key: &[u8]) -> Option { @@ -74,6 +133,15 @@ fn final_ref(doc: &Document, mut r: ObjRef) -> ObjRef { r } +/// Whether an annotation is a subordinate of a group rather than a reply: `/RT /Group` with an +/// `/IRT` reference (ISO 32000-2 §12.5.6.2, Table 172). The group's primary is the `/IRT` +/// annotation, and the subordinate's own text, author, colour, dates and pop-up are ignored. +/// `/RT` is `R` by default, and a value other than `Group` reads as `R` too. Keep in step with +/// `pdfcraft_render::inspect`. +pub fn is_group_subordinate(d: &Dict) -> bool { + d.name(b"RT") == Some(b"Group") && d.reference(b"IRT").is_some() +} + /// Every comment (see [`is_comment_subtype`]), ordered by page and then top edge. pub fn summaries(doc: &Document) -> Vec { let mut out = Vec::new(); @@ -105,7 +173,10 @@ pub fn summaries(doc: &Document) -> Vec { let r = nums(b"Rect"); let rect = if r.len() == 4 { [r[0].min(r[2]), r[1].min(r[3]), r[0].max(r[2]), r[1].max(r[3])] } else { [0.0; 4] }; let c = nums(b"C"); - let mut color = (c.len() == 3).then(|| [c[0], c[1], c[2]]); + let rgb = (c.len() == 3).then(|| [c[0], c[1], c[2]]); + // Other comments report the stroke colour as it is drawn (grey, CMYK, a null entry + // read as 0), like `props`; three numbers, some given by reference, are still read. + let mut color = if subtype == "FreeText" { rgb } else { summary_color(doc, d).or(rgb) }; if subtype == "FreeText" && let Some(da) = text_value(doc, d, b"DA") { @@ -127,15 +198,29 @@ pub fn summaries(doc: &Document) -> Vec { pdfcraft_geom::quad_points::usable_quads(entries).into_iter().map(|q| q.map(|v| v as f32)).collect() }) .unwrap_or_default(); - // Keep in step with `pdfcraft_render::inspect`. + // Keep in step with `pdfcraft_render::inspect`: a callout gives its points only with a + // readable text box, without which they can't be changed either. + let callout = (subtype == "FreeText").then(|| callout_line(doc, d).zip(callout_text_box(doc, d))).flatten(); let points = match subtype.as_str() { "Line" => Some(nums(b"L")).filter(|l| l.len() == 4).map(|l| l.as_chunks::<2>().0.to_vec()).unwrap_or_default(), "PolyLine" | "Polygon" => nums(b"Vertices").as_chunks::<2>().0.to_vec(), + "FreeText" => callout + .as_ref() + .and_then(|(line, _)| line.split_last()) + .map(|(_, lead)| lead.iter().map(|p| [p[0] as f32, p[1] as f32]).collect()) + .unwrap_or_default(), _ => Vec::new(), }; + let text_box = callout.map(|(_, t)| t.map(|v| v as f32)); let irt = d.get(b"IRT").map(|o| o.as_ref().map_or_else(|| doc.resolve(o), |r| doc.get(final_ref(doc, r)))); let in_reply_to = irt.and_then(|o| o.as_dict().cloned()).and_then(|p| text_value(doc, &p, b"NM")); let parent = d.get(b"IRT").and_then(Object::as_ref).and_then(|r| slots.get(&final_ref(doc, r)).copied()); + // A group is one page's, as Acrobat reads it: a subordinate whose primary is on + // another page is a comment of its own (keep in step with + // `pdfcraft_render::link_replies`). + let grouped = d.name(b"RT") == Some(b"Group"); + let cross_page = grouped && parent.is_some_and(|(p, _)| p != page); + let (parent, in_reply_to) = if cross_page { (None, None) } else { (parent, in_reply_to) }; out.push(Summary { page, index, @@ -148,16 +233,24 @@ pub fn summaries(doc: &Document) -> Vec { name: text_value(doc, d, b"NM"), in_reply_to, parent, - grouped: d.name(b"RT") == Some(b"Group"), + grouped, rect, color, - state: text_value(doc, d, b"State"), + // A reply that gives only `/StateModel` has the model's default state (ISO 32000-2 + // Table 175), as `last_state_reply` and the inspector read it. + state: text_value(doc, d, b"State").or_else(|| match text_value(doc, d, b"StateModel").as_deref() { + Some("Marked") => Some("Unmarked".into()), + Some("Review") => Some("None".into()), + _ => None, + }), quads, locked: annotation_flags(doc, d) & FLAG_LOCKED != 0, intent: d.get(b"IT").and_then(|o| doc.resolve(o).as_name().map(|n| String::from_utf8_lossy(n).into_owned())), fill_sign: is_fill_sign(doc, d), no_rotate: drawn_upright(doc, d), points, + text_box, + popup: popup_info(doc, entry.as_ref().map(|r| final_ref(doc, r)), d), }); } } @@ -242,13 +335,20 @@ pub fn props(doc: &Document, page: usize, index: usize) -> Option { let p = page_ref(doc, page).ok()?; let entry = annots(doc, p).get(index).cloned()?; let obj = doc.resolve(&entry); - let d = obj.as_dict()?; + // Values given by reference (`/C 7 0 R`, `/BS 5 0 R`, ISO 32000-2 §7.3.10) read as the display draws them. + let raw = obj.as_dict()?; + let direct = crate::draw::with_direct_values(doc, raw); + let d = direct.as_ref(); let subtype = String::from_utf8_lossy(d.name(b"Subtype")?).into_owned(); - let c: Vec = d.get(b"C").and_then(|o| o.as_array().map(|a| a.iter().filter_map(|x| x.as_f64()).collect())).unwrap_or_default(); - let color = if subtype == "FreeText" { Some(appearance::parse_da(d).0) } else { (c.len() == 3).then(|| [c[0], c[1], c[2]]) }; + // A text box reports the colour and size it is drawn in (`/DA`, or `/DS` where it sets one). + let style = (subtype == "FreeText").then(|| text_style::text_style(d, None)); + let color = match &style { + Some(s) => Some(s.color), + None => drawn_color(d, b"C"), + }; let width = matches!(subtype.as_str(), "Square" | "Circle" | "Line" | "Ink" | "Polygon" | "PolyLine" | "FreeText").then(|| { if subtype == "FreeText" { - if !d.contains(b"BS") && !d.contains(b"Border") { 0.0 } else { appearance::border_width(d) } + appearance::border_width(d) } else { d.get(b"BS").and_then(|b| b.as_dict()).and_then(|b| b.get(b"W")).and_then(|w| w.as_f64()).unwrap_or(1.0) } @@ -261,11 +361,11 @@ pub fn props(doc: &Document, page: usize, index: usize) -> Option { width, icon: (subtype == "Text").then(|| d.name(b"Name").and_then(|n| NoteIcon::from_name(&String::from_utf8_lossy(n))).unwrap_or(NoteIcon::Note)), modified: text_value(doc, d, b"M"), - restylable: appearance::build(doc, d).is_some(), + restylable: crate::draw::can_draw(doc, raw), locked: annotation_flags(doc, d) & FLAG_LOCKED != 0, fillable: fillable(&subtype), fill: fill_of(d), - font_size: (subtype == "FreeText").then(|| appearance::parse_da(d).1), + font_size: style.map(|s| s.size), border_color: (subtype == "FreeText").then(|| appearance::da_border(d)).flatten(), border_shape: (subtype == "FreeText" && !d.contains(b"CL")).then(|| BorderShape::of(d)), fill_opacity: fillable(&subtype).then(|| appearance::part_opacity(d).fill), @@ -275,16 +375,32 @@ pub fn props(doc: &Document, page: usize, index: usize) -> Option { }) } -/// A shape’s `/IC` or FreeText `/C` as RGB. Grey and CMYK fills are converted. +/// A shape’s `/IC` or FreeText `/C` as RGB, read as it is drawn. Grey and CMYK fills are converted. fn fill_of(d: &Dict) -> Option { let key: &[u8] = if d.name(b"Subtype") == Some(b"FreeText") { b"C" } else { b"IC" }; - let c: Vec = d.get(key)?.as_array()?.iter().filter_map(|x| x.as_f64()).collect(); - let rgb = match c.as_slice() { - [g] => [*g; 3], - [r, g, b] => [*r, *g, *b], - [c, m, y, k] => [(1.0 - c) * (1.0 - k), (1.0 - m) * (1.0 - k), (1.0 - y) * (1.0 - k)], - _ => return None, - }; + drawn_color(d, key) +} + +/// `/C` as [`drawn_color`] reads it, the array itself followed when it is given by reference. +/// Only an array of at most four entries is copied out of the reference. +fn summary_color(doc: &Document, d: &Dict) -> Option<[f32; 3]> { + let rgb = match d.get(b"C") { + Some(o @ Object::Ref(_)) => { + let held = doc.resolve(o); + let entries = held.as_array().filter(|a| a.len() <= 4)?; + let mut direct = Dict::new(); + direct.set(b"C".to_vec(), Object::Array(entries.clone())); + drawn_color(&direct, b"C") + } + _ => drawn_color(d, b"C"), + }?; + Some(rgb.map(|x| x as f32)) +} + +/// The colour array at `key` as RGB, read as the appearance builder draws it: grey and CMYK are +/// converted, and a four-entry array with a non-number in it is CMYK with that entry 0. +fn drawn_color(d: &Dict, key: &[u8]) -> Option { + let rgb = appearance::color(d, key).flatten()?; finite(&rgb).then(|| rgb.map(|x| x.clamp(0.0, 1.0))) } diff --git a/crates/annot/src/lib.rs b/crates/annot/src/lib.rs index 52c0aad74f..63ff5a5be0 100644 --- a/crates/annot/src/lib.rs +++ b/crates/annot/src/lib.rs @@ -29,9 +29,12 @@ mod types; #[cfg(test)] use create::num_array; -pub use create::{add_annotation, callout_attach, orient_image_stamp}; +pub use create::{ + CALLOUT_KNEE_LINE, CUSTOM_LINE_BAND, CUSTOM_LINE_TEXT, Side, add_annotation, callout_attach, callout_knee, callout_knee_on, callout_side, + custom_stamp_size, orient_image_stamp, +}; pub use draw::{AppearanceCache, set_appearance, with_missing_appearances, with_missing_appearances_cached}; -pub use info::{Props, Summary, is_comment_subtype, props, set_info, summaries}; +pub use info::{PopupInfo, Props, Summary, is_comment_subtype, is_group_subordinate, props, set_info, summaries}; #[cfg(test)] use lookup::{ANNOTS_ENTRIES_COPIED, PageWalk, annot_dict, annot_ref}; use lookup::{annots, page_ref}; @@ -41,9 +44,9 @@ pub use lookup::{ pub use model3d::{Model3d, Model3dKind, model_3d, model_3d_annotations}; pub(crate) use modify::n; pub use modify::{ - add_reply, add_text_replacement, deletable, delete_annotation, erase_ink, fillable, goes_with, lift_annotation, move_annotation, - set_border_color, set_border_shape, set_contents, set_fill, set_font_size, set_locked, set_marked, set_part_opacity, set_points, set_rect, - set_review_state, set_style, with_replies_and_popups, + add_reply, add_text_replacement, deletable, delete_annotation, erase_ink, fillable, goes_with, lift_annotation, move_annotation, release_members, + set_border_color, set_border_shape, set_contents, set_fill, set_font_size, set_locked, set_marked, set_part_opacity, set_points, set_popup, + set_rect, set_review_state, set_style, with_replies_and_popups, }; pub use threed::{Activation as Activation3d, MAX_MODEL_BYTES, ModelFormat, NewModel3d, PosterPicture, add_3d_model, check_model}; pub use types::{ @@ -53,6 +56,7 @@ pub use types::{ pub mod appearance; pub mod links; +pub mod text_style; #[cfg(test)] mod tests; diff --git a/crates/annot/src/links.rs b/crates/annot/src/links.rs index 7cf4328c7c..88ac097b74 100644 --- a/crates/annot/src/links.rs +++ b/crates/annot/src/links.rs @@ -8,13 +8,77 @@ use crate::{AnnotError, annots, page_ref, page_refs, set_annots}; /// Where a link goes. #[derive(Clone, Debug, PartialEq)] pub enum LinkAction { - /// A page of this document (0-based), shown fitting the window (`/Fit`). - Page(usize), + /// A page of this document (0-based), shown at `zoom`. + Page { + page: usize, + zoom: LinkZoom, + }, Uri(String), /// Something PdfCraft doesn't edit yet (named destination, JavaScript…): kept as is. Other(String), } +/// How a go-to-page link shows its page: Acrobat's Go to a page view zoom, written as an +/// explicit destination (ISO 32000-2 §12.3.2.2, Table 149). Positions are written `null`, so the +/// reader keeps its position in that direction, as a new bookmark does. +#[derive(Clone, Copy, Debug, PartialEq)] +pub enum LinkZoom { + /// `/Fit`: the whole page in the window. + FitPage, + /// `/FitH null`: the page's width across the window. + FitWidth, + /// `/FitV null`: the page's height down the window. + FitHeight, + /// `/FitB`: the page's content (its bounding box) in the window. + FitVisible, + /// `/XYZ null null null`: the reader's zoom is kept. + Inherit, + /// `/XYZ null null z` with z = percent / 100; 100 is Actual Size. A new zoom is written + /// within 1–6400%; a link rewritten with the zoom it has keeps its own factor. + Percent(f64), +} + +impl LinkZoom { + /// The zoom an explicit destination array (`[page /XYZ …]`, `[page /Fit]`, …) asks for. The + /// boxed variants read as their unboxed ones, and a `/FitR` rectangle or a missing or unknown + /// type as Fit Page, the view links had before they carried a zoom. A link whose page and zoom + /// are unchanged is not rewritten, so its position is kept; one given a new page keeps how its + /// zoom is written (`write_action`) but not its position. + fn of_dest(doc: &Document, dest: &[Object]) -> Self { + let kind = dest.get(1).map(|k| doc.resolve(k)).and_then(|k| k.as_name().map(<[u8]>::to_vec)); + match kind.as_deref() { + Some(b"FitB") => LinkZoom::FitVisible, + Some(b"FitH" | b"FitBH") => LinkZoom::FitWidth, + Some(b"FitV" | b"FitBV") => LinkZoom::FitHeight, + Some(b"XYZ") => match dest.get(4).and_then(|z| doc.resolve(z).as_f64()) { + // Finite even for a factor near f64::MAX. A link given a new page with this zoom + // keeps the factor itself, however far outside 1–6400% (`write_action`). + Some(z) if z.is_finite() && z > 0.0 => LinkZoom::Percent((z * 100.0).min(f64::MAX)), + // A zoom of 0 or null keeps the reader's (Table 149). + _ => LinkZoom::Inherit, + }, + _ => LinkZoom::FitPage, + } + } + + /// The destination's elements after the page reference. + fn operands(self) -> Result, AnnotError> { + Ok(match self { + LinkZoom::FitPage => vec![Object::name("Fit")], + LinkZoom::FitWidth => vec![Object::name("FitH"), Object::Null], + LinkZoom::FitHeight => vec![Object::name("FitV"), Object::Null], + LinkZoom::FitVisible => vec![Object::name("FitB")], + LinkZoom::Inherit => vec![Object::name("XYZ"), Object::Null, Object::Null, Object::Null], + LinkZoom::Percent(p) => { + if !(p.is_finite() && p > 0.0) { + return Err(AnnotError::Invalid("the link's zoom must be a positive percentage".into())); + } + vec![Object::name("XYZ"), Object::Null, Object::Null, Object::Real((p / 100.0).clamp(0.01, 64.0))] + } + }) + } +} + /// Acrobat's Link Properties ▸ Appearance. #[derive(Clone, Copy, Debug, PartialEq)] pub struct LinkStyle { @@ -87,17 +151,18 @@ fn nums(doc: &Document, o: Option<&Object>) -> Vec { } fn action_of(doc: &Document, d: &Dict, pages: &[ObjRef], uri_budget: &mut usize) -> LinkAction { - let dest_page = |dest: &Object| -> Option { + let page_view = |dest: &Object| -> Option { let a = doc.resolve(dest); - let first = a.as_array()?.first()?.clone(); - match first { - Object::Ref(r) => pages.iter().position(|p| *p == r), - Object::Int(i) => usize::try_from(i).ok(), + let a = a.as_array()?; + let page = match a.first()? { + Object::Ref(r) => pages.iter().position(|p| p == r), + Object::Int(i) => usize::try_from(*i).ok(), _ => None, - } + }?; + Some(LinkAction::Page { page, zoom: LinkZoom::of_dest(doc, a) }) }; if let Some(dest) = d.get(b"Dest") { - return dest_page(dest).map_or_else(|| LinkAction::Other("named destination".into()), LinkAction::Page); + return page_view(dest).unwrap_or_else(|| LinkAction::Other("named destination".into())); } let Some(a) = d.get(b"A").map(|a| doc.resolve(a)).and_then(|a| a.as_dict().cloned()) else { return LinkAction::Other("no action".into()) }; match a.name(b"S") { @@ -115,7 +180,7 @@ fn action_of(doc: &Document, d: &Dict, pages: &[ObjRef], uri_budget: &mut usize) None => LinkAction::Other("web address too long".into()), } } - Some(b"GoTo") => a.get(b"D").and_then(dest_page).map_or_else(|| LinkAction::Other("named destination".into()), LinkAction::Page), + Some(b"GoTo") => a.get(b"D").and_then(page_view).unwrap_or_else(|| LinkAction::Other("named destination".into())), Some(s) => LinkAction::Other(String::from_utf8_lossy(s).into_owned()), None => LinkAction::Other("no action".into()), } @@ -214,12 +279,52 @@ fn uri_bytes(u: &str) -> Vec { out } +/// The explicit destination array link `d` goes to now: its `/Dest`, or a GoTo action's `/D`. +fn current_dest(doc: &Document, d: &Dict) -> Option> { + let dest = match d.get(b"Dest") { + Some(dest) => doc.resolve(dest), + None => { + let action = doc.resolve(d.get(b"A")?); + let go_to = action.as_dict().filter(|a| a.name(b"S") == Some(b"GoTo"))?; + doc.resolve(go_to.get(b"D")?) + } + }; + dest.as_array().cloned() +} + fn write_action(doc: &Document, d: &mut Dict, a: &LinkAction) -> Result<(), AnnotError> { match a { - LinkAction::Page(p) => { - let target = page_ref(doc, *p)?; + LinkAction::Page { page, zoom } => { + let target = page_ref(doc, *page)?; + let mut operands = zoom.operands()?; + let current = current_dest(doc, d).filter(|c| LinkZoom::of_dest(doc, c) == *zoom); + if let Some(c) = current { + // The page and zoom the link has (`link_edit` naming its own page): the + // destination is left as it is, position on the page and GoTo form included. + let same_page = match c.first() { + Some(Object::Ref(r)) => *r == target, + Some(Object::Int(i)) => usize::try_from(*i).ok() == Some(*page), + _ => false, + }; + if same_page { + return Ok(()); + } + // A new page with the zoom the link has keeps how that zoom is written: a boxed + // fit (`/FitBH`, `/FitBV`) and its own `/XYZ` factor, even one outside the 1–6400% + // a new zoom is written within. Positions belong to the old page and are dropped. + let (slot, own) = match zoom { + LinkZoom::Percent(_) => (operands.get_mut(3), c.get(4)), + LinkZoom::FitWidth | LinkZoom::FitHeight => (operands.get_mut(0), c.get(1)), + _ => (None, None), + }; + if let (Some(slot), Some(own)) = (slot, own) { + *slot = doc.resolve(own).as_ref().clone(); + } + } + let mut dest = vec![Object::Ref(target)]; + dest.extend(operands); d.remove(b"A"); - d.set(b"Dest".to_vec(), Object::Array(vec![Object::Ref(target), Object::name("Fit")])); + d.set(b"Dest".to_vec(), Object::Array(dest)); } LinkAction::Uri(u) => { let u = u.trim(); @@ -398,10 +503,24 @@ fn email_at(chars: &[char], start: usize) -> Option<(usize, String)> { Some((end, format!("mailto:{address}"))) } +/// The longest web address [`find_urls`] reads, in characters (browsers and servers commonly +/// stop near 2,000). Each address it might be is read this far at most, so text whose brackets +/// never close (`https://a/(https://a/(…`, without a space) costs this much per address in it, +/// not a pass over the rest of the page each. +const MAX_URL_CHARS: usize = 2048; + /// Find web addresses in text: `http(s)://…`, `www.…` and bare e-mail addresses become /// `mailto:`. Returns character ranges and the URI for each. +/// +/// No other scheme is ever found, and a `www.` or e-mail address that is part of one written with +/// another scheme (`ftp://www.example.com`, `smb://admin@example.org`) is not taken out of it: +/// it would open with a scheme the text doesn't say. Brackets inside a web address belong to it +/// when they pair up (`…/wiki/Mercury_(planet)`); an unpaired one ends it. pub fn find_urls(chars: &[char]) -> Vec<(std::ops::Range, String)> { let url_char = |c: char| !c.is_whitespace() && !"<>\"'()[]{}".contains(c); + // Characters found inside an address but not between words: a `www.` after one is part of + // a longer address, and so is an e-mail address after a slash. + let inside_address = |i: usize, set: &str| i > 0 && chars.get(i - 1).is_some_and(|c| set.contains(*c)); // Case-insensitive prefix test that never steps past the end. Comparing against the original // characters keeps every offset in `chars`'s own index space: case-folding a copy can change // its length (U+0130 becomes "i" plus a combining dot), so a folded copy must never supply @@ -412,11 +531,38 @@ pub fn find_urls(chars: &[char]) -> Vec<(std::ops::Range, String)> { let mut i = 0; while i < chars.len() { let boundary = i == 0 || chars.get(i - 1).is_some_and(|c| !c.is_alphanumeric()); - if boundary && (starts_with(i, "http://") || starts_with(i, "https://") || starts_with(i, "www.")) { + let www = starts_with(i, "www.") && !inside_address(i, "/\\:@.-_~%+=&#"); + if boundary && (starts_with(i, "http://") || starts_with(i, "https://") || www) { let mut j = i; - while chars.get(j).is_some_and(|c| url_char(*c)) { + // Brackets pair up inside the address: `depth` are open, the outermost at `opened`. + let (mut depth, mut opened) = (0usize, i); + let mut too_long = false; + loop { + if j.saturating_sub(i) >= MAX_URL_CHARS { + too_long = true; + break; + } + match chars.get(j) { + Some('(') => { + if depth == 0 { + opened = j; + } + depth = depth.saturating_add(1); + } + Some(')') if depth > 0 => depth -= 1, + Some(c) if url_char(*c) => {} + _ => break, + } j += 1; } + // A bracket left open isn't the address's: it ends before it. A run of address + // characters longer than any address is none, and the search goes on after it. + if depth > 0 { + j = opened; + } else if too_long { + i = j; + continue; + } // Trailing punctuation belongs to the sentence. while j > i && chars.get(j - 1).is_some_and(|c| ".,;:!?".contains(*c)) { j -= 1; @@ -429,7 +575,10 @@ pub fn find_urls(chars: &[char]) -> Vec<(std::ops::Range, String)> { continue; } } - if boundary && let Some((end, uri)) = email_at(chars, i) { + if boundary + && !inside_address(i, "/\\") + && let Some((end, uri)) = email_at(chars, i) + { out.push((i..end, uri)); i = end; continue; diff --git a/crates/annot/src/modify.rs b/crates/annot/src/modify.rs index 70abf987aa..eb77e3f973 100644 --- a/crates/annot/src/modify.rs +++ b/crates/annot/src/modify.rs @@ -1,14 +1,17 @@ use super::create::{ - add_annotation, base_dict, bounds, callout_attach, ending_draws, ending_pair, finite, grow, ink_bounds, normalize, num_array, rect_for, rgb, + CALLOUT_SLACK, add_annotation, base_dict, bounds, callout_attach, callout_knee, callout_side, ending_draws, ending_pair, finite, grow, + ink_bounds, normalize, num_array, on_side, polygon_rect, rect_for, rgb, }; use super::draw::{draw_appearance, redraw_or_restore, set_appearance}; -use super::info::{line_endings_of, text_value}; -use super::lookup::{annot_dict, annot_ref, annotation_flags, annots, page_ref, set_annots}; +use super::info::{is_comment_subtype, is_group_subordinate, line_endings_of, own_popup, popup_info, text_value}; +use super::lookup::{annot_dict, annot_ref, annotation_flags, annots, is_fill_sign, page_ref, set_annots}; +use super::text_style::{self, TextStyle}; use super::types::{ BorderShape, FLAG_HIDDEN, FLAG_LOCKED, FLAG_NO_ROTATE, FLAG_NO_ZOOM, FLAG_PRINT, LineEnding, Markup, Meta, NewAnnotation, ReviewState, Rgb, Shape, Style, StyleChange, TextReplacement, }; use super::{AnnotError, Dict, Document, HashMap, HashSet, ObjRef, Object, PdfString, appearance}; +use pdfcraft_fonts::Std14; /// A comment another program drew, whose text the standard fonts can't draw. Redrawing it would /// replace those characters with `?`, so the edit is refused before the dictionary changes. @@ -25,6 +28,15 @@ pub(crate) fn refuse_foreign_text(doc: &Document, d: &Dict) -> Result<(), AnnotE Ok(()) } +/// Thai typed into a text box (`free_text`) that this build can't draw (it has no Thai face): refused +/// with what to do, rather than drawn as `?`. +pub(crate) fn refuse_undrawable_thai(free_text: bool, text: &str) -> Result<(), AnnotError> { + match free_text.then(|| pdfcraft_fonts::shaped_text::thai_refusal(text)).flatten() { + Some(why) => Err(AnnotError::Undrawable(why)), + None => Ok(()), + } +} + /// Format a number for content streams and DA strings. pub(crate) fn n(v: f64) -> String { let s = format!("{:.3}", if v.abs() < 5e-4 { 0.0 } else { v }); @@ -34,11 +46,36 @@ pub(crate) fn n(v: f64) -> String { // ── edits ─────────────────────────────────────────────────────────────────────────────────── -/// A text box's rich text (§12.7.4.3): `/DS` (default style) and `/RC` (XHTML) written from its -/// plain `/Contents`, `/DA` and `/Q`, so viewers that edit rich text show the same thing. +/// [`rich_text_in`] for a text box whose `/DA` was just written (in Helvetica, as PdfCraft writes it). pub(crate) fn rich_text(d: &mut Dict) { + let (color, size) = appearance::parse_da(d); + rich_text_in(d, TextStyle { face: Std14::Helvetica, size, color }); +} + +/// Give a text box the text `style`: `/DA` keeps the font resource it names (`/Helv` when it names +/// none) and the border colour it gives (`RG`, [`appearance::da_border`]) with the new colour and +/// size, and `/DS` and `/RC` carry the face, so the text keeps the face it is drawn in (ISO +/// 32000-2 Table 177). +fn restyle_text(d: &mut Dict, style: TextStyle) { + let border = appearance::da_border(d); + restyle_text_bordered(d, style, border); +} + +/// [`restyle_text`] with the border colour `border` (`None`: the text's). +fn restyle_text_bordered(d: &mut Dict, style: TextStyle, border: Option) { + let font = text_style::da_font_name(d).unwrap_or_else(|| "Helv".into()); + let [r, g, b] = style.color.map(|x| x.clamp(0.0, 1.0)); + let stroke = border.map(|c| c.map(|x| x.clamp(0.0, 1.0))).map(|[r, g, b]| format!("{} {} {} RG ", n(r), n(g), n(b))).unwrap_or_default(); + d.set(b"DA".to_vec(), PdfString::literal(format!("{} {} {} rg {stroke}/{font} {} Tf", n(r), n(g), n(b), n(style.size)).into_bytes())); + rich_text_in(d, TextStyle { color: [r, g, b], ..style }); +} + +/// A text box's rich text (§12.7.4.3): `/DS` (default style) and `/RC` (XHTML) written from its +/// plain `/Contents`, its text `style` and `/Q`, so viewers that edit rich text show the same thing. +fn rich_text_in(d: &mut Dict, style: TextStyle) { let text = d.get(b"Contents").and_then(|c| c.as_string()).map(PdfString::to_text).unwrap_or_default(); - let (col, size) = appearance::parse_da(d); + let TextStyle { face, size, color: col } = style; + let (family, weight, slant) = text_style::css(face); let hex = format!( "#{:02X}{:02X}{:02X}", (col[0].clamp(0.0, 1.0) * 255.0).round() as u8, @@ -53,13 +90,15 @@ pub(crate) fn rich_text(d: &mut Dict) { let esc = |s: &str| s.replace('&', "&").replace('<', "<").replace('>', ">").replace('"', """); let body: String = text.split('\n').map(|line| format!("

{}

", esc(line.trim_end_matches('\r')))).collect(); let style = format!( - "font-size:{}pt;text-align:{align};color:{hex};font-weight:normal;font-style:normal;font-family:Helvetica;font-stretch:normal", + "font-size:{}pt;text-align:{align};color:{hex};font-weight:{weight};font-style:{slant};font-family:{family};font-stretch:normal", n(size) ); let rc = format!( "{body}" ); - d.set(b"DS".to_vec(), PdfString::text(&format!("font: Helvetica {}pt; text-align:{align}; color:{hex}", n(size)))); + // "font: Helvetica 12pt" for PdfCraft's own boxes; other faces add their slant and weight. + let keywords = format!("{}{}", if slant == "normal" { "" } else { "italic " }, if weight == "normal" { "" } else { "bold " }); + d.set(b"DS".to_vec(), PdfString::text(&format!("font: {keywords}{family} {}pt; text-align:{align}; color:{hex}", n(size)))); d.set(b"RC".to_vec(), PdfString::text(&rc)); } @@ -70,9 +109,14 @@ pub(crate) fn touch(d: &mut Dict, meta: &Meta) { } /// Delete a comment together with its pop-up and its replies (and theirs), as Acrobat does. -/// Refused while it, or a reply that would go with it, is locked. +/// Refused while it, or a reply that would go with it, is locked. A member of a group (`/RT +/// /Group`), its primary included, goes alone, as Acrobat deletes the member clicked and keeps the +/// rest of its group, although ISO 32000-2 §12.5.6.2 lists deletion among the operations that act +/// on the entire group; a subordinate left without its primary is a comment of its own and +/// loses its `/IRT` ([`release_members`]). Replace Text's strikeout and caret are one comment and +/// go together from either ([`own_deletion`]). pub fn delete_annotation(doc: &mut Document, page: usize, index: usize) -> Result<(), AnnotError> { - deletable(doc, page, index)?; + let index = deleted_index(doc, page, index)?; lift_annotation(doc, page, index) } @@ -80,23 +124,44 @@ pub fn delete_annotation(doc: &mut Document, page: usize, index: usize) -> Resul /// changing anything: what a Delete command asks before it is offered. It is refused while the /// annotation is locked, or a reply that would go with it is. pub fn deletable(doc: &Document, page: usize, index: usize) -> Result<(), AnnotError> { + deleted_index(doc, page, index).map(|_| ()) +} + +/// [`deletable`], returning the index [`delete_annotation`] deletes from: the caret for Replace +/// Text's strikeout, else `index`. +fn deleted_index(doc: &Document, page: usize, index: usize) -> Result { let list = annots(doc, page_ref(doc, page)?); - let target = list.get(index).ok_or(AnnotError::NoSuchAnnotation { page, index })?; + let mut target = list.get(index).ok_or(AnnotError::NoSuchAnnotation { page, index })?; + let mut index = index; if let Some(r) = target.as_ref() { unlocked(doc, r)?; } - let doomed = with_replies_and_popups(doc, &list, seed(target)); + // Replace Text is one comment: its strikeout goes with its caret, the group's primary, and + // the caret with it, as Acrobat deletes the pair from either. + let caret = doc + .resolve(target) + .as_dict() + .filter(|d| is_group_subordinate(d) && !own_deletion(doc, d)) + .and_then(|d| d.reference(b"IRT")) + .and_then(|irt| list.iter().enumerate().find(|(_, e)| e.as_ref() == Some(irt))); + if let Some((i, e)) = caret { + (target, index) = (e, i); + } + let doomed = with_replies_and_popups(doc, &list, seed(doc, target)); // A locked reply can't be deleted on its own, so it doesn't go with the comment or reply it - // answers either. Pop-ups follow their comment. - for (_, e) in list.iter().enumerate().filter(|(i, e)| *i != index && goes_with(e, &doomed)) { + // answers either (ISO 32000-2 §12.5.3, Table 167: Locked); nor does the other half of Replace + // Text. Pop-ups follow their comment. + for (i, e) in list.iter().enumerate().filter(|(i, e)| (*i != index || caret.is_some()) && goes_with(doc, e, &doomed)) { let obj = doc.resolve(e); if let Some(d) = obj.as_dict().filter(|d| d.name(b"Subtype") != Some(b"Popup".as_slice())) && annotation_flags(doc, d) & FLAG_LOCKED != 0 { - return Err(AnnotError::Invalid("a reply to it is locked".into())); + let grouped = is_group_subordinate(d) || caret.is_some_and(|(c, _)| c == i); + let what = if grouped { "a comment in its group is locked" } else { "a reply to it is locked" }; + return Err(AnnotError::Invalid(what.into())); } } - Ok(()) + Ok(index) } /// Take the annotation at `(page, index)` out of its page with its pop-up and replies, as @@ -106,31 +171,95 @@ pub fn lift_annotation(doc: &mut Document, page: usize, index: usize) -> Result< let p = page_ref(doc, page)?; let list = annots(doc, p); let target = list.get(index).ok_or(AnnotError::NoSuchAnnotation { page, index })?; - let doomed = with_replies_and_popups(doc, &list, seed(target)); - let kept: Vec = list.iter().enumerate().filter(|(i, e)| *i != index && !goes_with(e, &doomed)).map(|(_, e)| e.clone()).collect(); + let doomed = with_replies_and_popups(doc, &list, seed(doc, target)); + let kept: Vec = list.iter().enumerate().filter(|(i, e)| *i != index && !goes_with(doc, e, &doomed)).map(|(_, e)| e.clone()).collect(); + let kept = release_members(doc, kept, &doomed)?; Ok(set_annots(doc, p, kept)?) } +/// The `/Annots` entries `kept` after deleting `going`, with every group subordinate that stays +/// ([`own_deletion`]) but whose `/IRT` names an annotation going made a comment of its own: its +/// `/IRT` is removed and its `/RT /Group` kept, as Acrobat does when a group's primary is deleted. +/// A deleted primary then stays reachable from nothing, so a full save leaves its text out. An +/// inline entry is rewritten in the list; a referenced one is updated in place. Each entry is +/// read once, following no chain. +pub fn release_members(doc: &mut Document, kept: Vec, going: &HashSet) -> Result, pdfcraft_cos::CosError> { + let orphaned = |d: &Dict| is_group_subordinate(d) && d.reference(b"IRT").is_some_and(|r| going.contains(&r)); + let mut out = Vec::with_capacity(kept.len()); + for e in kept { + match e.as_ref() { + Some(r) => { + if doc.get(r).as_dict().is_some_and(orphaned) { + doc.update_dict(r, |d| { + d.remove(b"IRT"); + })?; + } + out.push(e); + } + None => match e { + Object::Dict(mut d) if orphaned(&d) => { + d.remove(b"IRT"); + out.push(Object::Dict(d)); + } + other => out.push(other), + }, + } + } + Ok(out) +} + /// What deleting the `/Annots` entry `target` starts from: its reference, or for an inline -/// comment (which has none to collect) the pop-up it names, which still goes. -fn seed(target: &Object) -> Option { - target.as_ref().or_else(|| target.as_dict().and_then(|d| d.reference(b"Popup"))) +/// comment (which has none to collect) the pop-up it names, which still goes. The strikeout of a +/// Replace Text proposal also takes the caret it is grouped under, so the two go together +/// whichever is deleted. +fn seed(doc: &Document, target: &Object) -> Vec { + let own = target.as_ref().or_else(|| target.as_dict().and_then(|d| d.reference(b"Popup"))); + own.into_iter().chain(replaced_by(doc, target)).collect() +} + +/// The caret a Replace Text strikeout is grouped under (`/IT /StrikeOutTextEdit`, `/RT /Group`, +/// `/IRT` to a caret): the proposal's primary, which holds the replacement. +fn replaced_by(doc: &Document, strike: &Object) -> Option { + let obj = doc.resolve(strike); + let d = obj.as_dict()?; + let edit = d.name(b"Subtype") == Some(b"StrikeOut") && d.name(b"IT") == Some(b"StrikeOutTextEdit") && d.name(b"RT") == Some(b"Group"); + let caret = d.reference(b"IRT").filter(|_| edit)?; + (annot_dict(doc, caret).name(b"Subtype") == Some(b"Caret")).then_some(caret) } /// Whether the `/Annots` entry `entry` goes with `going` (from [`with_replies_and_popups`]): it is /// one of them, or it is written inline (a file can, though the standard asks for references) and /// replies to (`/IRT`) or pops up for (`/Parent`) one of them. An inline entry has no reference /// for the walk to collect, so it is matched here. -pub fn goes_with(entry: &Object, going: &HashSet) -> bool { +pub fn goes_with(doc: &Document, entry: &Object, going: &HashSet) -> bool { match entry.as_ref() { Some(r) => going.contains(&r), - None => entry.as_dict().is_some_and(|d| [d.reference(b"IRT"), d.reference(b"Parent")].into_iter().flatten().any(|r| going.contains(&r))), + None => entry.as_dict().is_some_and(|d| hangs_off(doc, d).into_iter().flatten().any(|r| going.contains(&r))), } } +/// What the annotation `d` goes with when it is deleted: what it replies to (`/IRT`) and what it +/// pops up for (`/Parent`). A group subordinate (`/RT /Group`) is deleted on its own +/// ([`own_deletion`]), so its `/IRT` doesn't count. +fn hangs_off(doc: &Document, d: &Dict) -> [Option; 2] { + [d.reference(b"IRT").filter(|_| !own_deletion(doc, d)), d.reference(b"Parent")] +} + +/// Whether the annotation `d` stays when its group's primary is deleted: a group subordinate +/// (`/RT /Group`) does, as Acrobat deletes only the member clicked, unless it is half of Replace +/// Text, which Acrobat deletes as one comment: the strikeout (`/IT /StrikeOutTextEdit`) under its +/// caret, or a caret under such a strikeout, as earlier PdfCraft builds wrote the pair. +fn own_deletion(doc: &Document, d: &Dict) -> bool { + let edit = |d: &Dict| d.name(b"IT") == Some(b"StrikeOutTextEdit".as_slice()); + let under_edit = + || d.name(b"Subtype") == Some(b"Caret".as_slice()) && d.reference(b"IRT").is_some_and(|r| doc.get(r).as_dict().is_some_and(edit)); + is_group_subordinate(d) && !edit(d) && !under_edit() +} + /// `doomed` and what goes with it, as Acrobat deletes a comment: the entries of `list` (one or /// more pages' `/Annots`) that reply to (`/IRT`) or pop up for (`/Parent`) anything going, their -/// own replies, and the pop-ups going annotations name (`/Popup`). An inline reply has no +/// own replies, and the pop-ups going annotations name (`/Popup`). A group subordinate stays +/// ([`own_deletion`]). An inline reply has no /// reference to collect, but the pop-up it names is. Each annotation is visited once, so a long or /// cyclic chain costs no more than its length. Filter a list with [`goes_with`], which also /// matches inline entries. @@ -142,7 +271,7 @@ pub fn with_replies_and_popups(doc: &Document, list: &[Object], doomed: impl Int // What goes with the annotation this entry hangs off: the entry, or for an inline one the // pop-up it names. let Some(goes) = e.as_ref().or_else(|| d.reference(b"Popup")) else { continue }; - for on in [d.reference(b"IRT"), d.reference(b"Parent")].into_iter().flatten() { + for on in hangs_off(doc, d).into_iter().flatten() { hanging.entry(on).or_default().push(goes); } } @@ -163,14 +292,17 @@ pub fn with_replies_and_popups(doc: &Document, list: &[Object], doomed: impl Int /// around its text stays snug around the new text, centred where it was. pub fn set_contents(doc: &mut Document, page: usize, index: usize, text: &str, meta: &Meta) -> Result<(), AnnotError> { let (_, r) = annot_ref(doc, page, index)?; - let d = annot_dict(doc, r); - let free_text = d.name(b"Subtype") == Some(b"FreeText"); - let snug = free_text && is_snug(&d); + let before = annot_dict(doc, r); + let free_text = before.name(b"Subtype") == Some(b"FreeText"); + refuse_undrawable_thai(free_text, text)?; + let snug = free_text && is_snug(&before); + // The text keeps the look it is drawn with (`/DA`, or `/DS` where it sets one). + let style = text_style::text_style(&before, text_style::dr_face(doc, &before)); doc.update_dict(r, |d| { d.set(b"Contents".to_vec(), PdfString::text(text)); // The rich-text version follows the new plain text (text boxes), or would contradict it. if free_text { - rich_text(d); + rich_text_in(d, style); } else { d.remove(b"RC"); } @@ -203,7 +335,9 @@ pub fn set_contents(doc: &mut Document, page: usize, index: usize, text: &str, m fn fitted_box(doc: &Document, r: ObjRef, text: &str) -> Option<[f64; 4]> { let d = annot_dict(doc, r); let nums = |key: &[u8]| -> Option> { d.get(key)?.as_array()?.iter().map(|o| o.as_f64()).collect() }; - let (_, size) = appearance::parse_da(&d); + // Measured in the face and size the text is drawn in (`/DA`, or `/DS` where it sets one). + let TextStyle { face, size, .. } = text_style::text_style(&d, text_style::dr_face(doc, &d)); + let measure = |s: &str, size: f64| appearance::text_width_in(face, s, size); let pad = 2.0 + border_width_of(&d); let rect = nums(b"Rect").filter(|v| v.len() == 4 && v.iter().all(|x| x.is_finite()))?; let rect = [rect[0], rect[1], rect[2], rect[3]]; @@ -221,11 +355,11 @@ fn fitted_box(doc: &Document, r: ObjRef, text: &str) -> Option<[f64; 4]> { w } else { // No usable old width: fall back to the creation-time width (at most 300 pt). - let longest = text.lines().map(|l| appearance::text_width(l, size)).fold(0.0, f64::max); + let longest = text.lines().map(|l| measure(l, size)).fold(0.0, f64::max); (longest + 2.0 * pad + 4.0).clamp(40.0, 300.0) }; - let lines = appearance::wrap(text, size, (w - 2.0 * pad).max(1.0)).len().max(1) as f64; - Some([x0, top - lines * size * 1.2 - 2.0 * pad - 2.0, x0 + w, top]) + let height = appearance::text_lines_height(&pdfcraft_fonts::wrap_with(text, size, (w - 2.0 * pad).max(1.0), measure), size); + Some([x0, top - height - 2.0 * pad - 2.0, x0 + w, top]) } /// Reply to a comment; returns the reply's index in the page's `/Annots`. @@ -237,8 +371,9 @@ pub fn add_reply(doc: &mut Document, page: usize, index: usize, text: &str, auth reply(doc, page, index, text, author, meta, None) } -/// Set a comment's review status (Acrobat: "Set status ▸ Accepted"…). Like Acrobat this adds a -/// state reply (`/State`, `/StateModel /Review`) by `author`; the latest one wins. +/// Set a comment's review status ("Set status ▸ Accepted"…). This adds a hidden state reply +/// (`/State`, `/StateModel /Review`) by `author`, in reply to `author`'s previous one when there is +/// one; each reviewer's latest is their status. pub fn set_review_state(doc: &mut Document, page: usize, index: usize, state: ReviewState, author: &str, meta: &Meta) -> Result { let text = format!("{} set by {}", state.name(), if author.is_empty() { "unknown" } else { author }); reply(doc, page, index, &text, author, meta, Some(state)) @@ -271,6 +406,11 @@ fn reply( d.set(b"State".to_vec(), PdfString::text(s.name())); d.set(b"StateModel".to_vec(), PdfString::text("Review")); d.set(b"Subj".to_vec(), PdfString::text("Status")); + // Hidden (Table 167, bit 2): the state "is not specified in the annotation itself but in a + // separate text annotation" (ISO 32000-2 §12.5.6.3), so this reply is a state record, not + // a note, and is neither drawn nor listed as one. + d.set(b"F".to_vec(), Object::Int(FLAG_HIDDEN | FLAG_PRINT | FLAG_NO_ZOOM | FLAG_NO_ROTATE)); + d.set(b"IRT".to_vec(), Object::Ref(last_state_reply(doc, p, parent, author, "Review").unwrap_or(parent))); } // An empty form: nothing is drawn for the reply itself. let mut fd = Dict::new(); @@ -298,6 +438,83 @@ pub(crate) fn unlocked(doc: &Document, r: ObjRef) -> Result<(), AnnotError> { if flags(doc, r) & FLAG_LOCKED != 0 { Err(AnnotError::Invalid("the comment is locked".into())) } else { Ok(()) } } +/// Open, close or move a comment's pop-up window (Acrobat: double-click a comment to open it, +/// drag its title bar to move it). `rect` is the pop-up's new `/Rect` in user space; `open` its +/// `/Open`. A comment without a `/Popup` gets one, placed to the right of the comment as new +/// notes get theirs. A text box, a Fill & Sign mark or a reply has no pop-up of its own (a reply +/// shows in its parent's). The lock does not apply: the window's place and state are not the +/// comment. The modification date is left alone: nothing of the comment changed. +pub fn set_popup(doc: &mut Document, page: usize, index: usize, rect: Option<[f64; 4]>, open: Option, meta: &Meta) -> Result<(), AnnotError> { + let (p, r) = annot_ref(doc, page, index)?; + let d = annot_dict(doc, r); + let subtype = String::from_utf8_lossy(d.name(b"Subtype").unwrap_or_default()).into_owned(); + if !is_comment_subtype(&subtype) || subtype == "FreeText" || is_fill_sign(doc, &d) || d.contains(b"IRT") { + return Err(AnnotError::Invalid(format!("{subtype} comments have no pop-up of their own"))); + } + let rect = match rect { + Some(rect) => { + let rect = normalize(rect); + if !finite(&rect) || rect[2] - rect[0] < 1.0 || rect[3] - rect[1] < 1.0 { + return Err(AnnotError::Invalid("invalid pop-up rectangle".into())); + } + Some(rect) + } + None => None, + }; + let _ = meta; + // Where a new pop-up goes: to the right of the comment, as new notes' do. + let own: Vec = d + .get(b"Rect") + .map(|o| doc.resolve(o)) + .and_then(|o| o.as_array().cloned()) + .unwrap_or_default() + .iter() + .filter_map(|v| doc.resolve(v).as_f64()) + .collect(); + let beside = match own[..] { + [_, _, x1, y1] if finite(&own) => [x1 + 10.0, y1 - 120.0, x1 + 210.0, y1], + _ => [0.0, 0.0, 200.0, 120.0], + }; + // Only a pop-up that is this comment's: `/Popup` naming another annotation (or a pop-up + // whose `/Parent` is another comment) would move or open that one, so it gets its own. + match own_popup(doc, Some(r), &d) { + Some(pr) => { + // One without a `/Rect` of four numbers reads as no pop-up (`summaries`), so it gets + // the new pop-up's place: opening it then shows it. + let rect = rect.or_else(|| popup_info(doc, Some(r), &d).is_none().then_some(beside)); + doc.update_dict(pr, |p| { + if let Some(rect) = rect { + p.set(b"Rect".to_vec(), num_array(&rect)); + } + if let Some(open) = open { + p.set(b"Open".to_vec(), Object::Bool(open)); + } + })? + } + None => { + let mut pd = Dict::new(); + pd.set(b"Type".to_vec(), Object::name("Annot")); + pd.set(b"Subtype".to_vec(), Object::name("Popup")); + pd.set(b"Rect".to_vec(), num_array(&rect.unwrap_or(beside))); + pd.set(b"Parent".to_vec(), Object::Ref(r)); + pd.set(b"Open".to_vec(), Object::Bool(open.unwrap_or(false))); + pd.set(b"F".to_vec(), Object::Int(FLAG_PRINT | FLAG_NO_ZOOM | FLAG_NO_ROTATE)); + let pref = doc.add(Object::Dict(pd)); + doc.update_dict(r, |d| d.set(b"Popup".to_vec(), Object::Ref(pref)))?; + let mut list = annots(doc, p); + list.push(Object::Ref(pref)); + set_annots(doc, p, list)?; + } + } + // A note's own `/Open` says the same thing to viewers that read it there (§12.5.6.4). + if let Some(open) = open + && (subtype == "Text" || d.contains(b"Open")) + { + doc.update_dict(r, |d| d.set(b"Open".to_vec(), Object::Bool(open)))?; + } + Ok(()) +} + /// Lock or unlock a comment (the Locked flag). A locked comment can't be moved, resized, /// restyled or deleted; its text and replies stay editable, as in Acrobat. pub fn set_locked(doc: &mut Document, page: usize, index: usize, locked: bool) -> Result<(), AnnotError> { @@ -307,8 +524,9 @@ pub fn set_locked(doc: &mut Document, page: usize, index: usize, locked: bool) - Ok(()) } -/// Mark or unmark a comment with a checkmark. Like Acrobat this adds a hidden state reply with -/// `/StateModel /Marked` by `author`; the latest one wins. It's private bookkeeping, not a status. +/// Mark or unmark a comment with a checkmark. This adds a hidden state reply with `/StateModel +/// /Marked` by `author`, in reply to `author`'s previous one when there is one; their latest one +/// wins. It's each reviewer's private bookkeeping, not a status. pub fn set_marked(doc: &mut Document, page: usize, index: usize, marked: bool, author: &str, meta: &Meta) -> Result { let (p, parent) = annot_ref(doc, page, index)?; let pd = annot_dict(doc, parent); @@ -319,7 +537,7 @@ pub fn set_marked(doc: &mut Document, page: usize, index: usize, marked: bool, a let text = format!("{state} set by {}", if author.is_empty() { "unknown" } else { author }); let mut d = base_dict("Text", [0.0; 4], p, &text, author, meta); d.set(b"Rect".to_vec(), pd.get(b"Rect").cloned().unwrap_or_else(|| num_array(&[0.0; 4]))); - d.set(b"IRT".to_vec(), Object::Ref(parent)); + d.set(b"IRT".to_vec(), Object::Ref(last_state_reply(doc, p, parent, author, "Marked").unwrap_or(parent))); d.set(b"F".to_vec(), Object::Int(FLAG_HIDDEN | FLAG_PRINT | FLAG_NO_ZOOM | FLAG_NO_ROTATE)); d.set(b"State".to_vec(), PdfString::text(state)); d.set(b"StateModel".to_vec(), PdfString::text("Marked")); @@ -331,6 +549,56 @@ pub fn set_marked(doc: &mut Document, page: usize, index: usize, marked: bool, a Ok(i) } +/// `author`'s latest `model` state reply about `target` on page `p`, for the next change to reply +/// to: ISO 32000-2 §12.5.6.3, "Additional state changes shall be made by adding text annotations +/// in reply to the previous reply for a given user". A state reply is about the +/// annotation its `/IRT` chain reaches past other state replies; a loop reaches nothing. Each +/// reply's answer is kept, so a long history is walked once. +fn last_state_reply(doc: &Document, p: ObjRef, target: ObjRef, author: &str, model: &str) -> Option { + use std::collections::{HashMap, HashSet}; + // As the readers see it: a readable `/State` (not null, empty or another type), or a known + // `/StateModel` whose default state applies. + let is_state = |r: ObjRef| { + doc.get(r).as_dict().is_some_and(|d| { + text_value(doc, d, b"State").is_some() || matches!(text_value(doc, d, b"StateModel").as_deref(), Some("Marked" | "Review")) + }) + }; + let irt = |r: ObjRef| doc.get(r).as_dict().and_then(|d| d.get(b"IRT").and_then(Object::as_ref)); + let mut reaches: HashMap> = HashMap::new(); + let mut mine = Vec::new(); + for r in annots(doc, p).iter().filter_map(Object::as_ref).filter(|&r| r != target) { + let d = annot_dict(doc, r); + if text_value(doc, &d, b"StateModel").as_deref() != Some(model) || text_value(doc, &d, b"T").unwrap_or_default() != author.trim() { + continue; + } + let (mut path, mut seen, mut at) = (vec![r], HashSet::from([r]), irt(r)); + let about = loop { + let Some(up) = at else { break None }; + if let Some(&known) = reaches.get(&up) { + break known; + } + if !is_state(up) { + break Some(up); + } + if !seen.insert(up) { + break None; + } + path.push(up); + at = irt(up); + }; + for x in path { + reaches.insert(x, about); + } + if about == Some(target) { + mine.push(r); + } + } + // The end of the chain, whatever order the page lists it in: a reply none of the others + // replies to; among several (changes that don't chain), the last listed. + let answered: HashSet = mine.iter().filter_map(|&r| irt(r)).collect(); + mine.iter().rev().copied().find(|r| !answered.contains(r)) +} + /// Move a comment (and its pop-up) by `(dx, dy)` points. The appearance moves with `/Rect`. pub fn move_annotation(doc: &mut Document, page: usize, index: usize, dx: f64, dy: f64, meta: &Meta) -> Result<(), AnnotError> { if !finite(&[dx, dy]) { @@ -459,7 +727,7 @@ fn text_extent(d: &Dict, wrap: f64) -> [f64; 2] { let (_, size) = appearance::parse_da(d); let lines = appearance::wrap(&text, size, wrap.max(1.0)); let widest = lines.iter().map(|l| appearance::text_width(l, size)).fold(0.0, f64::max); - [widest, lines.len().max(1) as f64 * size * 1.2] + [widest, appearance::text_lines_height(&lines, size)] } /// The width and height of FreeText `d`'s text box `tb` as displayed: swapped on a page turned by @@ -531,10 +799,12 @@ pub fn set_border_shape(doc: &mut Document, page: usize, index: usize, shape: Bo /// At most this many points reshape a comment (a drawn polygon has a handful). const MAX_POINTS: usize = 100_000; -/// Reshape a line, connected lines, a polygon or a cloud: its points in user space (a line's two -/// ends, the others' corners), as dragging the comment's handles sets them. `/Rect` follows the -/// points with room for the line's width and endings, as when the comment was drawn, and the -/// appearance is redrawn. A measurement keeps the points it measured. +/// Reshape a line, connected lines, a polygon, a cloud or a callout: its points in user space (a +/// line's two ends, the others' corners; a callout's arrow point and, if it has one, its knee), as +/// dragging the comment's handles sets them. `/Rect` follows the points with room for the line's +/// width and endings, as when the comment was drawn, and the appearance is redrawn. A callout's +/// text box stays where it is and its leader line re-attaches. A measurement keeps the points it +/// measured. pub fn set_points(doc: &mut Document, page: usize, index: usize, points: &[[f64; 2]], meta: &Meta) -> Result<(), AnnotError> { let (_, r) = annot_ref(doc, page, index)?; unlocked(doc, r)?; @@ -545,22 +815,30 @@ pub fn set_points(doc: &mut Document, page: usize, index: usize, points: &[[f64; if points.len() > MAX_POINTS { return Err(AnnotError::Invalid(format!("at most {MAX_POINTS} points"))); } - let style = Style { width: border_width_of(&d), ..Style::default() }; + if d.name(b"Subtype") == Some(b"FreeText") && callout_line(doc, &d).is_some() { + // A callout's points: its arrow point, then optionally its knee (a four-number `/CL` + // gains one; ISO 32000-2 Table 177 allows both forms). The arrow point alone keeps a knee. + if !matches!(points.len(), 1 | 2) || !finite(&points.concat()) { + return Err(AnnotError::Invalid("a callout has an arrow point and, optionally, a knee".into())); + } + let text_box = callout_text_box(doc, &d).ok_or_else(|| AnnotError::Invalid("the callout's text box (/RD) can't be read".into()))?; + return place_text_box(doc, r, text_box, Some(points), meta, false); + } + // The width the appearance draws with (`/BS`, else `/Border`), so the room is the drawn stroke's. + let style = Style { width: appearance::border_width(&d), ..Style::default() }; let ends = line_endings_of(&d).unwrap_or_default(); let (start, end) = (ends.first().copied().unwrap_or(LineEnding::None), ends.get(1).copied().unwrap_or(LineEnding::None)); let subtype = String::from_utf8_lossy(d.name(b"Subtype").unwrap_or_default()).into_owned(); - let (shape, key): (Shape, &[u8]) = match (subtype.as_str(), points) { - ("Line", [from, to]) => (Shape::Line { from: *from, to: *to, start, end }, b"L"), + // Each checks the points too: finite, and far enough apart to draw. + let (rect, key): ([f64; 4], &[u8]) = match (subtype.as_str(), points) { + ("Line", [from, to]) => (rect_for(&Shape::Line { from: *from, to: *to, start, end }, &style)?, b"L"), ("Line", _) => return Err(AnnotError::Invalid("a line has two ends".into())), - ("PolyLine", _) => (Shape::PolyLine { vertices: points.to_vec(), start, end }, b"Vertices"), - ("Polygon", _) => { - let cloud = d.get(b"BE").and_then(|b| b.as_dict()).is_some_and(|b| b.name(b"S") == Some(b"C")); - (Shape::Polygon { vertices: points.to_vec(), cloud }, b"Vertices") - } + ("PolyLine", _) => (rect_for(&Shape::PolyLine { vertices: points.to_vec(), start, end }, &style)?, b"Vertices"), + // Room for the cloud as the appearance draws it: `/BE /S /C` at its own intensity + // (Table 169), which may reach further than a new cloud's intensity 1. + ("Polygon", _) => (polygon_rect(points, style.width, appearance::cloud_intensity(&d))?, b"Vertices"), (other, _) => return Err(AnnotError::Invalid(format!("{other} comments have no points to move"))), }; - // Checks the points too: finite, and far enough apart to draw. - let rect = rect_for(&shape, &style)?; let before = d; doc.update_dict(r, |d| { d.set(key.to_vec(), num_array(&points.concat())); @@ -620,6 +898,43 @@ fn resize_ink(doc: &mut Document, r: ObjRef, rect: [f64; 4], meta: &Meta) -> Res /// leader line re-attaches and `/Rect` grows to hold it, and the appearance is redrawn /// (`new_text`: for changed contents, see [`set_appearance`] and `draw_appearance`). fn apply_text_box(doc: &mut Document, r: ObjRef, rect: [f64; 4], meta: &Meta, new_text: bool) -> Result<(), AnnotError> { + place_text_box(doc, r, rect, None, meta, new_text) +} + +/// A callout's leader line (`/CL`, ISO 32000-2 Table 177): four numbers (start, end) or six +/// (start, knee, end), as points in user space. `None` when the annotation has no usable `/CL`. +pub(crate) fn callout_line(doc: &Document, d: &Dict) -> Option> { + // All or nothing: a non-number would shift every later number into the wrong point. + let l: Vec = doc.resolve(d.get(b"CL")?).as_array()?.iter().map(|x| doc.resolve(x).as_f64()).collect::>()?; + (matches!(l.len(), 4 | 6) && finite(&l)).then(|| l.as_chunks::<2>().0.to_vec()) +} + +/// A callout's text box: `/Rect` inset by `/RD` (ISO 32000-2 Table 177). `None` without both. +pub(crate) fn callout_text_box(doc: &Document, d: &Dict) -> Option<[f64; 4]> { + let nums = |key: &[u8]| -> Option> { doc.resolve(d.get(key)?).as_array()?.iter().map(|o| doc.resolve(o).as_f64()).collect() }; + let r = nums(b"Rect").filter(|v| v.len() == 4 && finite(v))?; + let rd = nums(b"RD").filter(|v| v.len() == 4 && finite(v) && v.iter().all(|x| *x >= 0.0))?; + let r = normalize([r[0], r[1], r[2], r[3]]); + let tb = [r[0] + rd[0], r[1] + rd[1], r[2] - rd[2], r[3] - rd[3]]; + (tb[2] - tb[0] >= 1.0 && tb[3] - tb[1] >= 1.0).then_some(tb) +} + +/// As [`apply_text_box`]. For a callout, `lead` (its arrow point and optional knee, dragged by +/// their handles) replaces the leader line's own points. The arrow point never moves with the box +/// ("The text box moves around a stationary anchor point—the arrow on the end-point line", +/// Acrobat DC Help, 11/28/2019, p. 357), and the line follows the box as Acrobat Pro DC (a 2021 +/// Continuous release) was seen to: +/// - the box moved without changing size: the knee and the end move with it (also seen in +/// Acrobat Pro 2026.002.22031); +/// - the box resized with the side the line meets left where it was (the end still on it): the +/// line doesn't change; +/// - otherwise (that side moved): the line meets the middle of that side, the knee +/// [`CALLOUT_KNEE_LINE`] out from it ([`callout_knee_on`]). Acrobat also moves the box next to +/// the arrow point then; PdfCraft leaves the box where it was put. +/// +/// With `lead`, the end stays where it is while it is on the side facing the knee (dragging the +/// knee or the arrow point leaves it in Acrobat), and otherwise meets the middle of that side. +fn place_text_box(doc: &mut Document, r: ObjRef, rect: [f64; 4], lead: Option<&[[f64; 2]]>, meta: &Meta, new_text: bool) -> Result<(), AnnotError> { let rect = normalize(rect); if !finite(&rect) || rect[2] - rect[0] < 1.0 || rect[3] - rect[1] < 1.0 { return Err(AnnotError::Invalid("invalid rectangle (too small)".into())); @@ -628,17 +943,62 @@ fn apply_text_box(doc: &mut Document, r: ObjRef, rect: [f64; 4], meta: &Meta, ne if !new_text { refuse_foreign_text(doc, &d)?; } - // A callout's rectangle is its text box: the leader line re-attaches and `/Rect` grows to hold it. - let callout = d.get(b"CL").and_then(|o| o.as_array()).map(|a| a.iter().filter_map(|x| x.as_f64()).collect::>()).filter(|l| l.len() == 6); - let (outer, cl) = match callout { - Some(mut l) => { - let attach = callout_attach(rect, [l[2], l[3]]); - (l[4], l[5]) = (attach[0], attach[1]); - let pad = border_width_of(&d) / 2.0 + appearance::arrow_size(border_width_of(&d)) + 1.0; - let lead = grow(bounds([[l[0], l[1]], [l[2], l[3]]].into_iter()).unwrap_or_default(), pad); - ([rect[0].min(lead[0]), rect[1].min(lead[1]), rect[2].max(lead[2]), rect[3].max(lead[3])], Some(l)) + // A callout's rectangle is its text box: `/Rect` grows to hold the leader line. A four-number + // `/CL` (no knee) stays four numbers unless `lead` adds a knee. + let old_line = callout_line(doc, &d); + let old_box = callout_text_box(doc, &d); + let old_end = old_line.as_deref().and_then(<[[f64; 2]]>::last).copied(); + // An arrow point alone moves the arrow: a knee the line has stays where it is. + let kept: [[f64; 2]; 2]; + let lead = match (lead, old_line.as_deref()) { + (Some([point]), Some([_, knee, _])) => { + kept = [*point, *knee]; + Some(&kept[..]) } - None => (rect, None), + _ => lead, + }; + let line: Option> = match (lead, old_line.as_deref()) { + (Some(lead @ [.., toward]), _) => { + let side = callout_side(rect, *toward); + let end = old_end.filter(|e| on_side(rect, side, *e)).unwrap_or_else(|| callout_attach(rect, *toward)); + Some(lead.iter().copied().chain([end]).collect()) + } + (Some(_), _) => None, + (None, Some(old @ [start, .., end])) => { + let (start, end) = (*start, *end); + let knee = old.get(1).filter(|_| old.len() == 3).copied(); + let toward = knee.unwrap_or(start); + Some(match old_box { + // Moved, not resized: the knee and the end come along. + Some(b) + if ((rect[2] - rect[0]) - (b[2] - b[0])).abs() < CALLOUT_SLACK && ((rect[3] - rect[1]) - (b[3] - b[1])).abs() < CALLOUT_SLACK => + { + let (dx, dy) = (rect[0] - b[0], rect[1] - b[1]); + let shift = |p: [f64; 2]| [p[0] + dx, p[1] + dy]; + [start].into_iter().chain(knee.map(shift)).chain([shift(end)]).collect() + } + // Resized, the side the line meets still under its end: nothing changes. + Some(b) if on_side(rect, callout_side(b, toward), end) => old.to_vec(), + // That side moved: laid out again as a new callout is, from the side facing the + // arrow point (the box may now be on the knee's other side). + Some(_) => { + let new_knee = callout_knee(rect, start); + let end = callout_attach(rect, new_knee); + [start].into_iter().chain(knee.map(|_| new_knee)).chain([end]).collect() + } + None => [start].into_iter().chain(knee).chain([callout_attach(rect, toward)]).collect(), + }) + } + (None, _) => None, + }; + let (outer, cl) = match line.as_deref() { + Some(line @ [lead @ .., _]) => { + // The end is on the box, so only the arrow point and the knee need room around them. + let pad = border_width_of(&d) / 2.0 + appearance::arrow_size(border_width_of(&d)) + 1.0; + let lead_box = grow(bounds(lead.iter().copied()).unwrap_or_default(), pad); + ([rect[0].min(lead_box[0]), rect[1].min(lead_box[1]), rect[2].max(lead_box[2]), rect[3].max(lead_box[3])], Some(line.concat())) + } + _ => (rect, None), }; let (shape, width) = (BorderShape::of(&d), border_width_of(&d)); let free_text = d.name(b"Subtype") == Some(b"FreeText"); @@ -669,8 +1029,9 @@ fn apply_text_box(doc: &mut Document, r: ObjRef, rect: [f64; 4], meta: &Meta, ne } } -/// The border width as the appearance draws it: `/BS /W`, else the third number of `/Border` -/// (another app's), else 1. +/// The width a comment's border is drawn with ([`appearance::border_width`]: `/BS /W`, else +/// `/Border`'s third number, 1 when neither gives a usable one), so what is sized around it +/// leaves room for what is drawn. fn border_width_of(d: &Dict) -> f64 { appearance::border_width(d) } @@ -703,14 +1064,66 @@ fn ink_rect_for_width(d: &Dict, old_w: f64, w: f64) -> Option<[f64; 4]> { /// Change a comment's colour, opacity (of the whole comment), line width and/or line endings, and /// redraw it. +/// +/// A Replace Text proposal is one look: colour and opacity are the group's (`/C` is read from +/// the primary, ISO 32000-2 §12.5.6.2), so restyling either the caret or its strikeout restyles +/// and redraws the other too, in the same edit. The strikeout takes no line width (a text markup +/// annotation has no border) and neither part has line endings. All or nothing: if one member +/// can't be restyled, none is. pub fn set_style(doc: &mut Document, page: usize, index: usize, change: StyleChange, meta: &Meta) -> Result<(), AnnotError> { - let StyleChange { color, opacity, width, endings } = change; let (_, r) = annot_ref(doc, page, index)?; + let others = replace_text_partners(doc, page, r)?; + // Check every partner before changing anything. + for &o in &others { + unlocked(doc, o)?; + } + let before: Vec<(ObjRef, Dict)> = others.iter().map(|&o| (o, annot_dict(doc, o))).collect(); + let mut done: Vec<(ObjRef, Dict)> = vec![(r, annot_dict(doc, r))]; + style_one(doc, r, change.clone(), meta)?; + for (o, d) in before { + let caret = d.name(b"Subtype") == Some(b"Caret"); + let part = StyleChange { color: change.color, opacity: change.opacity, width: change.width.filter(|_| caret), endings: None }; + let unchanged = part.color.is_none() && part.opacity.is_none() && part.width.is_none(); + if unchanged { + continue; + } + if let Err(e) = style_one(doc, o, part, meta) { + // Put the members already restyled back, the target included. + for (o, d) in done.into_iter().rev() { + doc.set(o, Object::Dict(d)); + } + return Err(e); + } + done.push((o, d)); + } + Ok(()) +} + +/// The other members of the Replace Text proposal `r` belongs to, on `page`: for its caret, the +/// strikeouts grouped under it; for a strikeout, its caret and any other strikeout under it. +/// Empty for anything else. +fn replace_text_partners(doc: &Document, page: usize, r: ObjRef) -> Result, AnnotError> { + let own = Object::Ref(r); + let caret = match replaced_by(doc, &own) { + Some(c) => c, + None if annot_dict(doc, r).name(b"Subtype") == Some(b"Caret") => r, + None => return Ok(Vec::new()), + }; + let list = annots(doc, page_ref(doc, page)?); + let strikes = list.iter().filter(|e| replaced_by(doc, e) == Some(caret)).filter_map(Object::as_ref); + // `/Annots` may list one annotation twice: restyle each once. + let mut seen: HashSet = HashSet::from([r]); + Ok(std::iter::once(caret).chain(strikes).filter(|&o| seen.insert(o)).collect()) +} + +/// [`set_style`] on one annotation. +fn style_one(doc: &mut Document, r: ObjRef, change: StyleChange, meta: &Meta) -> Result<(), AnnotError> { + let StyleChange { color, opacity, width, endings } = change; unlocked(doc, r)?; let d = annot_dict(doc, r); let subtype = String::from_utf8_lossy(d.name(b"Subtype").unwrap_or_default()).into_owned(); // Check before changing anything: a stale appearance would contradict the new style. - if appearance::build(doc, &d).is_none() { + if !crate::draw::can_draw(doc, &d) { return Err(AnnotError::Unsupported(subtype.clone())); } if color.is_some_and(|c| !finite(&c)) || opacity.is_some_and(|o| !o.is_finite()) || width.is_some_and(|w| !w.is_finite()) { @@ -725,13 +1138,19 @@ pub fn set_style(doc: &mut Document, page: usize, index: usize, change: StyleCha } refuse_foreign_text(doc, &d)?; let free_text = subtype == "FreeText"; + // The face and size the text is drawn in, which the new colour keeps. + let drawn = text_style::text_style(&d, text_style::dr_face(doc, &d)); // Typewriter text has no box of its own: a border goes around its text, so the box grows // (or shrinks) by the change on every side and the text keeps its place and its lines. // So does a text box inside an oval or a circle, which goes around it again. let shape = if free_text { BorderShape::of(&d) } else { BorderShape::Rectangle }; let typewriter = free_text && (d.name(b"IT") == Some(b"FreeTextTypeWriter") || shape != BorderShape::Rectangle) && !d.contains(b"CL"); + // A border style given by reference (`/BS 5 0 R`, ISO 32000-2 §7.3.10): its width and other + // entries, which a new width keeps. + let direct = crate::draw::with_direct_values(doc, &d); + let (old_width, old_bs) = (border_width_of(&direct), direct.get(b"BS").and_then(Object::as_dict).cloned()); let regrow = width.filter(|_| typewriter).and_then(|w| { - let by = w.max(0.0) - if d.contains(b"BS") || d.contains(b"Border") { border_width_of(&d) } else { 0.0 }; + let by = w.max(0.0) - if d.contains(b"BS") || d.contains(b"Border") { old_width } else { 0.0 }; let rect = grow(text_box_of(&d)?, by); (by != 0.0 && finite(&rect) && rect[2] - rect[0] >= 1.0 && rect[3] - rect[1] >= 1.0).then_some((rect, w.max(0.0))) }); @@ -742,9 +1161,7 @@ pub fn set_style(doc: &mut Document, page: usize, index: usize, change: StyleCha } if let Some(c) = color { if free_text { - let (_, size) = appearance::parse_da(d); - d.set(b"DA".to_vec(), free_text_da(c, appearance::da_border(d), size)); - rich_text(d); + restyle_text(d, TextStyle { color: c, ..drawn }); } else { d.set(b"C".to_vec(), rgb(c)); } @@ -755,9 +1172,9 @@ pub fn set_style(doc: &mut Document, page: usize, index: usize, change: StyleCha set_part_keys(d, appearance::PartOpacity { line: o, fill: o, text: o }); } if let Some(w) = width { - let old_w = border_width_of(d); + let old_w = old_width; let w = w.max(0.0); - let mut bs = d.get(b"BS").and_then(|b| b.as_dict()).cloned().unwrap_or_default(); + let mut bs = old_bs.unwrap_or_default(); bs.set(b"W".to_vec(), Object::Real(w)); d.set(b"BS".to_vec(), Object::Dict(bs)); d.remove(b"Border"); @@ -803,11 +1220,11 @@ pub fn set_border_color(doc: &mut Document, page: usize, index: usize, color: Op return Err(AnnotError::Invalid("invalid colour".into())); } refuse_foreign_text(doc, &d)?; + // The face, size and colour the text is drawn in, which the border colour keeps. + let drawn = text_style::text_style(&d, text_style::dr_face(doc, &d)); let before = d; doc.update_dict(r, |d| { - let (text, size) = appearance::parse_da(d); - d.set(b"DA".to_vec(), free_text_da(text, color, size)); - rich_text(d); + restyle_text_bordered(d, drawn, color); touch(d, meta); })?; redraw_or_restore(doc, r, before) @@ -828,11 +1245,11 @@ pub fn set_font_size(doc: &mut Document, page: usize, index: usize, size: f64, m } refuse_foreign_text(doc, &d)?; let snug = is_snug(&d); + // The face and colour the text is drawn in, which the new size keeps. + let drawn = text_style::text_style(&d, text_style::dr_face(doc, &d)); let before = d; doc.update_dict(r, |d| { - let (col, _) = appearance::parse_da(d); - d.set(b"DA".to_vec(), free_text_da(col, appearance::da_border(d), size)); - rich_text(d); + restyle_text(d, TextStyle { size, ..drawn }); touch(d, meta); })?; let text = text_value(doc, &annot_dict(doc, r), b"Contents").unwrap_or_default(); @@ -871,7 +1288,7 @@ pub fn set_fill(doc: &mut Document, page: usize, index: usize, fill: Option return Err(AnnotError::Invalid("only rectangles, ovals, polygons and text boxes can be filled".into())); } // Check before changing anything: a stale appearance would contradict the new fill. - if appearance::build(doc, &d).is_none() { + if !crate::draw::can_draw(doc, &d) { return Err(AnnotError::Unsupported(subtype)); } if fill.is_some_and(|c| !finite(&c)) { @@ -912,7 +1329,7 @@ pub fn set_part_opacity(doc: &mut Document, page: usize, index: usize, fill: Opt return Err(AnnotError::Invalid("invalid opacity".into())); } // Check before changing anything: a stale appearance would contradict the new opacity. - if appearance::build(doc, &d).is_none() { + if !crate::draw::can_draw(doc, &d) { return Err(AnnotError::Unsupported(subtype)); } refuse_foreign_text(doc, &d)?; @@ -984,11 +1401,17 @@ fn pair_points(d: &Dict, key: &[u8]) -> Option> { } /// Replace Text (Acrobat's proposal): strike out `quads` and add a caret at the end of the -/// struck text holding `replacement`, grouped with the strikeout (`/IRT` + `/RT /Group`) so they -/// move, list and delete as one. Returns the strikeout's index. +/// struck text holding `replacement`, grouped (ISO 32000-2 §12.5.6.2) so they list, select and +/// delete as one. Returns the caret's index, the group's comment. +/// +/// The caret is the group's primary: it holds `replacement` in `/Contents` and has `/IT +/// /Replace`. The strikeout is its subordinate (`/IRT` to the caret, `/RT /Group`, `/IT +/// /StrikeOutTextEdit`) with no `/Contents`, since a reader ignores a subordinate's text. +/// Groups written before this the other way round (the strikeout primary, the text only on +/// the caret) still list their text: see `pdfcraft_render::fill_group_text`. /// /// The two annotations need their own `/NM`: XFDF names a reply by the `/NM` of what it replies -/// to, so one id for both writes `inreplyto` equal to the caret's own `name`, and a reader that +/// to, so one id for both writes `inreplyto` equal to the strikeout's own `name`, and a reader that /// resolves names (Acrobat, and our own importer) loses the pair. `caret_meta` is a second id /// from the caller, so the edit stays deterministic. pub fn add_text_replacement(doc: &mut Document, replacement: TextReplacement<'_>) -> Result { @@ -1006,7 +1429,6 @@ pub fn add_text_replacement(doc: &mut Document, replacement: TextReplacement<'_> meta, )?; let (_, sr) = annot_ref(doc, page, s)?; - doc.update_dict(sr, |d| d.set(b"IT".to_vec(), Object::name("StrikeOutTextEdit")))?; // The caret sits on the baseline at the end of the last line, its point at the text. let xs = [last[0], last[2], last[4], last[6]]; let ys = [last[1], last[3], last[5], last[7]]; @@ -1021,11 +1443,16 @@ pub fn add_text_replacement(doc: &mut Document, replacement: TextReplacement<'_> )?; let (_, cr) = annot_ref(doc, page, c)?; doc.update_dict(cr, |d| { - d.set(b"IRT".to_vec(), Object::Ref(sr)); - d.set(b"RT".to_vec(), Object::name("Group")); + d.set(b"IT".to_vec(), Object::name("Replace")); d.set(b"Subj".to_vec(), PdfString::text("Replace Text")); })?; - Ok(s) + doc.update_dict(sr, |d| { + d.set(b"IT".to_vec(), Object::name("StrikeOutTextEdit")); + d.set(b"IRT".to_vec(), Object::Ref(cr)); + d.set(b"RT".to_vec(), Object::name("Group")); + d.remove(b"Contents"); + })?; + Ok(c) } /// The Eraser (Draw tools): remove the parts of drawing `(page, index)` within `radius` points diff --git a/crates/annot/src/tests.rs b/crates/annot/src/tests.rs index e14838aac8..2e52ed9260 100644 --- a/crates/annot/src/tests.rs +++ b/crates/annot/src/tests.rs @@ -213,6 +213,122 @@ fn replies_and_status_thread_onto_the_parent() { assert_eq!(text(&all[2], b"Contents"), "Accepted set by Bob"); } +/// A state reply records a state, not a note (ISO 32000-2 §12.5.6.3: the state is kept "in a +/// separate text annotation"), so it is written Hidden; a reviewer's next change replies to their +/// previous state reply ("Additional state changes shall be made by adding text annotations in +/// reply to the previous reply for a given user"). Another reviewer's state, and another state +/// model, start their own chain on the comment. +#[test] +fn status_and_checkmark_replies_are_hidden_and_chain_per_reviewer() { + let mut doc = fixture(); + let first = set_review_state(&mut doc, 0, 0, ReviewState::Accepted, "Bob", &meta("s1")).unwrap(); + let carol = set_review_state(&mut doc, 0, 0, ReviewState::Completed, "Carol", &meta("s2")).unwrap(); + let second = set_review_state(&mut doc, 0, 0, ReviewState::Rejected, "Bob", &meta("s3")).unwrap(); + let mark = set_marked(&mut doc, 0, 0, true, "Bob", &meta("m1")).unwrap(); + let unmark = set_marked(&mut doc, 0, 0, false, "Bob", &meta("m2")).unwrap(); + let doc = reopen(&doc); + let p = page_refs(&doc).unwrap()[0]; + let entries = annots(&doc, p); + let at = |i: usize| entries[i].as_ref().unwrap(); + let all = list(&doc, 0); + for i in [first, carol, second, mark, unmark] { + assert_eq!(all[i].int(b"F"), Some(30), "hidden, printable, no zoom, no rotate: {i}"); + } + assert_eq!(all[first].reference(b"IRT"), Some(at(0))); + assert_eq!(all[carol].reference(b"IRT"), Some(at(0)), "another reviewer starts on the comment"); + assert_eq!(all[second].reference(b"IRT"), Some(at(first)), "Bob's change replies to Bob's last status"); + assert_eq!(all[mark].reference(b"IRT"), Some(at(0)), "a checkmark is its own chain"); + assert_eq!(all[unmark].reference(b"IRT"), Some(at(mark))); + // A reply that isn't a state stays visible. + let mut doc = doc; + let r = add_reply(&mut doc, 0, 0, "Agreed", "Bob", &meta("r")).unwrap(); + assert_eq!(list(&doc, 0)[r].int(b"F"), Some(28)); +} + +/// Review on #4126: a history has no length limit. The 70th change still replies to the 69th, +/// and a reviewer whose name was typed with spaces continues their own chain. +#[test] +fn a_long_status_history_keeps_chaining_to_the_latest() { + let mut doc = fixture(); + let mut prev: Option = None; + for i in 0..70 { + let state = if i % 2 == 0 { ReviewState::Accepted } else { ReviewState::Rejected }; + let at = set_review_state(&mut doc, 0, 0, state, "Bob", &meta(&format!("s{i}"))).unwrap(); + if let Some(p) = prev { + let entries = annots(&doc, page_refs(&doc).unwrap()[0]); + assert_eq!(list(&doc, 0)[at].reference(b"IRT"), entries[p].as_ref(), "change {i}"); + } + prev = Some(at); + } + let padded = set_review_state(&mut doc, 0, 0, ReviewState::Completed, " Bob ", &meta("pad")).unwrap(); + let entries = annots(&doc, page_refs(&doc).unwrap()[0]); + assert_eq!(list(&doc, 0)[padded].reference(b"IRT"), prev.and_then(|p| entries[p].as_ref())); + // Listed newest first, the first reply read walks the whole chain at once. + let latest = entries[padded].as_ref(); + let p = page_refs(&doc).unwrap()[0]; + let mut reversed = entries.clone(); + reversed.reverse(); + set_annots(&mut doc, p, reversed).unwrap(); + let next = set_review_state(&mut doc, 0, 71, ReviewState::Accepted, "Bob", &meta("next")).unwrap(); + assert_eq!(list(&doc, 0)[next].reference(b"IRT"), latest); +} + +/// Review on #4126: a reply with `/StateModel` but no `/State` has the model's default state +/// (ISO 32000-2 Table 175), for the reader as for the writer: it is Bob's latest status, None, +/// and his next change replies to it. +#[test] +fn a_state_model_without_a_state_is_the_default_state() { + let mut doc = fixture(); + let bare = add_reply(&mut doc, 0, 0, "", "Bob", &meta("bare")).unwrap(); + let p = page_refs(&doc).unwrap()[0]; + let r = annots(&doc, p)[bare].as_ref().unwrap(); + doc.update_dict(r, |d| { + d.set(b"StateModel".to_vec(), PdfString::text("Review")); + d.set(b"F".to_vec(), Object::Int(30)); + }) + .unwrap(); + let summary = summaries(&doc).into_iter().find(|s| s.index == bare).unwrap(); + assert_eq!(summary.state.as_deref(), Some("None")); + let next = set_review_state(&mut doc, 0, 0, ReviewState::Accepted, "Bob", &meta("next")).unwrap(); + assert_eq!(list(&doc, 0)[next].reference(b"IRT"), Some(r)); + + // A model PdfCraft doesn't know isn't a state: a status replying to such a reply is about + // that reply, not about the comment, so Bob's next change on the comment starts afresh. + let mut doc = fixture(); + let other = add_reply(&mut doc, 0, 0, "", "Ann", &meta("other")).unwrap(); + let p = page_refs(&doc).unwrap()[0]; + let o = annots(&doc, p)[other].as_ref().unwrap(); + // Nor is a `/State` the readers can't read, such as a number (review on #4126). + doc.update_dict(o, |d| { + d.set(b"StateModel".to_vec(), PdfString::text("Other")); + d.set(b"State".to_vec(), Object::Int(5)); + }) + .unwrap(); + assert_eq!(summaries(&doc).into_iter().find(|s| s.index == other).unwrap().state, None); + set_review_state(&mut doc, 0, other, ReviewState::Rejected, "Bob", &meta("on-other")).unwrap(); + let fresh = set_review_state(&mut doc, 0, 0, ReviewState::Completed, "Bob", &meta("fresh")).unwrap(); + assert_eq!(list(&doc, 0)[fresh].reference(b"IRT"), annots(&doc, p)[0].as_ref()); +} + +/// A third change replies to the second, not the first; and Bob's status on another comment +/// is never the one his next change on this comment replies to. +#[test] +fn a_reviewers_chain_continues_from_their_latest_status_on_that_comment() { + let mut doc = fixture(); + let note = add_annotation(&mut doc, &new(0, Shape::Note { at: [100.0, 100.0], icon: NoteIcon::Comment }), &meta("n")).unwrap(); + let first = set_review_state(&mut doc, 0, 0, ReviewState::Accepted, "Bob", &meta("s1")).unwrap(); + let elsewhere = set_review_state(&mut doc, 0, note, ReviewState::Cancelled, "Bob", &meta("s2")).unwrap(); + let second = set_review_state(&mut doc, 0, 0, ReviewState::Rejected, "Bob", &meta("s3")).unwrap(); + let third = set_review_state(&mut doc, 0, 0, ReviewState::Completed, "Bob", &meta("s4")).unwrap(); + let doc = reopen(&doc); + let entries = annots(&doc, page_refs(&doc).unwrap()[0]); + let at = |i: usize| entries[i].as_ref().unwrap(); + let all = list(&doc, 0); + assert_eq!(all[elsewhere].reference(b"IRT"), Some(at(note))); + assert_eq!(all[second].reference(b"IRT"), Some(at(first)), "not Bob's status on the note"); + assert_eq!(all[third].reference(b"IRT"), Some(at(second)), "the latest, not the first"); +} + #[test] fn delete_takes_popup_and_reply_chain() { let mut doc = fixture(); @@ -769,10 +885,10 @@ fn editing_a_callout_refits_the_box_and_keeps_the_leader() { let outer = rect(d); let f = |k: &[u8]| -> Vec { d.get(k).unwrap().as_array().unwrap().iter().map(|o| o.as_f64().unwrap()).collect() }; let cl = f(b"CL"); - // The leader line keeps its start and knee; the attach point follows the re-fitted box. - assert_eq!(&cl[..4], &[200.0, 520.0, 260.0, 560.0]); + // The box grows down from its top; the side the line meets stays where it was, so the line + // doesn't change (as Acrobat keeps it while a box is resized from its other corner). let want = expected_box(612.0, 200.0, long, 12.0, 2.0); - assert_eq!((cl[4], cl[5]), callout_attach([300.0, want[1], 500.0, 612.0], [260.0, 560.0]).into()); + assert_eq!(cl, [200.0, 520.0, 260.0, 560.0, 300.0, 606.0]); // The text box (recovered from /Rect and /RD) keeps its width and top edge. let tb = [outer[0] + f(b"RD")[0], outer[1] + f(b"RD")[1], outer[2] - f(b"RD")[2], outer[3] - f(b"RD")[3]]; assert_eq!((tb[0], tb[2], tb[3]), (300.0, 500.0, 612.0)); @@ -1003,12 +1119,12 @@ fn stamps_draw_their_label_and_by_line() { #[test] fn links_are_added_edited_listed_and_removed() { - use crate::links::{self, Highlight, LinkAction, LinkStyle}; + use crate::links::{self, Highlight, LinkAction, LinkStyle, LinkZoom}; let mut doc = fixture(); let before = links::list(&doc).len(); let i = links::add(&mut doc, 0, [10.0, 10.0, 110.0, 30.0], &LinkAction::Uri("https://example.org".into()), &LinkStyle::default()).unwrap(); let style = LinkStyle { visible: true, color: [1.0, 0.0, 0.0], width: 2.0, dashed: true, underline: false, highlight: Highlight::Outline }; - links::add(&mut doc, 1, [10.0, 10.0, 60.0, 30.0], &LinkAction::Page(0), &style).unwrap(); + links::add(&mut doc, 1, [10.0, 10.0, 60.0, 30.0], &LinkAction::Page { page: 0, zoom: LinkZoom::FitPage }, &style).unwrap(); let doc = reopen(&doc); let all = links::list(&doc); assert_eq!(all.len(), before + 2); @@ -1017,7 +1133,7 @@ fn links_are_added_edited_listed_and_removed() { (web.action.clone(), web.style.visible, web.style.highlight), (LinkAction::Uri("https://example.org".into()), false, Highlight::Invert) ); - let page = all.iter().find(|l| l.page == 1 && l.action == LinkAction::Page(0)).unwrap().clone(); + let page = all.iter().find(|l| l.page == 1 && l.action == LinkAction::Page { page: 0, zoom: LinkZoom::FitPage }).unwrap().clone(); assert_eq!(page.style, style); let mut doc = doc; links::set(&mut doc, 1, page.index, Some([0.0, 0.0, 50.0, 20.0]), Some(&LinkAction::Uri("mailto:ada@example.org".into())), None).unwrap(); @@ -1027,7 +1143,201 @@ fn links_are_added_edited_listed_and_removed() { assert!(links::delete(&mut doc, 0, 999).is_err(), "no such annotation"); let n = links::remove_all(&mut doc, None).unwrap(); assert!(n >= 1 && links::list(&doc).is_empty()); - assert!(links::add(&mut doc, 0, [0.0, 0.0, 1.0, 1.0], &LinkAction::Page(0), &LinkStyle::default()).is_err(), "too small"); + assert!( + links::add(&mut doc, 0, [0.0, 0.0, 1.0, 1.0], &LinkAction::Page { page: 0, zoom: LinkZoom::FitPage }, &LinkStyle::default()).is_err(), + "too small" + ); +} + +#[test] +fn page_links_keep_their_zoom() { + use crate::links::{self, LinkAction, LinkStyle, LinkZoom}; + let page1 = page_refs(&fixture()).unwrap()[1]; + let dest_of = |doc: &Document, obj: ObjRef| doc.get(obj).as_dict().and_then(|d| d.get(b"Dest").cloned()); + // Each zoom is written as its destination type (Table 149) and reads back after a save. + let null = Object::Null; + for (zoom, written) in [ + (LinkZoom::FitPage, vec![Object::name("Fit")]), + (LinkZoom::FitWidth, vec![Object::name("FitH"), null.clone()]), + (LinkZoom::FitHeight, vec![Object::name("FitV"), null.clone()]), + (LinkZoom::FitVisible, vec![Object::name("FitB")]), + (LinkZoom::Inherit, vec![Object::name("XYZ"), null.clone(), null.clone(), null.clone()]), + (LinkZoom::Percent(100.0), vec![Object::name("XYZ"), null.clone(), null.clone(), Object::Real(1.0)]), + (LinkZoom::Percent(250.0), vec![Object::name("XYZ"), null.clone(), null.clone(), Object::Real(2.5)]), + ] { + let mut doc = fixture(); + let i = links::add(&mut doc, 0, [10.0, 10.0, 60.0, 30.0], &LinkAction::Page { page: 1, zoom }, &LinkStyle::default()).unwrap(); + let link = links::list(&doc).into_iter().find(|l| l.page == 0 && l.index == i).unwrap(); + let mut expected = vec![Object::Ref(page1)]; + expected.extend(written); + assert_eq!(dest_of(&doc, link.obj.unwrap()), Some(Object::Array(expected)), "{zoom:?}"); + let doc = reopen(&doc); + let link = links::list(&doc).into_iter().find(|l| l.page == 0 && l.index == i).unwrap(); + assert_eq!(link.action, LinkAction::Page { page: 1, zoom }, "{zoom:?} after a save"); + } + // What other writers use reads as the nearest zoom: a zero zoom keeps the reader's, the boxed + // fits as the plain ones, a rectangle as Fit Page, and a GoTo action like a /Dest. + let mut doc = fixture(); + let i = links::add(&mut doc, 0, [10.0, 10.0, 60.0, 30.0], &LinkAction::Page { page: 1, zoom: LinkZoom::FitPage }, &LinkStyle::default()).unwrap(); + let obj = links::list(&doc).into_iter().find(|l| l.page == 0 && l.index == i).unwrap().obj.unwrap(); + let dest = |kind: &str, operands: Vec| { + let mut d = vec![Object::Ref(page1), Object::name(kind)]; + d.extend(operands); + Object::Array(d) + }; + for (key, value, zoom) in [ + ("Dest", dest("XYZ", vec![Object::Int(100), Object::Int(700), Object::Int(0)]), LinkZoom::Inherit), + ("Dest", dest("FitBH", vec![Object::Int(500)]), LinkZoom::FitWidth), + ("Dest", dest("FitBV", vec![Object::Int(20)]), LinkZoom::FitHeight), + ("Dest", dest("FitR", vec![Object::Int(0), Object::Int(0), Object::Int(10), Object::Int(10)]), LinkZoom::FitPage), + ("A", dest("XYZ", vec![null.clone(), null.clone(), Object::Real(1.5)]), LinkZoom::Percent(150.0)), + // A factor outside what PdfCraft writes reads as the destination has it, not capped, so + // it agrees with the viewer's `destination`; one whose percentage overflows stays finite. + ("Dest", dest("XYZ", vec![null.clone(), null.clone(), Object::Real(100.0)]), LinkZoom::Percent(10000.0)), + ("Dest", dest("XYZ", vec![null.clone(), null.clone(), Object::Real(0.005)]), LinkZoom::Percent(0.5)), + ("Dest", dest("XYZ", vec![null.clone(), null.clone(), Object::Real(1e307)]), LinkZoom::Percent(f64::MAX)), + ] { + let mut d = doc.get(obj).as_dict().cloned().unwrap(); + d.remove(b"Dest"); + d.remove(b"A"); + if key == "A" { + let mut go_to = pdfcraft_cos::Dict::new(); + go_to.set(b"S".to_vec(), Object::name("GoTo")); + go_to.set(b"D".to_vec(), value); + d.set(b"A".to_vec(), Object::Dict(go_to)); + } else { + d.set(b"Dest".to_vec(), value); + } + doc.set(obj, Object::Dict(d)); + let link = links::list(&doc).into_iter().find(|l| l.page == 0 && l.index == i).unwrap(); + assert_eq!(link.action, LinkAction::Page { page: 1, zoom }, "{key}"); + } + // A zoom that isn't a positive percentage is refused before anything is written. + for p in [0.0, -50.0, f64::NAN, f64::INFINITY] { + let before = links::list(&doc).len(); + let refused = + links::add(&mut doc, 0, [10.0, 10.0, 60.0, 30.0], &LinkAction::Page { page: 1, zoom: LinkZoom::Percent(p) }, &LinkStyle::default()); + assert!(refused.is_err(), "{p}"); + assert_eq!(links::list(&doc).len(), before, "{p}: no link added"); + } +} + +/// A link rewritten with the zoom it has keeps its own `/XYZ` factor, however far outside the +/// 1–6400% a new zoom is written within: giving it a new page (Link Properties' OK, `link_edit` +/// without `zoom`) must not change its zoom. A zoom chosen anew is still written within 1–6400%. +#[test] +fn a_rewritten_link_keeps_its_own_zoom_factor() { + use crate::links::{self, LinkAction, LinkStyle, LinkZoom}; + let pages = page_refs(&fixture()).unwrap(); + let dest_of = |doc: &Document, obj: ObjRef| doc.get(obj).as_dict().and_then(|d| d.get(b"Dest").cloned()); + let xyz = |page: ObjRef, factor: Object| Object::Array(vec![Object::Ref(page), Object::name("XYZ"), Object::Null, Object::Null, factor]); + for (key, factor) in [ + ("Dest", Object::Real(0.005)), + ("Dest", Object::Real(100.0)), + ("Dest", Object::Int(100)), + ("Dest", Object::Real(1e307)), + ("A", Object::Real(0.005)), + ] { + let mut doc = fixture(); + let i = + links::add(&mut doc, 0, [10.0, 10.0, 60.0, 30.0], &LinkAction::Page { page: 1, zoom: LinkZoom::FitPage }, &LinkStyle::default()).unwrap(); + let obj = links::list(&doc).into_iter().find(|l| l.page == 0 && l.index == i).unwrap().obj.unwrap(); + let mut d = doc.get(obj).as_dict().cloned().unwrap(); + d.remove(b"Dest"); + if key == "A" { + let mut go_to = pdfcraft_cos::Dict::new(); + go_to.set(b"S".to_vec(), Object::name("GoTo")); + go_to.set(b"D".to_vec(), xyz(pages[1], factor.clone())); + d.set(b"A".to_vec(), Object::Dict(go_to)); + } else { + d.set(b"Dest".to_vec(), xyz(pages[1], factor.clone())); + } + doc.set(obj, Object::Dict(d)); + let LinkAction::Page { zoom, .. } = links::list(&doc).into_iter().find(|l| l.page == 0 && l.index == i).unwrap().action else { + panic!("{key} {factor:?}") + }; + links::set(&mut doc, 0, i, None, Some(&LinkAction::Page { page: 0, zoom }), None).unwrap(); + assert_eq!(dest_of(&doc, obj), Some(xyz(pages[0], factor.clone())), "{key} {factor:?}: the factor is the link's own"); + let doc = reopen(&doc); + assert_eq!( + links::list(&doc).into_iter().find(|l| l.page == 0 && l.index == i).unwrap().action, + LinkAction::Page { page: 0, zoom }, + "{key} {factor:?}" + ); + } + // A zoom other than the link's is written within 1–6400%. + let mut doc = fixture(); + let i = links::add(&mut doc, 0, [10.0, 10.0, 60.0, 30.0], &LinkAction::Page { page: 1, zoom: LinkZoom::Percent(50.0) }, &LinkStyle::default()) + .unwrap(); + let obj = links::list(&doc).into_iter().find(|l| l.page == 0 && l.index == i).unwrap().obj.unwrap(); + links::set(&mut doc, 0, i, None, Some(&LinkAction::Page { page: 1, zoom: LinkZoom::Percent(10000.0) }), None).unwrap(); + assert_eq!(dest_of(&doc, obj), Some(xyz(pages[1], Object::Real(64.0)))); +} + +/// A go-to edited without a change (its own page and zoom, as `link_edit` naming its page does) +/// is left as it is: the `/XYZ` position, a fit's position and the GoTo form stay. Given a new +/// page with its zoom, a link keeps how that zoom is written, a boxed fit (`/FitBH`, `/FitBV`) +/// included, but not its position, which belongs to the old page. +#[test] +fn an_unchanged_go_to_keeps_its_destination_and_a_new_page_keeps_its_fit() { + use crate::links::{self, LinkAction, LinkStyle, LinkZoom}; + let pages = page_refs(&fixture()).unwrap(); + let dest = |page: ObjRef, kind: &str, operands: Vec| { + let mut d = vec![Object::Ref(page), Object::name(kind)]; + d.extend(operands); + Object::Array(d) + }; + let link_with = |key: &str, value: Object| { + let mut doc = fixture(); + let i = + links::add(&mut doc, 0, [10.0, 10.0, 60.0, 30.0], &LinkAction::Page { page: 1, zoom: LinkZoom::FitPage }, &LinkStyle::default()).unwrap(); + let obj = links::list(&doc).into_iter().find(|l| l.page == 0 && l.index == i).unwrap().obj.unwrap(); + let mut d = doc.get(obj).as_dict().cloned().unwrap(); + d.remove(b"Dest"); + if key == "A" { + let mut go_to = pdfcraft_cos::Dict::new(); + go_to.set(b"S".to_vec(), Object::name("GoTo")); + go_to.set(b"D".to_vec(), value); + d.set(b"A".to_vec(), Object::Dict(go_to)); + } else { + d.set(b"Dest".to_vec(), value); + } + doc.set(obj, Object::Dict(d)); + (doc, i, obj) + }; + let (num, null) = (|n: i64| Object::Int(n), Object::Null); + // The same page and zoom: nothing is rewritten. + for (key, value) in [ + ("Dest", dest(pages[1], "XYZ", vec![num(72), num(500), Object::Real(2.0)])), + ("Dest", dest(pages[1], "FitBH", vec![num(250)])), + ("Dest", dest(pages[1], "FitV", vec![num(20)])), + ("A", dest(pages[1], "XYZ", vec![num(72), num(500), Object::Real(0.005)])), + ] { + let (mut doc, i, obj) = link_with(key, value); + let before = doc.get(obj).as_dict().cloned(); + let LinkAction::Page { page, zoom } = links::list(&doc).into_iter().find(|l| l.page == 0 && l.index == i).unwrap().action else { + panic!("{key}") + }; + links::set(&mut doc, 0, i, None, Some(&LinkAction::Page { page, zoom }), None).unwrap(); + assert_eq!(doc.get(obj).as_dict().cloned(), before, "{key} {zoom:?}: left as it is"); + } + // A new page: the fit's kind and the factor stay, positions go. + for (value, written) in [ + (dest(pages[1], "FitBH", vec![num(250)]), dest(pages[0], "FitBH", vec![null.clone()])), + (dest(pages[1], "FitBV", vec![num(20)]), dest(pages[0], "FitBV", vec![null.clone()])), + (dest(pages[1], "FitH", vec![num(250)]), dest(pages[0], "FitH", vec![null.clone()])), + ( + dest(pages[1], "XYZ", vec![num(72), num(500), Object::Real(2.0)]), + dest(pages[0], "XYZ", vec![null.clone(), null.clone(), Object::Real(2.0)]), + ), + ] { + let (mut doc, i, obj) = link_with("Dest", value.clone()); + let LinkAction::Page { zoom, .. } = links::list(&doc).into_iter().find(|l| l.page == 0 && l.index == i).unwrap().action else { + panic!("{value:?}") + }; + links::set(&mut doc, 0, i, None, Some(&LinkAction::Page { page: 0, zoom }), None).unwrap(); + assert_eq!(doc.get(obj).as_dict().and_then(|d| d.get(b"Dest").cloned()), Some(written), "{value:?}"); + } } #[test] @@ -1120,6 +1430,44 @@ fn urls_are_found_after_length_changing_lowercase() { assert_eq!(range, 21..39, "range is in source-character indices"); } +/// Review on #4472: an address inside one written with another scheme isn't taken out of it +/// (`ftp://www.example.com` opened as `http://www.example.com`), and brackets that pair up +/// inside an address belong to it. +#[test] +fn urls_keep_their_written_scheme_and_paired_brackets() { + let found = |s: &str| -> Vec { crate::links::find_urls(&s.chars().collect::>()).into_iter().map(|(_, u)| u).collect() }; + for s in + ["ftp://www.example.com/pub", "file://www.example.com/x", "javascript:www.example.com", "ftp://user@example.com/x", "smb://a@example.org/s"] + { + assert_eq!(found(s), Vec::::new(), "{s}"); + } + assert_eq!(found("mailto:someone@example.com and (www.example.com)"), ["mailto:someone@example.com", "http://www.example.com"]); + assert_eq!(found("See https://en.wikipedia.org/wiki/Mercury_(planet)."), ["https://en.wikipedia.org/wiki/Mercury_(planet)"]); + assert_eq!(found("(https://example.com/a_(b)_c)"), ["https://example.com/a_(b)_c"]); + assert_eq!(found("https://example.com/a(b c"), ["https://example.com/a"]); + // Deeply nested or unbalanced brackets end the address without trouble. + let deep = format!("https://example.com/{}x{}", "(".repeat(10_000), ")".repeat(9_999)); + assert_eq!(found(&deep), ["https://example.com/"]); +} + +/// Review on #4472: brackets that never close in text without spaces (`https://a/(` again and +/// again) made each address in it read the rest of the text, then start again one character on, +/// a pass over the page per address. Each is read 2,048 characters at most now, and the +/// addresses around them are still found. A run of address characters longer than that is no +/// address. +#[test] +fn unclosed_brackets_in_text_without_spaces_are_read_once_per_address() { + let found = |s: &str| -> Vec { crate::links::find_urls(&s.chars().collect::>()).into_iter().map(|(_, u)| u).collect() }; + let text = format!("www.before.org {} www.after.org", "https://a/(".repeat(60_000)); + let started = std::time::Instant::now(); + assert_eq!(found(&text), ["http://www.before.org", "http://www.after.org"]); + assert!(started.elapsed().as_secs() < 5, "{:?}", started.elapsed()); + let long = format!("https://example.com/{} www.after.org", "a".repeat(3000)); + assert_eq!(found(&long), ["http://www.after.org"]); + let fits = format!("https://example.com/{}", "a".repeat(2000)); + assert_eq!(found(&fits), std::slice::from_ref(&fits)); +} + #[test] fn url_scheme_and_www_match_case_insensitively() { let t: Vec = "HTTP://Example.COM".chars().collect(); @@ -1261,25 +1609,46 @@ fn replace_text_groups_a_strikeout_and_a_caret() { .unwrap(); let doc = reopen(&doc); let a = list(&doc, 0); - let strike = &a[i]; - assert_eq!((strike.name(b"Subtype"), strike.name(b"IT")), (Some(&b"StrikeOut"[..]), Some(&b"StrikeOutTextEdit"[..]))); - let caret = a.iter().find(|d| d.name(b"Subtype") == Some(b"Caret")).unwrap(); - assert_eq!((text(caret, b"Contents").as_str(), caret.name(b"RT")), ("new words", Some(&b"Group"[..]))); - assert!(caret.reference(b"IRT").is_some()); + // The caret is the group's primary (ISO 32000-2 §12.5.6.2): it holds the text, has no + // `/IRT`, and is what the call returns. + let caret = &a[i]; + assert_eq!((caret.name(b"Subtype"), caret.name(b"IT")), (Some(&b"Caret"[..]), Some(&b"Replace"[..]))); + assert_eq!(text(caret, b"Contents").as_str(), "new words"); + assert!(caret.get(b"IRT").is_none() && caret.get(b"RT").is_none()); + // The strikeout is its subordinate, with no text of its own. + let strike = a.iter().find(|d| d.name(b"Subtype") == Some(b"StrikeOut")).unwrap(); + assert_eq!((strike.name(b"IT"), strike.name(b"RT")), (Some(&b"StrikeOutTextEdit"[..]), Some(&b"Group"[..]))); + assert!(strike.get(b"Contents").is_none()); + assert!(is_group_subordinate(strike)); // #2666: the pair needs its own `/NM` each. XFDF names a reply by the `/NM` of what it - // replies to, so sharing one id writes `inreplyto` equal to the caret's own `name` and the - // pair does not survive an export and import. + // replies to, so sharing one id writes `inreplyto` equal to the subordinate's own `name` + // and the pair does not survive an export and import. assert_eq!(text(caret, b"NM").as_str(), "r-caret"); assert_eq!(text(strike, b"NM").as_str(), "r"); - assert_ne!(text(caret, b"NM"), text(strike, b"NM"), "the caret must not carry the strikeout's name"); - let irt = caret.reference(b"IRT").unwrap(); - assert_eq!(text(doc.get(irt).as_dict().unwrap(), b"NM").as_str(), "r", "the caret replies to the strikeout"); + let irt = strike.reference(b"IRT").unwrap(); + assert_eq!(text(doc.get(irt).as_dict().unwrap(), b"NM").as_str(), "r-caret", "the strikeout is grouped with the caret"); let r = rect(caret); assert!(r[0] < 200.0 && r[2] > 200.0 && r[1] < 706.0, "at the end of the line, below its baseline: {r:?}"); - // Deleting the strikeout takes the caret with it (the group). + let summary = summaries(&doc); + assert_eq!(summary.iter().filter(|s| s.page == 0 && !s.is_group_member()).count(), 2, "the fixture's Square and the caret: {summary:?}"); + // Replace Text is one comment: deleting either the caret or the strikeout deletes both, as + // Acrobat does. + for member in [i, a.iter().position(|d| d.name(b"Subtype") == Some(b"StrikeOut")).unwrap()] { + let mut doc = doc.clone(); + delete_annotation(&mut doc, 0, member).unwrap(); + assert!(!list(&doc, 0).iter().any(|d| matches!(d.name(b"Subtype"), Some(b"Caret" | b"StrikeOut"))), "{member}"); + } + let marks = |doc: &Document| list(doc, 0).iter().filter(|d| matches!(d.name(b"Subtype"), Some(b"Caret" | b"StrikeOut"))).count(); + let si = a.iter().position(|d| d.name(b"Subtype") == Some(b"StrikeOut")).unwrap(); let mut doc = doc; - delete_annotation(&mut doc, 0, i).unwrap(); - assert!(!list(&doc, 0).iter().any(|d| d.name(b"Subtype") == Some(b"Caret"))); + let c = list(&doc, 0).iter().position(|d| d.name(b"Subtype") == Some(b"Caret")).unwrap(); + // A locked caret keeps the strikeout too: the pair can't be broken by deleting the other. + let mut locked = doc.clone(); + set_locked(&mut locked, 0, c, true).unwrap(); + assert!(deletable(&locked, 0, si).is_err() && delete_annotation(&mut locked, 0, si).is_err(), "a locked caret refuses its strikeout's deletion"); + assert_eq!(marks(&locked), 2); + delete_annotation(&mut doc, 0, c).unwrap(); + assert_eq!(marks(&doc), 0, "deleting the caret deletes the strikeout"); assert!( add_text_replacement( &mut doc, @@ -1298,6 +1667,268 @@ fn replace_text_groups_a_strikeout_and_a_caret() { ); } +/// ISO 32000-2 §12.5.6.2: `/RT /Group` makes an annotation a subordinate of the group its +/// `/IRT` annotation is primary of. Another `/RT` reads as a reply, and so does a bare `/IRT`. +/// Deleting a member deletes just it, as Acrobat does, although §12.5.6.2 lists deletion among the +/// operations that act on the entire group: the subordinate goes alone, and the primary goes with +/// its replies and leaves its subordinate, which reads as a comment of its own. +#[test] +fn a_group_reads_as_one_and_deletes_by_member() { + let mut doc = fixture(); + let square = add_annotation(&mut doc, &new(0, Shape::Rectangle { rect: [100.0, 100.0, 200.0, 200.0] }), &meta("sq")).unwrap(); + let (_, sr) = annot_ref(&mut doc, 0, square).unwrap(); + let irt = |doc: &mut Document, id: &str, rt: Option<&str>| { + let i = add_annotation(doc, &new(0, Shape::Oval { rect: [300.0, 300.0, 350.0, 350.0] }), &meta(id)).unwrap(); + let (_, r) = annot_ref(doc, 0, i).unwrap(); + doc.update_dict(r, |d| { + d.set(b"IRT".to_vec(), Object::Ref(sr)); + if let Some(rt) = rt { + d.set(b"RT".to_vec(), Object::name(rt)); + } + }) + .unwrap(); + i + }; + let member = irt(&mut doc, "member", Some("Group")); + irt(&mut doc, "reply", None); + irt(&mut doc, "explicit", Some("R")); + irt(&mut doc, "odd", Some("Bogus")); + let doc = reopen(&doc); + let grouped: Vec<(String, bool)> = + summaries(&doc).into_iter().filter(|s| s.in_reply_to.is_some()).map(|s| (s.name.clone().unwrap_or_default(), s.is_group_member())).collect(); + assert_eq!( + grouped, + [("member".into(), true), ("reply".into(), false), ("explicit".into(), false), ("odd".into(), false)], + "only /RT /Group groups" + ); + let mut alone = doc.clone(); + delete_annotation(&mut alone, 0, member).unwrap(); + assert_eq!(list(&alone, 0).len(), list(&doc, 0).len() - 1, "the subordinate goes alone"); + let mut doc = doc; + delete_annotation(&mut doc, 0, square).unwrap(); + let left: Vec> = list(&doc, 0).iter().map(|d| d.get(b"NM").map(|_| text(d, b"NM"))).collect(); + assert_eq!(left, [None, Some("member".into())], "the primary and its replies are gone; the fixture's Square and the subordinate stay"); + let summary = summaries(&doc); + let member = summary.iter().find(|s| s.name.as_deref() == Some("member")).unwrap(); + assert!(member.parent.is_none() && !member.is_group_member(), "left without its primary, it is a comment of its own: {member:?}"); +} + +/// A primary with one subordinate and one reply, all on page 0 of a reopened file. Returns the +/// primary's and the subordinate's `/Annots` positions and references. +fn group_of_two(text: &str) -> (Document, (usize, ObjRef), (usize, ObjRef)) { + let mut doc = fixture(); + let primary = add_annotation(&mut doc, &new(0, Shape::Rectangle { rect: [100.0, 100.0, 200.0, 200.0] }), &meta("primary")).unwrap(); + let (_, pr) = annot_ref(&mut doc, 0, primary).unwrap(); + doc.update_dict(pr, |d| d.set(b"Contents".to_vec(), pdfcraft_cos::PdfString::text(text))).unwrap(); + let member = add_annotation(&mut doc, &new(0, Shape::Oval { rect: [300.0, 300.0, 350.0, 350.0] }), &meta("member")).unwrap(); + let (_, mr) = annot_ref(&mut doc, 0, member).unwrap(); + doc.update_dict(mr, |d| { + d.set(b"IRT".to_vec(), Object::Ref(pr)); + d.set(b"RT".to_vec(), Object::name("Group")); + }) + .unwrap(); + add_reply(&mut doc, 0, primary, "a reply", "Ada", &meta("reply")).unwrap(); + (reopen(&doc), (primary, pr), (member, mr)) +} + +/// Deleting a group's primary leaves its subordinate a comment of its own, as Acrobat does: the +/// subordinate loses its `/IRT` and keeps `/RT /Group`, so nothing refers to the deleted primary +/// and a full save leaves its text out. +#[test] +fn deleting_a_primary_releases_its_subordinate() { + let (mut doc, (primary, _), (_, mr)) = group_of_two("primary secret text"); + delete_annotation(&mut doc, 0, primary).unwrap(); + let member = doc.get(mr).as_dict().cloned().unwrap(); + assert!(member.get(b"IRT").is_none(), "the subordinate no longer names the deleted primary: {member:?}"); + assert_eq!(member.name(b"RT"), Some(&b"Group"[..]), "its /RT stays, as Acrobat leaves it"); + let s = summaries(&doc); + let listed = s.iter().find(|s| s.name.as_deref() == Some("member")).unwrap(); + assert!(listed.parent.is_none() && listed.in_reply_to.is_none() && !listed.is_group_member(), "a comment of its own: {listed:?}"); + assert!(!s.iter().any(|s| s.name.as_deref() == Some("reply")), "the primary's reply went with it"); + let full = pdfcraft_cos::write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap(); + assert!(!full.windows(19).any(|w| w == b"primary secret text"), "the deleted primary's text is not in a full save"); +} + +/// Deleting a subordinate leaves its primary as it was. +#[test] +fn deleting_a_subordinate_leaves_its_primary_untouched() { + let (mut doc, (_, pr), (member, _)) = group_of_two("primary text"); + let before = doc.get(pr).as_dict().cloned(); + delete_annotation(&mut doc, 0, member).unwrap(); + assert_eq!(doc.get(pr).as_dict().cloned(), before); + assert!(summaries(&doc).iter().any(|s| s.name.as_deref() == Some("reply")), "the primary keeps its reply"); +} + +/// An inline subordinate (`/Annots` may hold a dictionary, though the standard asks for a +/// reference) whose primary is deleted loses its `/IRT` in the page's list. +#[test] +fn an_inline_subordinate_is_released_too() { + let mut doc = fixture(); + let primary = add_annotation(&mut doc, &new(0, Shape::Rectangle { rect: [100.0, 100.0, 200.0, 200.0] }), &meta("primary")).unwrap(); + let (_, pr) = annot_ref(&mut doc, 0, primary).unwrap(); + let p = page_refs(&doc).unwrap()[0]; + let mut entries = annots(&doc, p); + let mut inline = Dict::new(); + inline.set(b"Type".to_vec(), Object::name("Annot")); + inline.set(b"Subtype".to_vec(), Object::name("Square")); + inline.set(b"Rect".to_vec(), Object::Array([300, 300, 350, 350].map(Object::Int).to_vec())); + inline.set(b"IRT".to_vec(), Object::Ref(pr)); + inline.set(b"RT".to_vec(), Object::name("Group")); + entries.push(Object::Dict(inline)); + set_annots(&mut doc, p, entries).unwrap(); + delete_annotation(&mut doc, 0, primary).unwrap(); + let left = list(&doc, 0); + let inline = left.iter().find(|d| d.name(b"RT") == Some(b"Group")).expect("the inline subordinate stays"); + assert!(inline.get(b"IRT").is_none()); +} + +/// Deleting follows `/IRT` replies by visiting each annotation once, with no step limit to cut a +/// thread short: a thread of 200 replies (past the 64 steps the listings follow) goes whole, and +/// replies or group members in a loop (`/IRT` back to each other) end the walk. A loop of group +/// members deletes only the one picked; the other is released. +#[test] +fn deleting_follows_a_long_or_looping_irt_chain_once_each() { + let mut doc = fixture(); + let top = add_annotation(&mut doc, &new(0, Shape::Rectangle { rect: [100.0, 100.0, 200.0, 200.0] }), &meta("top")).unwrap(); + let mut on = top; + for k in 0..200 { + on = add_reply(&mut doc, 0, on, "reply", "Ada", &meta(&format!("r{k}"))).unwrap(); + } + let before = list(&doc, 0).len(); + delete_annotation(&mut doc, 0, top).unwrap(); + assert_eq!(list(&doc, 0).len(), before - 201, "the comment and its 200-deep thread go together"); + for rt in [None, Some("Group")] { + let mut doc = fixture(); + let a = add_annotation(&mut doc, &new(0, Shape::Rectangle { rect: [100.0, 100.0, 200.0, 200.0] }), &meta("a")).unwrap(); + let b = add_annotation(&mut doc, &new(0, Shape::Oval { rect: [300.0, 300.0, 350.0, 350.0] }), &meta("b")).unwrap(); + let (_, ar) = annot_ref(&mut doc, 0, a).unwrap(); + let (_, br) = annot_ref(&mut doc, 0, b).unwrap(); + for (r, to) in [(ar, br), (br, ar)] { + doc.update_dict(r, |d| { + d.set(b"IRT".to_vec(), Object::Ref(to)); + if let Some(rt) = rt { + d.set(b"RT".to_vec(), Object::name(rt)); + } + }) + .unwrap(); + } + let before = list(&doc, 0).len(); + delete_annotation(&mut doc, 0, a).unwrap(); + let left = list(&doc, 0); + match rt { + None => assert_eq!(left.len(), before - 2, "a reply loop goes whole"), + Some(_) => { + assert_eq!(left.len(), before - 1, "a group member goes alone"); + assert!(doc.get(br).as_dict().is_some_and(|d| d.get(b"IRT").is_none()), "the other member is released"); + } + } + } +} + +/// A subordinate deep in a chain of subordinates, within the 64 annotations the listing follows +/// up a chain or past them, is deleted alone: the chain above it stays. +#[test] +fn a_subordinate_deep_in_a_chain_is_deleted_alone() { + for subordinates in [63, 64, 65] { + let mut doc = fixture(); + let top = add_annotation(&mut doc, &new(0, Shape::Rectangle { rect: [100.0, 100.0, 200.0, 200.0] }), &meta("top")).unwrap(); + let (_, mut up) = annot_ref(&mut doc, 0, top).unwrap(); + let mut last = top; + for k in 0..subordinates { + last = add_annotation(&mut doc, &new(0, Shape::Oval { rect: [300.0, 300.0, 350.0, 350.0] }), &meta(&format!("s{k}"))).unwrap(); + let (_, r) = annot_ref(&mut doc, 0, last).unwrap(); + doc.update_dict(r, |d| { + d.set(b"IRT".to_vec(), Object::Ref(up)); + d.set(b"RT".to_vec(), Object::name("Group")); + }) + .unwrap(); + up = r; + } + let before = list(&doc, 0).len(); + delete_annotation(&mut doc, 0, last).unwrap(); + assert_eq!(list(&doc, 0).len(), before - 1, "{subordinates}: the subordinate is gone, the rest of its group stays"); + } +} + +/// A locked member doesn't hold back the deletion of another, the primary included: each goes +/// alone (ISO 32000-2 §12.5.3, Table 167: Locked). +#[test] +fn a_locked_group_member_holds_back_no_other_deletion() { + let mut doc = fixture(); + let top = add_annotation(&mut doc, &new(0, Shape::Rectangle { rect: [100.0, 100.0, 200.0, 200.0] }), &meta("top")).unwrap(); + let (_, tr) = annot_ref(&mut doc, 0, top).unwrap(); + let mut members = Vec::new(); + for id in ["a", "b"] { + let i = add_annotation(&mut doc, &new(0, Shape::Oval { rect: [300.0, 300.0, 350.0, 350.0] }), &meta(id)).unwrap(); + let (_, r) = annot_ref(&mut doc, 0, i).unwrap(); + doc.update_dict(r, |d| { + d.set(b"IRT".to_vec(), Object::Ref(tr)); + d.set(b"RT".to_vec(), Object::name("Group")); + }) + .unwrap(); + members.push(i); + } + set_locked(&mut doc, 0, members[1], true).unwrap(); + let before = list(&doc, 0).len(); + let mut alone = doc.clone(); + delete_annotation(&mut alone, 0, members[0]).unwrap(); + assert_eq!(list(&alone, 0).len(), before - 1, "the unlocked subordinate goes alone"); + delete_annotation(&mut doc, 0, top).unwrap(); + assert_eq!(list(&doc, 0).len(), before - 1, "the primary goes alone too"); + let locked = list(&doc, 0).iter().filter(|d| d.get(b"F").and_then(|f| f.as_int()).is_some_and(|f| f & 128 != 0)).count(); + assert_eq!(locked, 1, "the locked member stays"); +} + +/// Review on #5689 (CodeRabbit): clicking a Replace Text strikeout selects its caret, the +/// group's primary, so a restyle from Comment Properties lands on the caret. Colour and opacity +/// are the proposal's, so the strikeout is restyled and redrawn in the same edit, whichever +/// member is restyled; it takes no border width. A locked member refuses the whole change. +#[test] +fn restyling_a_replace_text_pair_restyles_both_members() { + let mut doc = fixture(); + let quads = [[100.0, 720.0, 200.0, 720.0, 100.0, 706.0, 200.0, 706.0]]; + let red = Style { color: [0.89, 0.13, 0.13], opacity: 1.0, width: 1.0, fill: None, ..Style::default() }; + let tr = TextReplacement { + page: 0, + quads: &quads, + replacement: "new words", + author: "Ada", + strike: &red, + caret: &red, + strike_meta: &meta("r"), + caret_meta: &meta("r-caret"), + }; + // The call returns the caret, the group's primary. + let c = add_text_replacement(&mut doc, tr).unwrap(); + let s = list(&doc, 0).iter().position(|d| d.name(b"Subtype") == Some(b"StrikeOut")).unwrap(); + let color = |doc: &Document, i: usize| { + list(doc, 0)[i].get(b"C").and_then(|o| o.as_array().cloned()).unwrap().iter().filter_map(Object::as_f64).collect::>() + }; + let green = [0.0, 0.5, 0.0]; + + // Restyled through the caret (what a click on either part selects). + let mut by_caret = doc.clone(); + set_style(&mut by_caret, 0, c, StyleChange::new(Some(green), Some(0.5), Some(3.0), None), &meta("")).unwrap(); + let by_caret = reopen(&by_caret); + assert_eq!(color(&by_caret, c), green.to_vec()); + assert_eq!(color(&by_caret, s), green.to_vec(), "the strikeout takes the proposal's colour"); + let strike = &list(&by_caret, 0)[s]; + assert_eq!(strike.get(b"CA").and_then(Object::as_f64), Some(0.5), "and its opacity"); + assert!(strike.get(b"BS").is_none(), "a strikeout has no border to widen: {strike:?}"); + assert!(ap_content(&by_caret, strike).contains("0 0.5 0 RG"), "its appearance is redrawn: {}", ap_content(&by_caret, strike)); + + // Restyled through the strikeout: the caret follows. + let mut by_strike = doc.clone(); + set_style(&mut by_strike, 0, s, StyleChange::new(Some(green), None, None, None), &meta("")).unwrap(); + assert_eq!(color(&by_strike, c), green.to_vec(), "the caret takes the proposal's colour"); + + // A locked strikeout refuses the change, and nothing moves. + let mut locked = doc.clone(); + set_locked(&mut locked, 0, s, true).unwrap(); + assert!(set_style(&mut locked, 0, c, StyleChange::new(Some(green), None, None, None), &meta("")).is_err()); + assert_eq!(color(&locked, c), red.color.to_vec(), "the caret keeps its colour"); +} + #[test] fn files_attach_as_comments() { let mut doc = fixture(); @@ -1467,6 +2098,190 @@ fn malformed_optional_interior_color_does_not_hide_shape_strokes() { assert!(list(&doc, 0)[0].get(b"AP").is_none(), "display generation leaves the source document unchanged"); } +/// Any value may be given by reference (ISO 32000-2 §7.3.10). A Square whose border style +/// dictionary (Table 168), colour and dash array are indirect objects is drawn with them, not +/// with the 1 pt solid default; a reference cycle inside `/BS` is not followed for ever. +#[test] +fn indirect_border_style_and_colour_are_drawn() { + let doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 200 200] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R] >>", + b"<< /Type /Annot /Subtype /Square /P 3 0 R /Rect [20 20 180 180] /BS 5 0 R /C 6 0 R >>", + b"<< /Type /Border /W 7 0 R /S /D /D 8 0 R /Self 5 0 R >>", + b"[0 0 1]", + b"6", + b"[4 2]", + ]); + let display = with_missing_appearances(&doc).expect("the square gets a display appearance"); + let shown = list(&display, 0).remove(0); + let content = ap_content(&display, &shown); + assert!(content.contains("0 0 1 RG\n6 w\n[4 2] 0 d\n"), "the indirect /BS, /W, /D and /C are used: {content}"); + assert!(content.contains("23 23 154 154 re"), "the 6 pt border is inset by half its width: {content}"); + assert_eq!(shown.get(b"BS"), Some(&Object::Ref(ObjRef::new(5, 0))), "the display copy keeps the reference"); + assert_eq!(shown.get(b"P"), Some(&Object::Ref(ObjRef::new(3, 0)))); + let props = props(&doc, 0, 0).expect("props"); + assert_eq!(props.width, Some(6.0), "the properties show the width drawn"); +} + +/// A followed reference the builder can't draw (a cloudy border effect on a square, which it +/// leaves alone) keeps the drawing made without it instead of none. +#[test] +fn an_undrawable_indirect_value_keeps_the_plain_drawing() { + let doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 200 200] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R] >>", + b"<< /Type /Annot /Subtype /Square /Rect [20 20 180 180] /BE 5 0 R /C [1 0 0] >>", + b"<< /S /C /I 1 >>", + ]); + let display = with_missing_appearances(&doc).expect("the square is still drawn"); + let shown = list(&display, 0).remove(0); + assert!(ap_content(&display, &shown).contains("1 0 0 RG\n1 w\n")); +} + +/// Following references copies a bounded number of values: an indirect array past the budget +/// stays a reference (and the comment is not drawn from it), a chain that never ends is not +/// followed, and links to other objects are never copied. +#[test] +fn direct_values_are_bounded() { + let mut doc = fixture(); + let huge = doc.add(Object::Array(vec![Object::Int(0); (1 << 16) + 1])); + let chain = doc.add(Object::Null); + doc.set(chain, Object::Ref(chain)); + let mut d = Dict::new(); + d.set(b"Subtype".to_vec(), Object::name("Square")); + d.set(b"Rect".to_vec(), Object::Ref(doc.add(num_array(&[0.0, 0.0, 10.0, 10.0])))); + d.set(b"Vertices".to_vec(), Object::Ref(huge)); + d.set(b"IC".to_vec(), Object::Ref(chain)); + d.set(b"Popup".to_vec(), Object::Ref(huge)); + let direct = crate::draw::with_direct_values(&doc, &d); + assert_eq!(direct.get(b"Rect").and_then(Object::as_array).map(Vec::len), Some(4), "a small array is followed"); + assert_eq!(direct.get(b"Vertices"), Some(&Object::Ref(huge)), "over the budget: left as a reference"); + assert_eq!(direct.get(b"IC"), Some(&Object::Ref(chain)), "a chain of references is left alone"); + assert_eq!(direct.get(b"Popup"), Some(&Object::Ref(huge)), "a link is not copied"); + let mut plain = Dict::new(); + plain.set(b"W".to_vec(), Object::Int(2)); + assert!(matches!(crate::draw::with_direct_values(&doc, &plain), std::borrow::Cow::Borrowed(_)), "nothing to follow: no copy"); +} + +/// A new width for a comment whose border style is an indirect object (§7.3.10) keeps that +/// dictionary's other entries (a dashed style stays dashed) and leaves the shared object alone. +#[test] +fn new_width_keeps_an_indirect_border_style() { + let mut doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 200 200] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R] >>", + b"<< /Type /Annot /Subtype /Square /Rect [20 20 180 180] /BS 5 0 R /C [1 0 0] >>", + b"<< /W 6 /S /D /D [4 2] >>", + ]); + set_style(&mut doc, 0, 0, StyleChange::new(None, None, Some(3.0), None), &meta("")).unwrap(); + let doc = reopen(&doc); + let d = list(&doc, 0).remove(0); + let bs = d.get(b"BS").and_then(Object::as_dict).expect("a direct /BS"); + assert_eq!(bs.get(b"W").and_then(Object::as_f64), Some(3.0)); + assert_eq!(bs.name(b"S"), Some(b"D".as_slice()), "the dashed style is kept"); + assert!(ap_content(&doc, &d).contains("3 w\n[4 2] 0 d\n")); + let shared = doc.get(ObjRef::new(5, 0)); + assert_eq!(shared.as_dict().and_then(|b| b.get(b"W")).and_then(Object::as_f64), Some(6.0), "the shared object is unchanged"); +} + +/// A colour given by reference (`/C 6 0 R`, §7.3.10) is read by the properties and the edits as +/// the display draws it: the properties show it, the appearance can be restyled and filled. +#[test] +fn an_indirect_colour_reads_and_restyles_as_drawn() { + let mut doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 200 200] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R] >>", + b"<< /Type /Annot /Subtype /Square /Rect [20 20 180 180] /C 5 0 R /IC 6 0 R >>", + b"[0 0 1]", + b"[1 1 0]", + ]); + let p = props(&doc, 0, 0).expect("props"); + assert_eq!((p.color, p.fill, p.restylable), (Some([0.0, 0.0, 1.0]), Some([1.0, 1.0, 0.0]), true)); + set_style(&mut doc, 0, 0, StyleChange::new(None, None, Some(3.0), None), &meta("")).expect("restylable"); + set_fill(&mut doc, 0, 0, Some([0.0, 1.0, 0.0]), &meta("")).expect("fillable"); + let d = list(&doc, 0).remove(0); + let content = ap_content(&doc, &d); + assert!(content.contains("0 0 1 RG") && content.contains("0 1 0 rg") && content.contains("3 w"), "{content}"); +} + +/// References are followed into everything they point at, and all of it is counted: a referenced +/// array of two arrays of 40,000 numbers is over the per-annotation budget. One display pass +/// copies at most [`crate::draw::PASS_BUDGET`] values in all, so an array many annotations point +/// at is not copied again for each of them. +#[test] +fn following_references_is_bounded_per_value_and_per_pass() { + let mut doc = fixture(); + let inner = Object::Array(vec![Object::Int(0); 40_000]); + let nested = doc.add(Object::Array(vec![inner.clone(), inner])); + let mut d = Dict::new(); + d.set(b"Subtype".to_vec(), Object::name("Square")); + d.set(b"Junk".to_vec(), Object::Ref(nested)); + let direct = crate::draw::with_direct_values(&doc, &d); + assert!(direct.get(b"Junk") == Some(&Object::Ref(nested)), "80,003 values: left as a reference"); + + let shared = doc.add(Object::Array(vec![Object::Int(0); 60_000])); + d.set(b"Junk".to_vec(), Object::Ref(shared)); + let mut pass = crate::draw::PASS_BUDGET; + let copies = (0..100).filter(|_| matches!(crate::draw::with_direct_values_within(&doc, &d, &mut pass), std::borrow::Cow::Owned(_))).count(); + assert_eq!(copies, crate::draw::PASS_BUDGET / 60_001, "copies stop when the pass budget is spent"); +} + +/// Finding an object too large to copy costs the walk that found it out: an array of two arrays +/// of 40,000 numbers that many annotations point at is walked about 40,000 values deep before it +/// is rejected, and that is charged to the annotation and to the pass, so after a bounded number +/// of such walks the pass has no room left to walk it again. A reference to it repeated in one annotation +/// is not walked again in full, and a small referenced colour after it is still read. +#[test] +fn a_rejected_large_object_is_charged_to_the_budgets() { + let mut doc = fixture(); + let inner = Object::Array(vec![Object::Int(0); 40_000]); + let nested = doc.add(Object::Array(vec![inner.clone(), inner])); + let red = doc.add(Object::Array(vec![Object::Int(1), Object::Int(0), Object::Int(0)])); + let mut d = Dict::new(); + d.set(b"Subtype".to_vec(), Object::name("Square")); + d.set(b"Junk".to_vec(), Object::Array(vec![Object::Ref(nested); 1_000])); + d.set(b"C".to_vec(), Object::Ref(red)); + let mut pass = crate::draw::PASS_BUDGET; + let direct = crate::draw::with_direct_values_within(&doc, &d, &mut pass); + assert_eq!(direct.get(b"C"), Some(&Object::Array(vec![Object::Int(1), Object::Int(0), Object::Int(0)]))); + let charged = crate::draw::PASS_BUDGET - pass; + assert!((40_000..=crate::draw::INLINE_BUDGET).contains(&charged), "the rejected walk is charged once: {charged}"); + d.set(b"Junk".to_vec(), Object::Ref(nested)); + for _ in 0..crate::draw::PASS_BUDGET / 40_000 { + crate::draw::with_direct_values_within(&doc, &d, &mut pass); + } + assert!(pass < 40_000, "after {} annotations the pass has no room for another full walk: {pass}", crate::draw::PASS_BUDGET / 40_000 + 1); +} + +/// `/RD` is optional (ISO 32000-2 Table 177): a callout without it has its text box at `/Rect` +/// and is drawn, box, text and leader line. A callout line and `/RD` given by reference are read too. +#[test] +fn callout_without_rd_is_drawn() { + let doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 400 400] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R 5 0 R 6 0 R] >>", + b"<< /Type /Annot /Subtype /FreeText /IT /FreeTextCallout /Rect [100 100 300 160] /CL [20 20 60 60 100 130] /LE /OpenArrow /C [1 1 0] /DA (0 g /Helv 12 Tf) /Contents (callout) >>", + b"<< /Type /Annot /Subtype /FreeText /IT /FreeTextCallout /Rect [10 200 300 300] /CL 7 0 R /RD 8 0 R /C [1 1 0] /DA (0 g /Helv 12 Tf) /Contents (indirect) >>", + b"<< /Type /Annot /Subtype /FreeText /IT /FreeTextCallout /Rect [10 300 300 390] /CL [5 305 10 350] /RD [1 2] /C [1 1 0] /DA (0 g /Helv 12 Tf) /Contents (bad) >>", + b"[15 205 40 250 100 250]", + b"[90 0 0 0]", + ]); + let display = with_missing_appearances(&doc).expect("the callouts get a display appearance"); + let shown = list(&display, 0); + let plain = ap_content(&display, &shown[0]); + assert!(plain.contains("1 1 0 rg\n100 100 200 60 re f"), "the yellow box fills /Rect: {plain}"); + assert!(plain.contains("20 20 m\n60 60 l\n100 130 l\nS"), "the leader line: {plain}"); + assert!(plain.contains("(callout) Tj"), "the text: {plain}"); + let indirect = ap_content(&display, &shown[1]); + assert!(indirect.contains("100 200 200 100 re f") && indirect.contains("15 205 m"), "indirect /CL and /RD: {indirect}"); + assert!(shown[2].get(b"AP").is_none(), "a malformed /RD is still not guessed at"); +} + /// #3175: viewers draw a note without an appearance as a fixed-size icon at the upper-left corner /// of its `/Rect`. The display copy must not stretch the icon over a large `/Rect` or shrink it /// to nothing in one with no width. @@ -1979,7 +2794,8 @@ fn free_text_properties_show_legacy_or_absent_border_width() { d.remove(b"Border"); }) .unwrap(); - assert_eq!(props(&doc, 0, index).unwrap().width, Some(0.0)); + // Neither /BS nor /Border: the /Border default [0 0 1] (ISO 32000-2 Table 166), as drawn. + assert_eq!(props(&doc, 0, index).unwrap().width, Some(1.0)); doc.update_dict(reference, |d| { d.set(b"Border".to_vec(), Object::Array([0.0, 0.0, 4.0].into_iter().map(Object::Real).collect())); }) @@ -2571,6 +3387,1338 @@ fn degenerate_and_absurd_leader_and_caption_values_do_not_panic() { assert!(!long.contains(&"w".repeat(257))); } +/// A comment's built appearance for the dictionary `body` (no `/AP`), as text, or `None` when +/// PdfCraft can't draw it. +fn built(body: &str) -> Option { + let doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 600] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R] >>", + body.as_bytes(), + ]); + let d = list(&doc, 0).into_iter().next().unwrap(); + crate::appearance::build(&doc, &d).map(|s| String::from_utf8_lossy(&s.decoded().unwrap()).into_owned()) +} + +/// The x and y range a drawing's path covers: its points, and each curve's middle (a curve's +/// control points lie outside it, so they aren't counted). +fn extent(ops: &str) -> [f64; 4] { + let mut b = [f64::MAX, f64::MAX, f64::MIN, f64::MIN]; + let mut at = (0.0, 0.0); + let add = |p: (f64, f64), b: &mut [f64; 4]| *b = [b[0].min(p.0), b[1].min(p.1), b[2].max(p.0), b[3].max(p.1)]; + for line in ops.lines() { + let t: Vec<&str> = line.split_whitespace().collect(); + let Some((op, nums)) = t.split_last() else { continue }; + let v: Vec = nums.iter().filter_map(|n| n.parse().ok()).collect(); + match (*op, v.as_slice()) { + ("m" | "l", [x, y]) => { + at = (*x, *y); + add(at, &mut b); + } + ("c", [x1, y1, x2, y2, x3, y3]) => { + let mid = ((at.0 + 3.0 * x1 + 3.0 * x2 + x3) / 8.0, (at.1 + 3.0 * y1 + 3.0 * y2 + y3) / 8.0); + add(mid, &mut b); + at = (*x3, *y3); + add(at, &mut b); + } + _ => {} + } + } + b +} + +/// ISO 32000-2 §12.5.4, Table 169: a cloudy border (`/BE /S /C`) applies to squares, circles, +/// polygons and (PDF 1.6) text boxes, honours the `/BS` width and dashes, and its intensity +/// `/I` defaults to 0. Squares, circles and text boxes without an appearance were not drawn at +/// all, so a reviewer's marks vanished; intensity 0 (or none) drew a cloud on polygons. +#[test] +fn cloudy_borders_are_drawn_on_squares_circles_polygons_and_text_boxes() { + let base = "/Type /Annot /Rect [100 100 300 250] /C [1 0 0] /BS << /W 2 >>"; + for kind in ["Square", "Circle"] { + let ops = built(&format!("<< {base} /Subtype /{kind} /BE << /S /C /I 1 >> >>")).unwrap_or_else(|| panic!("{kind} is drawn")); + assert!(ops.matches(" c\n").count() > 8 && !ops.contains(" re\n"), "{kind} as bumps: {ops}"); + let e = extent(&ops); + assert!(e[0] >= 100.0 && e[1] >= 100.0 && e[2] <= 300.0 && e[3] <= 250.0, "{kind} stays in /Rect: {e:?}"); + } + let dashed = built(&format!("<< {base} /Subtype /Square /BS << /W 2 /S /D /D [6 3] >> /BE << /S /C /I 1 >> >>")).unwrap(); + assert!(dashed.contains("[6 3] 0 d") && dashed.contains(" c\n"), "dashes honoured: {dashed}"); + let text = built(&format!("<< {base} /Subtype /FreeText /DA (0 0 1 rg /Helv 10 Tf) /Contents (Look here) /BE << /S /C /I 1 >> >>")).unwrap(); + assert!(text.contains(" c\n") && text.contains("(Look here) Tj"), "a cloud around the text: {text}"); + let dashed_text = built( + "<< /Type /Annot /Rect [100 100 300 250] /C [1 0 0] /BS << /W 2 /S /D /D [4 2] >> /Subtype /FreeText /DA (0 g /Helv 10 Tf) /Contents (x) /BE << /S /C /I 1 >> >>", + ) + .unwrap(); + assert!(dashed_text.contains("[4 2] 0 d"), "a text box's cloud is dashed too: {dashed_text}"); + // Intensity 0, or none: a plain border (the default is 0). + for be in ["<< /S /C /I 0 >>", "<< /S /C >>", "<< /S /S /I 2 >>"] { + let square = built(&format!("<< {base} /Subtype /Square /BE {be} >>")).unwrap(); + assert!(square.contains(" re\n") && !square.contains(" c\n"), "{be}: {square}"); + let polygon = + built(&format!("<< /Type /Annot /Subtype /Polygon /Rect [0 0 100 100] /C [1 0 0] /Vertices [10 10 90 10 90 90 10 90] /BE {be} >>")) + .unwrap(); + assert!(!polygon.contains(" c\n"), "{be}: polygon edges stay straight: {polygon}"); + } + let cloudy_polygon = + built("<< /Type /Annot /Subtype /Polygon /Rect [0 0 100 100] /C [1 0 0] /Vertices [10 10 90 10 90 90 10 90] /BE << /S /C /I 1 >> >>") + .unwrap(); + assert!(cloudy_polygon.contains(" c\n")); + // A cloudy callout is still left to its own appearance. + assert!( + built(&format!("<< {base} /Subtype /FreeText /DA (0 g /Helv 10 Tf) /CL [10 10 100 100] /RD [5 5 5 5] /BE << /S /C /I 1 >> >>")).is_none() + ); +} + +/// Table 180: `/RD` gives the differences between "the left, top, right, and bottom coordinates +/// of Rect and those of the square or circle, respectively"; a cloudy square's bumps sit on that +/// inner square. +#[test] +fn a_cloudy_square_follows_rd_left_top_right_bottom() { + let ops = built("<< /Type /Annot /Subtype /Square /Rect [100 100 300 300] /C [1 0 0] /BS << /W 1 >> /BE << /S /C /I 1 >> /RD [20 40 30 25] >>") + .unwrap(); + // The corners of the inner square are path points: left 120, bottom 125, right 270, top 260. + for corner in ["120 125", "270 125", "270 260", "120 260"] { + assert!(ops.contains(corner), "corner {corner} in {ops}"); + } +} + +/// Review on #4195: without `/RD`, a small cloudy square still keeps its bumps inside `/Rect`. +/// Its edges are short enough that each rounds to a single bump about three radii long, which +/// bulges further than one radius. An `/RD` leaving less than that margin is widened to it, so +/// the cloud stays inside `/Rect` too. +#[test] +fn a_small_cloudy_square_without_rd_stays_inside_its_rect() { + for side in [40.0, 48.8, 60.0, 82.0] { + let ops = built(&format!( + "<< /Type /Annot /Subtype /Square /Rect [100 100 {x} {x}] /C [1 0 0] /BS << /W 2 >> /BE << /S /C /I 1 >> >>", + x = 100.0 + side + )) + .unwrap_or_else(|| panic!("side {side} is drawn")); + let e = extent(&ops); + assert!(e[0] >= 99.0 && e[1] >= 99.0 && e[2] <= 101.0 + side && e[3] <= 101.0 + side, "side {side}: {e:?}"); + } + let small_rd = + built("<< /Type /Annot /Subtype /Square /Rect [100 100 200 200] /C [1 0 0] /BS << /W 2 >> /BE << /S /C /I 1 >> /RD [1 1 1 1] >>").unwrap(); + let e = extent(&small_rd); + assert!(e[0] >= 99.0 && e[1] >= 99.0 && e[2] <= 201.0 && e[3] <= 201.0, "/RD [1 1 1 1] keeps the cloud inside /Rect: {e:?}"); +} + +/// ISO 32000-2 Table 180: `/RD` gives the differences between `/Rect` and "the actual boundaries +/// of the underlying square or circle", so a plain square or circle (no cloudy border) is drawn +/// inside `/Rect` less `/RD` (left, top, right, bottom) too, its stroke centred half the width +/// inside that box. An `/RD` that isn't four finite non-negative numbers, or leaves no box, is +/// ignored and the shape fills `/Rect` as before. +#[test] +fn plain_squares_and_circles_are_drawn_inside_rect_less_rd() { + let square = |rd: &str| built(&format!("<< /Type /Annot /Subtype /Square /Rect [100 100 200 200] /C [1 0 0] /BS << /W 3 >> {rd} >>")).unwrap(); + // Rect + 15 + 1.5 on every side: a 67 × 67 square from 116.5. + assert!(square("/RD [15 15 15 15]").contains("116.5 116.5 67 67 re"), "{}", square("/RD [15 15 15 15]")); + // Left 10, top 40, right 20, bottom 30: the box is 110..180 × 130..160. + assert!(square("/RD [10 40 20 30]").contains("111.5 131.5 67 27 re"), "{}", square("/RD [10 40 20 30]")); + for ignored in ["", "/RD [-1 0 0 0]", "/RD [1 2 3]", "/RD [60 0 60 0]", "/RD [0 50 0 49.5]", "/RD (x)"] { + assert!(square(ignored).contains("101.5 101.5 97 97 re"), "{ignored}: {}", square(ignored)); + } + let circle = built("<< /Type /Annot /Subtype /Circle /Rect [100 100 200 200] /C [1 0 0] /BS << /W 3 >> /RD [15 15 15 15] >>").unwrap(); + // The ellipse starts at its rightmost point: 200 - 15 - 1.5, at mid-height. + assert!(circle.starts_with("1 0 0 RG\n3 w\n183.5 150 m"), "{circle}"); +} + +/// The circles and rectangles PdfCraft draws write no `/RD`, so their appearance is unchanged by +/// it: the stroke still sits half its width inside `/Rect`. +#[test] +fn pdfcraft_rectangles_and_ellipses_still_fill_their_rect() { + let mut doc = fixture(); + let rect = [10.0, 10.0, 110.0, 60.0]; + for shape in [Shape::Rectangle { rect }, Shape::Oval { rect }] { + let oval = matches!(shape, Shape::Oval { .. }); + let i = add_annotation(&mut doc, &new(0, shape), &meta("r")).unwrap(); + let d = list(&doc, 0).remove(i); + assert!(!d.contains(b"RD")); + let w = appearance::border_width(&d); + let ap = ap_content(&doc, &d); + let r: Vec = d.get(b"Rect").and_then(Object::as_array).unwrap().iter().filter_map(Object::as_f64).collect(); + let expected = if oval { + format!("{} {} m", n(r[2] - w / 2.0), n((r[1] + r[3]) / 2.0)) + } else { + format!("{} {} {} {} re", n(r[0] + w / 2.0), n(r[1] + w / 2.0), n(r[2] - r[0] - w), n(r[3] - r[1] - w)) + }; + assert!(ap.contains(&expected), "{expected} in {ap}"); + } +} + +/// Review on #4195: a polygon reshaped with `/BE /S /C` and no `/I` (or `/I 0`) is a plain one, +/// as its appearance is drawn (Table 169: intensity 0 is no cloud), so its `/Rect` gets no room +/// for bumps; with an intensity above 0 it does. +#[test] +fn a_reshaped_polygon_pads_its_rect_only_for_a_drawn_cloud() { + // `/BE << /S /C >>` with `/I` when given; no `/BE` at all for `None`. + let reshaped = |be: Option>| { + let mut doc = fixture(); + let shape = Shape::Polygon { vertices: vec![[100.0, 100.0], [200.0, 100.0], [150.0, 200.0]], cloud: false }; + let i = add_annotation(&mut doc, &new(0, shape), &meta("p")).unwrap(); + let (_, r) = annot_ref(&mut doc, 0, i).unwrap(); + if let Some(intensity) = be { + let mut b = pdfcraft_cos::Dict::new(); + b.set(b"S".to_vec(), Object::name("C")); + if let Some(i) = intensity { + b.set(b"I".to_vec(), Object::Int(i)); + } + doc.update_dict(r, |d| d.set(b"BE".to_vec(), Object::Dict(b))).unwrap(); + } + set_points(&mut doc, 0, i, &[[100.0, 100.0], [300.0, 100.0], [200.0, 250.0]], &meta("")).unwrap(); + rect(&list(&doc, 0)[i]) + }; + let plain = reshaped(None); + for be in [None, Some(0)] { + assert_eq!(reshaped(Some(be)), plain, "/I {be:?}: no cloud drawn, no room for one"); + } + let cloudy = reshaped(Some(Some(1))); + assert!(cloudy[0] < plain[0] && cloudy[2] > plain[2], "room for the bumps: {cloudy:?} around {plain:?}"); +} + +/// Review on #4195: a reshaped polygon whose cloud has intensity 2 gets room for its bumps at +/// that intensity. With room only for intensity 1, the bumps on a 200 pt edge reached past +/// `/Rect` and the appearance's `/BBox` clipped them. +#[test] +fn a_reshaped_intensity_two_cloud_stays_inside_its_rect() { + let mut doc = fixture(); + let shape = Shape::Polygon { vertices: vec![[100.0, 100.0], [200.0, 100.0], [150.0, 200.0]], cloud: true }; + let i = add_annotation(&mut doc, &new(0, shape), &meta("p")).unwrap(); + let (_, r) = annot_ref(&mut doc, 0, i).unwrap(); + let mut be = pdfcraft_cos::Dict::new(); + be.set(b"S".to_vec(), Object::name("C")); + be.set(b"I".to_vec(), Object::Int(2)); + doc.update_dict(r, |d| d.set(b"BE".to_vec(), Object::Dict(be))).unwrap(); + set_points(&mut doc, 0, i, &[[100.0, 100.0], [300.0, 100.0], [200.0, 250.0]], &meta("")).unwrap(); + let d = &list(&doc, 0)[i]; + let bbox = rect(d); + // Every point along the drawn bumps, with half the 2 pt line, is inside `/Rect`. + let content = ap_content(&doc, d); + let (mut at, mut lowest, mut leftmost) = ((0.0, 0.0), f64::MAX, f64::MAX); + for line in content.lines() { + let v: Vec = line.split_whitespace().filter_map(|t| t.parse().ok()).collect(); + if line.ends_with(" m") && v.len() == 2 { + at = (v[0], v[1]); + } else if line.ends_with(" c") && v.len() == 6 { + for step in 0..=20 { + let t = f64::from(step) / 20.0; + let u = 1.0 - t; + let (a, b, c, e) = (u * u * u, 3.0 * u * u * t, 3.0 * u * t * t, t * t * t); + lowest = lowest.min(a * at.1 + b * v[1] + c * v[3] + e * v[5]); + leftmost = leftmost.min(a * at.0 + b * v[0] + c * v[2] + e * v[4]); + } + at = (v[4], v[5]); + } + } + assert!(lowest < 100.0 - 8.0, "the bumps bulge below the bottom edge: {lowest}"); + assert!(lowest - 1.0 >= bbox[1], "the bottom bumps ({lowest}) and the line fit in /Rect {bbox:?}"); + assert!(leftmost - 1.0 >= bbox[0], "the left bumps ({leftmost}) and the line fit in /Rect {bbox:?}"); +} + +/// Review on #4195: a reshaped polygon gets room for the border it is drawn with. With no `/BS`, +/// a `/Border [0 0 5]` is drawn 5 pt wide (`appearance::border_width`), so the new `/Rect` is the +/// one `/BS /W 5` gets; it was sized for a 1-pt border, and the cloud's outer bumps were clipped. +#[test] +fn a_reshaped_polygon_has_room_for_the_border_it_is_drawn_with() { + let reshaped = |border: &[u8], width: Object| { + let mut doc = fixture(); + let shape = Shape::Polygon { vertices: vec![[100.0, 100.0], [200.0, 100.0], [150.0, 200.0]], cloud: true }; + let i = add_annotation(&mut doc, &new(0, shape), &meta("p")).unwrap(); + let (_, r) = annot_ref(&mut doc, 0, i).unwrap(); + doc.update_dict(r, |d| { + d.remove(b"BS"); + if border == b"BS" { + let mut bs = pdfcraft_cos::Dict::new(); + bs.set(b"W".to_vec(), width); + d.set(b"BS".to_vec(), Object::Dict(bs)); + } else { + d.set(b"Border".to_vec(), Object::Array(vec![Object::Int(0), Object::Int(0), width])); + } + }) + .unwrap(); + set_points(&mut doc, 0, i, &[[100.0, 100.0], [300.0, 100.0], [200.0, 250.0]], &meta("")).unwrap(); + rect(&list(&doc, 0)[i]) + }; + let bs = reshaped(b"BS", Object::Int(5)); + assert_eq!(reshaped(b"Border", Object::Int(5)), bs, "the same 5-pt border, the same room"); + assert_ne!(reshaped(b"BS", Object::Int(1)), bs, "a 1-pt border needs less"); +} + +/// Every number a drawing writes, which must all be finite. +fn drawn_numbers(ops: &str) -> Vec { + ops.split_whitespace().filter(|t| t.starts_with(|c: char| c.is_ascii_digit() || c == '-' || c == '.')).filter_map(|t| t.parse().ok()).collect() +} + +/// Review on #4195: a cloud around a zero-size, inverted, sliver or huge `/Rect`, at any border +/// width and any `/BE /I` (tiny, negative, far above Table 169's 2), is either drawn inside +/// `/Rect` with finite numbers and a bounded number of bumps, or not built at all, never a +/// panic, a division by zero or a NaN. +#[test] +fn hostile_cloud_sizes_widths_and_intensities_draw_inside_rect_or_not_at_all() { + let rects = [ + "0 0 0 0", + "100 100 100 100", + "100 100 100.5 300", + "100 100 101 101", + "300 250 100 100", + "100 100 300 250", + "100 100 110 110", + "-1000000000000000 -1000000000000000 1000000000000000 1000000000000000", + "0 0 99999999999999999999999 1", + &format!("0 0 {} 100", "9".repeat(400)), + &format!("-{n} -{n} {n} {n}", n = "9".repeat(308)), + ]; + let (mut drawn, mut refused) = (0, 0); + for rect in rects { + let r: Vec = rect.split_whitespace().filter_map(|t| t.parse().ok()).collect(); + let bounds = [r[0].min(r[2]), r[1].min(r[3]), r[0].max(r[2]), r[1].max(r[3])]; + for width in ["0", "0.001", "2", "1000000", "-3"] { + for intensity in ["0.0000001", "2", "1000000000000000000000000000000", "-1"] { + for kind in ["Square", "Circle", "FreeText"] { + let body = format!( + "<< /Type /Annot /Subtype /{kind} /Rect [{rect}] /C [1 0 0] /BS << /W {width} >> /BE << /S /C /I {intensity} >> /DA (0 g /Helv 10 Tf) /Contents (x) >>" + ); + let Some(ops) = built(&body) else { + refused += 1; + continue; + }; + drawn += 1; + assert!(!ops.contains("NaN") && !ops.contains("inf"), "{body}: {ops}"); + assert!(drawn_numbers(&ops).iter().all(|v| v.is_finite()), "{body}"); + assert!(ops.matches(" c\n").count() <= crate::appearance::MAX_CLOUD_BUMPS + 8, "{body}: bumps are capped"); + // A cloud (intensity above 0); a negative one is a plain border (Table 169). + if !intensity.starts_with('-') && ops.contains(" c\n") { + let e = extent(&ops); + let slack = 0.5 + 1e-9 * (bounds[2] - bounds[0]).abs().max((bounds[3] - bounds[1]).abs()); + assert!( + e[0] >= bounds[0] - slack && e[1] >= bounds[1] - slack && e[2] <= bounds[2] + slack && e[3] <= bounds[3] + slack, + "{body}: the cloud {e:?} stays in /Rect" + ); + } + } + } + } + } + assert!(drawn > 0 && refused > 0, "both outcomes are exercised: {drawn} drawn, {refused} refused"); +} + +/// Review on #4195: a cloud around a rotated rectangle (a polygon turned 45°, its corners in +/// either order) bulges outwards the same way and stays inside its `/Rect`, and `cloud_path` +/// itself takes a zero, negative, infinite or NaN bump size and repeated corners without +/// dividing by zero. +#[test] +fn a_cloud_around_a_rotated_rectangle_bulges_outwards_either_way_round() { + let ccw = "150 100 200 150 150 200 100 150"; + let cw = "150 100 100 150 150 200 200 150"; + let extents: Vec<[f64; 4]> = [ccw, cw] + .iter() + .map(|v| { + let ops = built(&format!("<< /Type /Annot /Subtype /Polygon /Rect [80 80 220 220] /C [1 0 0] /Vertices [{v}] /BE << /S /C /I 1 >> >>")) + .unwrap(); + assert!(ops.contains(" c\n"), "{ops}"); + let e = extent(&ops); + assert!(e[0] >= 80.0 && e[1] >= 80.0 && e[2] <= 220.0 && e[3] <= 220.0, "inside /Rect: {e:?}"); + // Each bump's middle lies outside the turned square |x − 150| + |y − 150| = 50. + let mut at = (0.0, 0.0); + for line in ops.lines() { + let v: Vec = line.split_whitespace().filter_map(|t| t.parse().ok()).collect(); + if line.ends_with(" m") && v.len() == 2 { + at = (v[0], v[1]); + } else if line.ends_with(" c") && v.len() == 6 { + let mid = ((at.0 + 3.0 * v[0] + 3.0 * v[2] + v[4]) / 8.0, (at.1 + 3.0 * v[1] + 3.0 * v[3] + v[5]) / 8.0); + assert!((mid.0 - 150.0).abs() + (mid.1 - 150.0).abs() > 50.0 + 1.0, "the bump at {mid:?} bulges outwards"); + at = (v[4], v[5]); + } + } + e + }) + .collect(); + for k in 0..4 { + assert!((extents[0][k] - extents[1][k]).abs() < 0.5, "the same cloud either way round: {extents:?}"); + } + let square = [(0.0, 0.0), (10.0, 0.0), (10.0, 10.0), (0.0, 10.0)]; + for r in [0.0, -1.0, f64::INFINITY, f64::NAN, 1e-300, 1e300] { + let path = crate::appearance::cloud_path(&square, r); + assert!(!path.contains("NaN") && !path.contains("inf"), "r {r}: {path}"); + assert!(path.ends_with("h\n")); + } + for pts in [vec![], vec![(5.0, 5.0)], vec![(5.0, 5.0); 4], vec![(0.0, 0.0), (10.0, 0.0), (10.0, 0.0), (0.0, 0.0)]] { + let path = crate::appearance::cloud_path(&pts, 2.0); + assert!(!path.contains("NaN") && !path.contains("inf"), "{pts:?}: {path}"); + } +} + +/// PdfCraft's own text boxes, typewriter text and redaction marks write `/BS` (width 0 for text +/// boxes and typewriter text), `/DA /Helv` and a `/DS` that agrees with it, and quadrilaterals: +/// the default border, `/DA` fonts, `/DS` and the `/Rect` outline leave them exactly as they were. +#[test] +fn pdfcraft_text_boxes_and_redaction_marks_look_as_before() { + let mut doc = fixture(); + let mut tb = new(0, Shape::TextBox { rect: [100.0, 100.0, 220.0, 160.0], font_size: 12.5 }); + tb.style = Style { color: [0.1, 0.2, 0.3], fill: Some([1.0, 1.0, 0.8]), ..Style::default_for(&tb.shape) }; + tb.contents = "PdfCraft text box that wraps onto a second line".into(); + let t = add_annotation(&mut doc, &tb, &meta("t")).unwrap(); + let mut tw = new(0, Shape::Typewriter { rect: [100.0, 300.0, 300.0, 320.0], font_size: 10.0 }); + tw.style = Style::default_for(&tw.shape); + let w = add_annotation(&mut doc, &tw, &meta("w")).unwrap(); + let quads = vec![[100.0, 520.0, 200.0, 520.0, 100.0, 500.0, 200.0, 500.0]]; + let mut rd = new(0, Shape::Redact { quads, overlay: String::new(), look: OverlayLook::default() }); + rd.style = Style::default_for(&rd.shape); + let r = add_annotation(&mut doc, &rd, &meta("r")).unwrap(); + let all = list(&doc, 0); + assert_eq!( + ap_content(&doc, &all[t]), + "1 1 0.8 rg\n100 100 120 60 re f\n100 100 120 60 re W n\nBT\n/Helv 12.5 Tf\n0.1 0.2 0.3 rg\n1 0 0 1 102 146.75 Tm (PdfCraft text box) Tj\n1 0 0 1 102 131.75 Tm (that wraps onto a) Tj\n1 0 0 1 102 116.75 Tm (second line) Tj\nET\n" + ); + assert_eq!(text(&all[t], b"DS"), "font: Helvetica 12.5pt; text-align:left; color:#1A334D"); + assert_eq!(ap_content(&doc, &all[w]), "100 300 200 20 re W n\nBT\n/Helv 10 Tf\n0 0 0 rg\n1 0 0 1 102 309 Tm (hello) Tj\nET\n"); + assert_eq!(ap_content(&doc, &all[r]), "0.89 0.13 0.13 RG\n1 w\n100 520 m 200 520 l 200 500 l 100 500 l h S\n"); + // New text keeps the box's look: Helvetica, /DA's exact colour (not /DS's 8-bit one). + set_contents(&mut doc, 0, t, "Edited", &meta("")).unwrap(); + let d = &list(&doc, 0)[t]; + assert_eq!(text(d, b"DS"), "font: Helvetica 12.5pt; text-align:left; color:#1A334D"); + let ap = ap_content(&doc, d); + assert!(ap.contains("/Helv 12.5 Tf\n0.1 0.2 0.3 rg\n") && !ap.contains(" re S"), "{ap}"); +} + +/// The display appearance of the annotation dictionary `annot` on a page of a document whose +/// form `/DR` maps `Helv`, `Cour`, `TiRo`, `F1` (Courier) and `F2` (a font that isn't standard), +/// and the font resources it draws with. +fn display_of(annot: &str) -> (String, Dict) { + let doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R /AcroForm << /DR << /Font << /Helv 5 0 R /Cour 6 0 R /TiRo 7 0 R /F1 6 0 R /F2 9 0 R >> >> >> >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [8 0 R] >>", + b"<< /Length 0 >>\nstream\n\nendstream", + b"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>", + b"<< /Type /Font /Subtype /Type1 /BaseFont /Courier /Encoding /WinAnsiEncoding >>", + b"<< /Type /Font /Subtype /Type1 /BaseFont /Times-Roman /Encoding /WinAnsiEncoding >>", + annot.as_bytes(), + b"<< /Type /Font /Subtype /TrueType /BaseFont /ABCDEF+HouseGothic >>", + ]); + let shown = with_missing_appearances(&doc).expect("drawn for display"); + let d = list(&shown, 0).remove(0); + let n = d.get(b"AP").and_then(|a| a.as_dict()).and_then(|a| a.reference(b"N")).expect("/AP /N"); + let Object::Stream(s) = &*shown.get(n) else { panic!("not a stream") }; + let fonts = s.dict.get(b"Resources").and_then(|r| r.as_dict()).and_then(|r| r.get(b"Font")).and_then(|f| f.as_dict()).cloned(); + (String::from_utf8_lossy(&s.decoded().unwrap()).into_owned(), fonts.unwrap_or_default()) +} + +fn base_fonts(fonts: &Dict) -> Vec { + fonts.iter().filter_map(|(_, f)| f.as_dict()?.name(b"BaseFont").map(|b| String::from_utf8_lossy(b).into_owned())).collect() +} + +/// A text box without `/BS` or `/Border` has the `/Border` default [0 0 1] (ISO 32000-2 +/// Table 166): a 1-pt border in the text colour. `/BS /W 0` or `/Border [0 0 0]` turn it off. +#[test] +fn free_text_without_a_border_entry_has_the_default_one() { + let (ap, _) = display_of("<< /Type /Annot /Subtype /FreeText /Rect [45 630 165 700] /DA (0 0 1 rg /Helv 12 Tf) /Contents (Hi) >>"); + assert!(ap.contains("0 0 1 RG\n1 w\n45.5 630.5 119 69 re S\n"), "{ap}"); + assert!(ap.contains("1 0 0 1 48 "), "text inset by the border: {ap}"); + for off in ["/BS << /W 0 >>", "/Border [0 0 0]"] { + let (ap, _) = display_of(&format!("<< /Type /Annot /Subtype /FreeText /Rect [45 630 165 700] /DA (0 g /Helv 12 Tf) {off} /Contents (Hi) >>")); + assert!(!ap.contains(" re S"), "{off}: {ap}"); + } +} + +/// Review on #4365: a text box's new text is fitted with the border it is drawn with. A 4-pt border +/// written as `/Border [0 0 4]` (no `/BS`) is drawn 4 pt wide, so the box grows as for `/BS /W 4`; +/// before, the fit read only `/BS` and measured it with a 1-pt border, so the drawn text, wrapped +/// 6 pt narrower, could run past the box. +#[test] +fn new_text_is_fitted_with_the_border_the_box_is_drawn_with() { + let fitted = |border: &str| { + let annot = format!("<< /Type /Annot /Subtype /FreeText /Rect [45 600 165 700] /DA (0 g /Helv 12 Tf) {border} /Contents (Hi) >>"); + let mut doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R] >>", + annot.as_bytes(), + ]); + set_contents(&mut doc, 0, 0, "Several words that wrap over a few lines of this box", &meta("")).unwrap(); + list(&doc, 0).remove(0).get(b"Rect").and_then(|r| r.as_array()).map(|r| r.iter().filter_map(|x| x.as_f64()).collect::>()).unwrap() + }; + let bs = fitted("/BS << /W 4 >>"); + assert_eq!(fitted("/Border [0 0 4]"), bs, "the same 4-pt border, the same box"); + assert_ne!(fitted("/BS << /W 1 >>"), bs, "a 1-pt border fits another box"); +} + +/// `/DA` names its font in the form's `/DR` (§12.7.4.3): a standard face is drawn in that face, +/// with its widths for wrapping; any other font stays Helvetica. +#[test] +fn free_text_da_fonts_draw_in_their_standard_face() { + use pdfcraft_fonts::Std14; + let width = 120.0 - 2.0 * 2.0; + let words = "iiiiiiiiii iiiiiiiiii"; + let lines = |face: Std14| pdfcraft_fonts::wrap_with(words, 12.0, width, |s, z| face.text_width(s, z)).len(); + assert_eq!((lines(Std14::Helvetica), lines(Std14::Courier)), (1, 2), "the case tells the faces apart"); + for (da, base, key, n) in [ + ("0 g /Cour 12 Tf", "Courier", "/Cour 12 Tf", 2), + ("1 0 0 rg /TiRo 12 Tf", "Times-Roman", "/TiRo 12 Tf", 1), + ("0 g /F1 12 Tf", "Courier", "/Cour 12 Tf", 2), + ("0 g /F2 12 Tf", "Helvetica", "/Helv 12 Tf", 1), + ("0 g /Nowhere 12 Tf", "Helvetica", "/Helv 12 Tf", 1), + ("0 g /CoBo 12 Tf", "Courier-Bold", "/CoBo 12 Tf", 2), + ] { + let annot = format!("<< /Type /Annot /Subtype /FreeText /Rect [45 630 165 700] /DA ({da}) /BS << /W 0 >> /Contents ({words}) >>"); + let (ap, fonts) = display_of(&annot); + assert_eq!(base_fonts(&fonts), [base], "{da}"); + assert!(ap.contains(key), "{da}: {ap}"); + assert_eq!(ap.matches(" Tj").count(), n, "{da}: wrapped at {base}'s widths: {ap}"); + } +} + +/// Thai in a text box whose `/DA` names Times is drawn from the Thai face and the rest of its +/// text in Times: after a Thai run the text goes back to `/TiRo`, never to `/Helv`, and both +/// kinds of font are in the appearance's resources. +#[test] +fn thai_in_a_times_text_box_keeps_times_for_the_rest() { + // "สวัสดี Hi": Thai first, then Latin. + let annot = "<< /Type /Annot /Subtype /FreeText /Rect [45 630 245 700] /DA (0 g /TiRo 12 Tf) /BS << /W 0 >> /Contents >>"; + let (ap, fonts) = display_of(annot); + assert!(!ap.contains("/Helv"), "{ap}"); + assert!(base_fonts(&fonts).contains(&"Times-Roman".to_string()), "{fonts:?}"); + if pdfcraft_fonts::document_thai_font().is_none() { + return; + } + assert!(ap.contains("/PCTh1 ") && !ap.contains('?'), "the Thai is shaped: {ap}"); + assert_eq!(ap.matches("/TiRo 12 Tf").count(), 2, "Times is selected again after the Thai: {ap}"); + assert!(fonts.get(b"PCTh1").and_then(|f| f.as_dict()).is_some_and(|f| f.name(b"Subtype") == Some(&b"Type3"[..])), "{fonts:?}"); +} + +/// `/DS` (Table 177) sets the face, size and colour over `/DA`'s; a malformed one is ignored. +#[test] +fn free_text_ds_style_is_drawn() { + let annot = + "<< /Type /Annot /Subtype /FreeText /Rect [45 630 165 700] /DA (0 g /Helv 12 Tf) /DS (font: Times 14pt; color: #008000) /Contents (Hi) >>"; + let (ap, fonts) = display_of(annot); + assert_eq!(base_fonts(&fonts), ["Times-Roman"]); + assert!(ap.contains("/TiRo 14 Tf\n0 0.502 0 rg\n"), "{ap}"); + let junk = format!("font: NaNpt; font-size: inf; color: rgb(1e999, x); {}", "\u{FFFD}".repeat(20_000)); + let annot = format!("<< /Type /Annot /Subtype /FreeText /Rect [45 630 165 700] /DA (0 0 1 rg /Cour 9 Tf) /DS ({junk}) /Contents (Hi) >>"); + let (ap, fonts) = display_of(&annot); + assert_eq!(base_fonts(&fonts), ["Courier"]); + assert!(ap.contains("/Cour 9 Tf\n0 0 1 rg\n"), "{ap}"); +} + +/// New text in a box drawn in Courier stays Courier: its rich text says so. +#[test] +fn new_text_keeps_a_foreign_box_face() { + let mut doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R] >>", + b"<< /Type /Annot /Subtype /FreeText /Rect [45 630 165 700] /DA (0 g /Cour 12 Tf) /Contents (Hi) >>", + ]); + set_contents(&mut doc, 0, 0, "Hello", &meta("")).unwrap(); + let d = &list(&doc, 0)[0]; + assert_eq!(text(d, b"DS"), "font: Courier 12pt; text-align:left; color:#000000"); + assert!(ap_content(&doc, d).contains("/Cour 12 Tf")); +} + +/// A text box whose `/DS` overrides `/DA` reports the colour and size it is drawn in, and a colour +/// or size change keeps the other drawn value. +#[test] +fn text_box_properties_report_the_drawn_style() { + let doc = || { + document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R] >>", + b"<< /Type /Annot /Subtype /FreeText /Rect [45 500 300 700] /Contents (Hi) /DA (1 0 0 rg /Helv 12 Tf) /DS (font: Helvetica 24pt; color:#0000FF) >>", + ]) + }; + let mut d = doc(); + let p = props(&d, 0, 0).unwrap(); + assert_eq!((p.color, p.font_size), (Some([0.0, 0.0, 1.0]), Some(24.0))); + set_style(&mut d, 0, 0, StyleChange::new(Some([0.0, 1.0, 0.0]), None, None, None), &meta("")).unwrap(); + let p = props(&d, 0, 0).unwrap(); + assert_eq!((p.color, p.font_size), (Some([0.0, 1.0, 0.0]), Some(24.0)), "a colour change keeps the drawn size"); + let mut d = doc(); + set_font_size(&mut d, 0, 0, 18.0, &meta("")).unwrap(); + let p = props(&d, 0, 0).unwrap(); + assert_eq!((p.color, p.font_size), (Some([0.0, 0.0, 1.0]), Some(18.0)), "a size change keeps the drawn colour"); +} + +/// A redaction mark without `/QuadPoints` marks `/Rect` (Table 195). While marked it is outlined +/// only when it names an outline colour (an array `/OC` of three numbers); without one it draws +/// nothing. Marks with quadrilaterals and no `/OC` colour are outlined in `/C` as before. +#[test] +fn redaction_marks_without_quadpoints_outline_their_rect_in_their_oc_colour() { + let (ap, _) = display_of("<< /Type /Annot /Subtype /Redact /Rect [45 470 165 540] /IC [0 0 0] /OC [1 0 0] /C [0 0 1] >>"); + assert_eq!(ap, "1 0 0 RG\n1 w\n45.5 539.5 m 164.5 539.5 l 164.5 470.5 l 45.5 470.5 l h S\n"); + // No /OC colour: nothing to draw while marked (no /C outline, no /IC fill, no overlay text). + let blank = |annot: &str| { + let annot = format!("<< /Type /Annot /Subtype /Redact /Rect [45 470 165 540] {annot} >>"); + let doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R] >>", + annot.as_bytes(), + ]); + with_missing_appearances(&doc).is_none() + }; + // An /OC that isn't three finite numbers is no colour; an optional-content dictionary keeps its meaning. + for annot in [ + "/C [0.9 0.1 0.1]", + "/IC [0 0 0] /OverlayText (REDACTED) /DA (1 1 1 rg /Helv 10 Tf) /C [0.9 0.1 0.1]", + "/OC [1 0]", + "/OC [(a) 0 0]", + "/OC << /Type /OCG /Name (L) >>", + ] { + assert!(blank(annot), "{annot}"); + } + // With /OC, the interior colour and overlay text still wait for the redaction to be applied. + let (ap, _) = + display_of("<< /Type /Annot /Subtype /Redact /Rect [45 470 165 540] /OC [1 0 0] /IC [0 0 0] /OverlayText (X) /DA (1 1 1 rg /Helv 10 Tf) >>"); + assert!(!ap.contains(" f\n") && !ap.contains("Tj"), "{ap}"); + // With quadrilaterals and no /OC: the /C outline, as before. + let (ap, _) = display_of("<< /Type /Annot /Subtype /Redact /Rect [45 470 165 540] /QuadPoints [45 540 165 540 45 470 165 470] /C [0 0 1] >>"); + assert_eq!(ap, "0 0 1 RG\n1 w\n45 540 m 165 540 l 165 470 l 45 470 l h S\n"); +} + +/// A redaction mark with quadrilaterals is outlined in its array `/OC` colour when it has one, +/// over `/C`; without `/OC` in `/C`, and without either in the default red. +#[test] +fn redaction_marks_with_quadpoints_take_their_outline_from_oc_then_c() { + let quad = |extra: &str| { + display_of(&format!("<< /Type /Annot /Subtype /Redact /Rect [45 470 165 540] /QuadPoints [45 540 165 540 45 470 165 470] {extra} >>")).0 + }; + let outline = "1 w\n45 540 m 165 540 l 165 470 l 45 470 l h S\n"; + assert_eq!(quad("/OC [0 0 1]"), format!("0 0 1 RG\n{outline}"), "/OC blue"); + assert_eq!(quad("/OC [0 0 1] /C [0 1 0]"), format!("0 0 1 RG\n{outline}"), "/OC over /C"); + assert_eq!(quad("/C [0 1 0]"), format!("0 1 0 RG\n{outline}"), "/C only"); + assert_eq!(quad(""), format!("0.89 0.13 0.13 RG\n{outline}"), "neither: red"); + // An optional-content /OC is not a colour. + assert_eq!(quad("/OC << /Type /OCG /Name (L) >> /C [0 1 0]"), format!("0 1 0 RG\n{outline}"), "OCG /OC"); +} + +/// The six unapplied marks of the synthetic fixture the parity note's observation used: an array +/// `/OC` outlines the mark in its colour with or without `/QuadPoints`, beside `/C` or `/IC`; a +/// mark without `/QuadPoints` or `/OC` draws nothing while marked. +#[test] +fn unapplied_redaction_marks_follow_their_oc_outline_colour() { + let quad = "/QuadPoints [60 520 260 520 60 420 260 420]"; + let cases = [ + ("/OC [0 0 1]", Some("0 0 1 RG")), + ("/C [0 1 0]", None), + ("/OC [0 0 1] /C [0 1 0]", Some("0 0 1 RG")), + ("/OC [0 0 1] /IC [0 0 0]", Some("0 0 1 RG")), + ("", None), + (&format!("/OC [0 0 1] {quad}"), Some("0 0 1 RG")), + ]; + for (extra, colour) in cases { + let annot = format!("<< /Type /Annot /Subtype /Redact /Rect [60 420 260 520] {extra} >>"); + match colour { + Some(colour) => { + let (ap, _) = display_of(&annot); + assert!(ap.starts_with(&format!("{colour}\n1 w\n")) && ap.ends_with(" h S\n") && !ap.contains(" f\n"), "{extra}: {ap}"); + } + None => { + let doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R] >>", + annot.as_bytes(), + ]); + assert!(with_missing_appearances(&doc).is_none(), "{extra}: drawn"); + } + } + } +} + +/// New text re-fits the box in the face and size it is drawn in (`/DA`, or `/DS` where it sets +/// one), so the rectangle holds every drawn line. +#[test] +fn new_text_refits_the_box_in_its_drawn_face_and_size() { + let bottom = |annot: &str, new_text: &str| { + let mut doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R] >>", + annot.as_bytes(), + ]); + set_contents(&mut doc, 0, 0, new_text, &meta("")).unwrap(); + let d = &list(&doc, 0)[0]; + let rect: Vec = d.get(b"Rect").and_then(|r| r.as_array()).unwrap().iter().filter_map(|x| x.as_f64()).collect(); + assert_eq!(rect[3], 700.0, "the top edge stays"); + (rect[1], ap_content(&doc, d).matches(" Tj").count()) + }; + // The default 1-pt border: a 3-pt pad, so 114 pt for the text; the box is lines × 1.2 × size + 8 tall. + let words = "iiiiiiiiii iiiiiiiiii"; + let fits = |b: f64, want: f64| (b - want).abs() < 1e-9; + let (b, n) = bottom("<< /Type /Annot /Subtype /FreeText /Rect [45 630 165 700] /DA (0 g /Helv 12 Tf) /Contents (Hi) >>", words); + assert!(n == 1 && fits(b, 700.0 - 14.4 - 8.0), "Helvetica: one line, bottom {b}"); + let (b, n) = bottom("<< /Type /Annot /Subtype /FreeText /Rect [45 630 165 700] /DA (0 g /Cour 12 Tf) /Contents (Hi) >>", words); + assert!(n == 2 && fits(b, 700.0 - 2.0 * 14.4 - 8.0), "Courier: two lines, bottom {b}"); + let (b, n) = + bottom("<< /Type /Annot /Subtype /FreeText /Rect [45 630 165 700] /DA (0 g /Helv 12 Tf) /DS (font: Helvetica 24pt) /Contents (Hi) >>", "Hi"); + assert!(n == 1 && fits(b, 700.0 - 28.8 - 8.0), "/DS 24 pt: bottom {b}"); +} + +/// A colour or size change keeps the face a text box is drawn in: `/DA` keeps naming its font +/// resource (here Courier, and a `/DR` font that is Times Bold), and `/DS` says the same face +/// (ISO 32000-2 Table 177). +#[test] +fn a_colour_or_size_change_keeps_the_drawn_face() { + use pdfcraft_fonts::Std14; + let doc = |da: &str| { + let annot = format!("<< /Type /Annot /Subtype /FreeText /Rect [45 500 300 700] /Contents (Hi) /DA ({da}) >>"); + document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R /AcroForm << /DR << /Font << /F1 << /Type /Font /Subtype /Type1 /BaseFont /Times-Bold >> >> >> >> >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R] >>", + annot.as_bytes(), + ]) + }; + let face = |doc: &Document| { + let d = &list(doc, 0)[0]; + text_style::text_style(d, text_style::dr_face(doc, d)).face + }; + for (da, font, want) in [("0 g /Cour 12 Tf", "/Cour", Std14::Courier), ("0 g /F1 12 Tf", "/F1", Std14::TimesBold)] { + let mut d = doc(da); + assert_eq!(face(&d), want, "{da}"); + set_style(&mut d, 0, 0, StyleChange::new(Some([1.0, 0.0, 0.0]), None, None, None), &meta("")).unwrap(); + assert_eq!(face(&d), want, "colour change, {da}"); + assert!(text(&list(&d, 0)[0], b"DA").contains(&format!("{font} 12 Tf")), "{da}"); + set_font_size(&mut d, 0, 0, 18.0, &meta("")).unwrap(); + assert_eq!(face(&d), want, "size change, {da}"); + let p = props(&d, 0, 0).unwrap(); + assert_eq!((p.color, p.font_size), (Some([1.0, 0.0, 0.0]), Some(18.0)), "{da}"); + } + // The Courier box's appearance draws Courier. + let mut d = doc("0 g /Cour 12 Tf"); + set_style(&mut d, 0, 0, StyleChange::new(Some([1.0, 0.0, 0.0]), None, None, None), &meta("")).unwrap(); + assert!(ap_content(&d, &list(&d, 0)[0]).contains("/Cour 12 Tf")); +} + +/// A text box without an appearance is drawn from its `/DA` however the file wrote it: a `Tf` +/// without a size, a zero size, a font name the form's `/DR` doesn't list, and hostile operands. +/// The stream names the face the box is drawn in at a usable size, and no number in it is `NaN` +/// or infinite. +#[test] +fn text_boxes_draw_from_any_da_string() { + let doc = |da: &str| { + let annot = format!("<< /Type /Annot /Subtype /FreeText /Rect [45 500 300 700] /Contents (Hi) /DA ({da}) >>"); + document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R /AcroForm << /DR << /Font << /F1 << /Type /Font /Subtype /Type1 /BaseFont /Times-Bold >> >> >> >> >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R] >>", + annot.as_bytes(), + ]) + }; + for (da, tf, colour) in [ + ("0 g /Cour Tf", "/Cour 12 Tf", "0 0 0 rg"), + ("0 g /Cour 0 Tf", "/Cour 12 Tf", "0 0 0 rg"), + ("0 g /Cour -5 Tf", "/Cour 12 Tf", "0 0 0 rg"), + ("NaN NaN NaN rg /Cour NaN Tf", "/Cour 12 Tf", "0 0 0 rg"), + ("inf g /Cour inf Tf", "/Cour 12 Tf", "0 0 0 rg"), + ("1 0 0 rg /Cour 1e300 Tf", "/Cour 400 Tf", "1 0 0 rg"), + // In `/DR`: Times Bold. Not in `/DR` and no standard name: Helvetica. + ("0 g /F1 Tf", "/TiBo 12 Tf", "0 0 0 rg"), + ("0 g /F9 11 Tf", "/Helv 11 Tf", "0 0 0 rg"), + // No Tf at all, or garbage. + ("0 0 1 rg", "/Helv 12 Tf", "0 0 1 rg"), + ("] [ << >> Tf", "/Helv 12 Tf", "0 0 0 rg"), + ] { + let mut d = doc(da); + set_appearance(&mut d, ObjRef::new(4, 0)).unwrap(); + let content = ap_content(&d, &list(&d, 0)[0]); + assert!(content.contains(tf), "{da:?}: {content}"); + assert!(content.contains(colour), "{da:?}: {content}"); + let lower = content.to_ascii_lowercase(); + assert!(!lower.contains("nan") && !lower.contains("inf"), "{da:?}: {content}"); + } +} + +/// A callout with its text box at [300 600 500 640], knee [260 620] and arrow point [200 520]: +/// the knee line runs level from the box's left side, as Acrobat draws it. +fn callout_fixture() -> (Document, usize) { + let mut doc = fixture(); + let c = add_annotation( + &mut doc, + &new( + 0, + Shape::Callout { + rect: [300.0, 600.0, 500.0, 640.0], + knee: [260.0, 620.0], + point: [200.0, 520.0], + font_size: 12.0, + ending: LineEnding::OpenArrow, + }, + ), + &meta("c"), + ) + .unwrap(); + (doc, c) +} + +fn nums(d: &Dict, k: &[u8]) -> Vec { + d.get(k).unwrap().as_array().unwrap().iter().map(|o| o.as_f64().unwrap()).collect() +} + +/// The text box a callout's `/Rect` and `/RD` describe. +fn callout_box_of(d: &Dict) -> [f64; 4] { + let (r, rd) = (rect(d), nums(d, b"RD")); + [r[0] + rd[0], r[1] + rd[1], r[2] - rd[2], r[3] - rd[3]] +} + +/// #1152: a callout's arrow point and knee move on their own, as its handles drag them (Acrobat: +/// "Callout text boxes have three parts: a text box, a knee line, and an end-point line. You can +/// resize each part by dragging a handle", Acrobat DC Help, 11/28/2019, p. 357). The text box +/// stays and the line re-attaches to it. +#[test] +fn a_callouts_arrow_point_and_knee_move_on_their_own() { + let (mut doc, c) = callout_fixture(); + // The arrow point alone: the knee and the box stay. + set_points(&mut doc, 0, c, &[[150.0, 400.0], [260.0, 620.0]], &meta("p")).unwrap(); + let doc = reopen(&doc); + let d = &list(&doc, 0)[c]; + assert_eq!(nums(d, b"CL"), [150.0, 400.0, 260.0, 620.0, 300.0, 620.0]); + assert_eq!(callout_box_of(d), [300.0, 600.0, 500.0, 640.0]); + // /Rect holds the line (padded) and the box. + let r = rect(d); + assert!(r[0] < 150.0 && r[1] < 400.0 && r[2] >= 500.0 && r[3] >= 640.0, "{r:?}"); + assert!(ap_content(&doc, d).contains("150 400 m"), "the line starts at the new arrow point"); + // The knee alone, to the right of the box: the line now meets the right side. + let mut doc = doc; + set_points(&mut doc, 0, c, &[[150.0, 400.0], [560.0, 620.0]], &meta("k")).unwrap(); + let d = &list(&doc, 0)[c]; + assert_eq!(nums(d, b"CL"), [150.0, 400.0, 560.0, 620.0, 500.0, 620.0]); + assert_eq!(callout_box_of(d), [300.0, 600.0, 500.0, 640.0]); + assert_eq!(summaries(&doc).iter().find(|s| s.index == c).map(|s| s.points.clone()), Some(vec![[150.0, 400.0], [560.0, 620.0]])); +} + +/// A callout's leader line as its box moves or is resized, as Acrobat Pro DC (a 2021 Continuous +/// release) was seen to keep it (#1152): the arrow point never moves ("The text box moves around +/// a stationary anchor point", Acrobat DC Help, 11/28/2019, p. 357); moved, the knee and the end +/// come along; +/// resized with the side the line meets left in place, nothing changes; that side moved, the line +/// is laid out again from the side facing the arrow point with a 12 pt knee line, and the box stays +/// where it was put (Acrobat moves it next to the arrow point). +#[test] +fn a_callouts_box_moves_around_its_arrow_and_keeps_the_knee_beside_it() { + let (mut doc, c) = callout_fixture(); + let cl = |doc: &Document| nums(&list(doc, 0)[c], b"CL"); + // Wider and taller, its left side where it was: the line stays. + set_rect(&mut doc, 0, c, [300.0, 580.0, 540.0, 640.0], &meta("r")).unwrap(); + assert_eq!(cl(&doc), [200.0, 520.0, 260.0, 620.0, 300.0, 620.0]); + // Moved 40 right and 10 up, same size: the knee and the end come along. + set_rect(&mut doc, 0, c, [340.0, 590.0, 580.0, 650.0], &meta("m")).unwrap(); + assert_eq!(cl(&doc), [200.0, 520.0, 300.0, 630.0, 340.0, 630.0]); + // The left side moved: the line meets its middle, the knee 12 pt out. + set_rect(&mut doc, 0, c, [380.0, 590.0, 580.0, 650.0], &meta("s")).unwrap(); + assert_eq!(cl(&doc), [200.0, 520.0, 368.0, 620.0, 380.0, 620.0]); + assert_eq!(callout_box_of(&list(&doc, 0)[c]), [380.0, 590.0, 580.0, 650.0], "the box stays where it was put"); + // Past the arrow point: the line leaves from the side facing it. + set_rect(&mut doc, 0, c, [100.0, 600.0, 180.0, 640.0], &meta("p")).unwrap(); + assert_eq!(cl(&doc), [200.0, 520.0, 192.0, 620.0, 180.0, 620.0]); +} + +/// The line is laid out again when the end no longer lies on the side it met (the box shrank past +/// it), and from the side facing the arrow point, not the old knee, which can end up over the box. +#[test] +fn a_callout_is_laid_out_again_when_its_end_leaves_the_box() { + let (mut doc, c) = callout_fixture(); + // Left side unmoved, but the box now starts above the end (300, 620). + set_rect(&mut doc, 0, c, [300.0, 625.0, 500.0, 640.0], &meta("a")).unwrap(); + assert_eq!(nums(&list(&doc, 0)[c], b"CL"), [200.0, 520.0, 288.0, 632.5, 300.0, 632.5]); + let (mut doc, c) = callout_fixture(); + // Over the old knee (260, 620), which would face the bottom; the arrow faces the left side. + set_rect(&mut doc, 0, c, [250.0, 590.0, 450.0, 640.0], &meta("b")).unwrap(); + assert_eq!(nums(&list(&doc, 0)[c], b"CL"), [200.0, 520.0, 238.0, 615.0, 250.0, 615.0]); +} + +/// Review on #4094: the page sends a callout's text box back as `f32` values, so a box moved or +/// resized on screen comes back a few hundred-thousandths of a point off. That still counts as a +/// move (the knee comes along) and as the same side (the line stays), not as a new layout. +#[test] +fn a_callout_box_sent_back_as_f32_still_moves_and_keeps_its_line() { + let mut doc = fixture(); + let c = add_annotation( + &mut doc, + &new( + 0, + Shape::Callout { + rect: [323.05, 292.14, 506.98, 374.91], + knee: [311.05, 333.53], + point: [299.05, 274.83], + font_size: 12.0, + ending: LineEnding::OpenArrow, + }, + ), + &meta("c"), + ) + .unwrap(); + let f32s = |r: [f64; 4]| r.map(|v| f64::from(v as f32)); + let cl = |doc: &Document| nums(&list(doc, 0)[c], b"CL"); + let before = cl(&doc); + // Resized from the bottom right, its left side where it was. + set_rect(&mut doc, 0, c, f32s([323.05, 250.14, 540.98, 374.91]), &meta("r")).unwrap(); + assert_eq!(cl(&doc), before, "the line stays"); + // Moved by (47.62, 26.13), same size. + let tb = callout_box_of(&list(&doc, 0)[c]); + set_rect(&mut doc, 0, c, f32s([tb[0] + 47.62, tb[1] + 26.13, tb[2] + 47.62, tb[3] + 26.13]), &meta("m")).unwrap(); + let after = cl(&doc); + assert!((after[2] - (before[2] + 47.62)).abs() < 0.01 && (after[3] - (before[3] + 26.13)).abs() < 0.01, "the knee came along: {after:?}"); + assert_eq!((after[0], after[1]), (before[0], before[1]), "the arrow point stayed"); +} + +/// Review on #4094: a `/CL` with something other than a number in it is not read at all, rather +/// than read with its numbers shifted into the wrong points. +#[test] +fn a_callout_line_with_a_non_number_is_not_read() { + let (mut doc, c) = callout_fixture(); + let (_, r) = annot_ref(&mut doc, 0, c).unwrap(); + doc.update_dict(r, |d| { + d.set( + b"CL".to_vec(), + Object::Array(vec![Object::name("x"), Object::Real(200.0), Object::Real(520.0), Object::Real(260.0), Object::Real(620.0)]), + ) + }) + .unwrap(); + assert!(matches!(set_points(&mut doc, 0, c, &[[1.0, 1.0]], &meta("x")), Err(AnnotError::Invalid(_))), "not a callout line it can reshape"); + assert_eq!(summaries(&doc).into_iter().find(|s| s.index == c).map(|s| (s.points.len(), s.text_box.is_some())), Some((0, false))); +} + +/// A new callout's knee is Acrobat's: 12 pt out from the middle of the side facing the arrow. +#[test] +fn a_new_callouts_knee_line_is_twelve_points() { + let tb = [300.0, 600.0, 500.0, 640.0]; + assert_eq!(callout_knee(tb, [200.0, 520.0]), [288.0, 620.0]); + assert_eq!(callout_knee(tb, [600.0, 700.0]), [512.0, 620.0]); + assert_eq!(callout_knee(tb, [400.0, 700.0]), [400.0, 652.0]); + assert_eq!(callout_knee(tb, [400.0, 500.0]), [400.0, 588.0]); +} + +/// ISO 32000-2 Table 177: `/CL` may have four numbers (no knee). Resizing such a callout kept its +/// line only with six, so the box lost `/RD` and the line was left outside `/Rect`. +#[test] +fn a_callout_without_a_knee_keeps_its_line() { + let (mut doc, c) = callout_fixture(); + let (_, r) = annot_ref(&mut doc, 0, c).unwrap(); + doc.update_dict(r, |d| d.set(b"CL".to_vec(), num_array(&[200.0, 520.0, 300.0, 620.0]))).unwrap(); + set_rect(&mut doc, 0, c, [300.0, 580.0, 500.0, 640.0], &meta("r")).unwrap(); + let d = &list(&doc, 0)[c]; + assert_eq!(nums(d, b"CL"), [200.0, 520.0, 300.0, 620.0], "still four numbers, the left side unmoved"); + assert_eq!(callout_box_of(d), [300.0, 580.0, 500.0, 640.0]); + assert!(rect(d)[1] < 520.0, "/Rect holds the line"); + // Its one point is the arrow; a second adds a knee. + assert_eq!(summaries(&doc).iter().find(|s| s.index == c).map(|s| s.points.clone()), Some(vec![[200.0, 520.0]])); + set_points(&mut doc, 0, c, &[[210.0, 500.0]], &meta("p")).unwrap(); + assert_eq!(nums(&list(&doc, 0)[c], b"CL"), [210.0, 500.0, 300.0, 620.0]); + set_points(&mut doc, 0, c, &[[210.0, 500.0], [250.0, 610.0]], &meta("k")).unwrap(); + assert_eq!(nums(&list(&doc, 0)[c], b"CL"), [210.0, 500.0, 250.0, 610.0, 300.0, 620.0]); + // Back to no knee, then the left side moved: the end meets its middle. + let (_, r) = annot_ref(&mut doc, 0, c).unwrap(); + doc.update_dict(r, |d| d.set(b"CL".to_vec(), num_array(&[210.0, 500.0, 300.0, 620.0]))).unwrap(); + set_rect(&mut doc, 0, c, [280.0, 590.0, 500.0, 640.0], &meta("s")).unwrap(); + assert_eq!(nums(&list(&doc, 0)[c], b"CL"), [210.0, 500.0, 280.0, 615.0]); +} + +#[test] +fn a_callout_refuses_points_it_cant_have() { + let (mut doc, c) = callout_fixture(); + let before = list(&doc, 0)[c].clone(); + for bad in [&[][..], &[[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]][..], &[[f64::NAN, 1.0]][..], &[[1e9, 1.0]][..]] { + assert!(matches!(set_points(&mut doc, 0, c, bad, &meta("x")), Err(AnnotError::Invalid(_))), "{bad:?}"); + } + assert_eq!(list(&doc, 0)[c], before, "left as it was"); + // Without a readable /RD there is no box to keep, so nothing moves. + let (_, r) = annot_ref(&mut doc, 0, c).unwrap(); + doc.update_dict(r, |d| { + d.remove(b"RD"); + }) + .unwrap(); + assert!(matches!(set_points(&mut doc, 0, c, &[[1.0, 1.0]], &meta("x")), Err(AnnotError::Invalid(_)))); + // Nor with an /RD that puts the box outside /Rect (negative) or leaves no box (under 1 pt): + // set_points refuses, and summaries offer no text box to drag. + let outer = rect(&list(&doc, 0)[c]); + for rd in [[-5.0, 50.0, 0.0, 0.0], [0.0, 0.0, outer[2] - outer[0] - 0.5, 0.0]] { + doc.update_dict(r, |d| d.set(b"RD".to_vec(), num_array(&rd))).unwrap(); + assert!(matches!(set_points(&mut doc, 0, c, &[[1.0, 1.0]], &meta("x")), Err(AnnotError::Invalid(_))), "{rd:?}"); + assert_eq!(summaries(&doc).into_iter().find(|s| s.index == c).and_then(|s| s.text_box), None, "{rd:?}"); + } +} + +/// The side a knee faces: left or right of the box first, then above or below. A knee on the line +/// of the left or right edge past the box is above or below it; the ties are in +/// `a_callouts_knee_has_one_side_for_every_tie`. +#[test] +fn a_callouts_knee_faces_the_side_it_is_beyond() { + let tb = [100.0, 100.0, 200.0, 150.0]; + assert_eq!(callout_side(tb, [99.0, 125.0]), Side::Left); + assert_eq!(callout_side(tb, [201.0, 125.0]), Side::Right); + assert_eq!(callout_side(tb, [100.0, 160.0]), Side::Top, "on the left edge's line"); + assert_eq!(callout_side(tb, [200.0, 90.0]), Side::Bottom, "on the right edge's line"); + assert_eq!(callout_side(tb, [150.0, 125.0]), Side::Bottom, "inside"); + assert!(Side::Left.horizontal() && Side::Right.horizontal() && !Side::Top.horizontal() && !Side::Bottom.horizontal()); +} + +#[test] +fn summaries_give_a_callouts_points_and_text_box() { + let (doc, c) = callout_fixture(); + let s = summaries(&doc).into_iter().find(|s| s.index == c).unwrap(); + assert_eq!(s.points, vec![[200.0, 520.0], [260.0, 620.0]]); + assert_eq!(s.text_box, Some([300.0, 600.0, 500.0, 640.0])); + let square = summaries(&doc).into_iter().find(|s| s.subtype == "Square").unwrap(); + assert_eq!((square.points.len(), square.text_box), (0, None)); +} + +/// A callout whose text box can't be read (no `/RD`, a negative inset, or no box left inside +/// `/Rect`) lists no points either, as `pdfcraft_render::inspect` does: `set_points` refuses it. +#[test] +fn summaries_give_no_callout_points_without_a_text_box() { + for rd in [None, Some([-1.0, 0.0, 0.0, 0.0]), Some([0.0, 0.0, 0.0, 1000.0])] { + let (mut doc, c) = callout_fixture(); + let (_, r) = annot_ref(&mut doc, 0, c).unwrap(); + doc.update_dict(r, |d| match rd { + Some(rd) => d.set(b"RD".to_vec(), num_array(&rd)), + None => { + d.remove(b"RD"); + } + }) + .unwrap(); + let s = summaries(&doc).into_iter().find(|s| s.index == c).unwrap(); + assert_eq!((s.points.len(), s.text_box), (0, None), "/RD {rd:?}"); + } +} + +/// Review on #4094: a callout given its arrow point alone keeps its knee. The new line was built +/// from the one point and the end, so a six-number `/CL` silently became four. +#[test] +fn a_callouts_arrow_point_alone_keeps_its_knee() { + let (mut doc, c) = callout_fixture(); + let before = nums(&list(&doc, 0)[c], b"CL"); + assert_eq!(before.len(), 6, "{before:?}"); + set_points(&mut doc, 0, c, &[[190.0, 500.0]], &meta("p")).unwrap(); + let after = nums(&list(&doc, 0)[c], b"CL"); + assert_eq!(after.len(), 6, "the knee stays: {after:?}"); + assert_eq!(after[..2], [190.0, 500.0], "the arrow point moved"); + assert_eq!(after[2..], before[2..], "the knee and the end are where they were"); +} + +/// Review on #4094: every tie has one defined side. Beside the box (left or right of it) wins over +/// above or below it, so a knee off a corner diagonally, or on the line of the top or bottom edge +/// past the box, takes the left or right side. On the box's outline a knee takes the side whose +/// edge it lies on (within 0.01 pt), and a corner the left or right side, as beside the box. +/// Inside the box, or with a coordinate that isn't a number, it takes the bottom side. +#[test] +fn a_callouts_knee_has_one_side_for_every_tie() { + let tb = [100.0, 100.0, 200.0, 150.0]; + let cases: [([f64; 2], Side, &str); 30] = [ + ([100.004, 125.0], Side::Left, "on the left edge, give or take an f32"), + ([199.996, 125.0], Side::Right, "on the right edge, give or take an f32"), + ([150.0, 149.996], Side::Top, "on the top edge, give or take an f32"), + ([100.02, 125.0], Side::Bottom, "inside, past the slack"), + ([150.0, 125.0], Side::Bottom, "inside, at the middle"), + ([110.0, 145.0], Side::Bottom, "inside, near the top-left corner"), + ([100.0, 125.0], Side::Left, "on the left edge"), + ([200.0, 125.0], Side::Right, "on the right edge"), + ([150.0, 150.0], Side::Top, "on the top edge"), + ([150.0, 100.0], Side::Bottom, "on the bottom edge"), + ([100.0, 150.0], Side::Left, "the top-left corner"), + ([200.0, 150.0], Side::Right, "the top-right corner"), + ([100.0, 100.0], Side::Left, "the bottom-left corner"), + ([200.0, 100.0], Side::Right, "the bottom-right corner"), + ([90.0, 150.0], Side::Left, "the top edge's line, left of the box"), + ([210.0, 150.0], Side::Right, "the top edge's line, right of the box"), + ([90.0, 100.0], Side::Left, "the bottom edge's line, left of the box"), + ([100.0, 160.0], Side::Top, "the left edge's line, above the box"), + ([100.0, 90.0], Side::Bottom, "the left edge's line, below the box"), + ([200.0, 160.0], Side::Top, "the right edge's line, above the box"), + ([90.0, 160.0], Side::Left, "diagonally off the top-left corner"), + ([210.0, 90.0], Side::Right, "diagonally off the bottom-right corner"), + ([f64::NAN, 125.0], Side::Bottom, "x not a number"), + ([150.0, f64::NAN], Side::Bottom, "y not a number"), + ([f64::NAN, 160.0], Side::Bottom, "x not a number, y above the box"), + ([f64::NAN, 90.0], Side::Bottom, "x not a number, y below the box"), + ([90.0, f64::NAN], Side::Bottom, "y not a number, x left of the box"), + ([210.0, f64::NAN], Side::Bottom, "y not a number, x right of the box"), + ([f64::NEG_INFINITY, 125.0], Side::Left, "infinitely far left"), + ([150.0, f64::INFINITY], Side::Top, "infinitely far up"), + ]; + for (knee, side, what) in cases { + assert_eq!(callout_side(tb, knee), side, "{what}: {knee:?}"); + // Written corners first, the box is the same box. + assert_eq!(callout_side([200.0, 150.0, 100.0, 100.0], knee), side, "{what}, corners first"); + // The line meets the middle of that side. + let want = match side { + Side::Left => [100.0, 125.0], + Side::Right => [200.0, 125.0], + Side::Top => [150.0, 150.0], + Side::Bottom => [150.0, 100.0], + }; + assert_eq!(callout_attach(tb, knee), want, "{what}"); + } +} + +/// A knee dragged exactly onto the box's left edge keeps the line on the left side, where its end +/// already is, instead of sending it across the box to the bottom; a knee dragged inside the box +/// meets the bottom side. +#[test] +fn a_callouts_knee_on_or_inside_the_box_meets_a_defined_side() { + let (mut doc, c) = callout_fixture(); + set_points(&mut doc, 0, c, &[[200.0, 520.0], [300.0, 630.0]], &meta("edge")).unwrap(); + assert_eq!(nums(&list(&doc, 0)[c], b"CL"), [200.0, 520.0, 300.0, 630.0, 300.0, 620.0]); + set_points(&mut doc, 0, c, &[[200.0, 520.0], [350.0, 620.0]], &meta("inside")).unwrap(); + assert_eq!(nums(&list(&doc, 0)[c], b"CL"), [200.0, 520.0, 350.0, 620.0, 400.0, 600.0]); + assert_eq!(callout_box_of(&list(&doc, 0)[c]), [300.0, 600.0, 500.0, 640.0], "the box stays"); +} + +/// A callout's `/CL` read as numbers, and the check every edit must pass: four numbers (arrow +/// point, end) or six (arrow point, knee, end), all finite (ISO 32000-2 Table 177), every point +/// inside `/Rect`, and a text box (`/Rect` inset by `/RD`) at least 1 pt a side. +fn assert_callout_line(d: &Dict, len: usize, what: &str) -> Vec { + let cl = nums(d, b"CL"); + assert_eq!(cl.len(), len, "{what}: {cl:?}"); + assert!(cl.iter().all(|v| v.is_finite()), "{what}: {cl:?}"); + let r = rect(d); + for p in cl.chunks(2) { + assert!(p[0] >= r[0] - 1e-6 && p[0] <= r[2] + 1e-6 && p[1] >= r[1] - 1e-6 && p[1] <= r[3] + 1e-6, "{what}: {p:?} outside {r:?}"); + } + let tb = callout_box_of(d); + assert!(tb[2] - tb[0] >= 1.0 && tb[3] - tb[1] >= 1.0, "{what}: text box {tb:?}"); + cl +} + +/// Review on #4094: every edit of a callout keeps `/CL` four numbers when it had four and six when +/// it had six: the arrow point dragged, the knee dragged, the box moved, the box resized, the whole +/// callout moved, and the file saved and opened again. +#[test] +fn every_callout_edit_keeps_four_or_six_numbers() { + for len in [4usize, 6] { + let (mut doc, c) = callout_fixture(); + if len == 4 { + let (_, r) = annot_ref(&mut doc, 0, c).unwrap(); + doc.update_dict(r, |d| d.set(b"CL".to_vec(), num_array(&[200.0, 520.0, 300.0, 620.0]))).unwrap(); + } + let cl = |doc: &Document, what: &str| assert_callout_line(&list(doc, 0)[c], len, &format!("{len}: {what}")); + set_points(&mut doc, 0, c, &[[150.0, 400.0]], &meta("tip")).unwrap(); + assert_eq!(cl(&doc, "the arrow point")[..2], [150.0, 400.0]); + if len == 6 { + set_points(&mut doc, 0, c, &[[150.0, 400.0], [260.0, 630.0]], &meta("knee")).unwrap(); + assert_eq!(cl(&doc, "the knee")[2..4], [260.0, 630.0]); + } + let tb = callout_box_of(&list(&doc, 0)[c]); + set_rect(&mut doc, 0, c, [tb[0] + 40.0, tb[1] + 10.0, tb[2] + 40.0, tb[3] + 10.0], &meta("box")).unwrap(); + assert_eq!(cl(&doc, "the box moved")[..2], [150.0, 400.0], "the arrow point stays"); + let tb = callout_box_of(&list(&doc, 0)[c]); + set_rect(&mut doc, 0, c, [tb[0] + 30.0, tb[1] - 20.0, tb[2] + 10.0, tb[3]], &meta("size")).unwrap(); + let before = cl(&doc, "the box resized"); + move_annotation(&mut doc, 0, c, 20.0, -15.0, &meta("move")).unwrap(); + let after = cl(&doc, "the callout moved"); + let shifted: Vec = before.iter().enumerate().map(|(i, v)| v + if i % 2 == 0 { 20.0 } else { -15.0 }).collect(); + assert_eq!(after, shifted, "{len}: every point moves with it"); + let doc = reopen(&doc); + cl(&doc, "saved and opened again"); + } +} + +/// Review on #4094: a `/CL` that is neither four nor six finite numbers (two, three, five or eight +/// numbers, an infinite or not-a-number value, an empty array, a nested array, a name or a string) +/// is not a callout line PdfCraft can reshape. No edit panics, the arrow point and knee are +/// refused, and none writes a `/CL` of another length: resizing leaves it as it was and moving +/// shifts its numbers in place. A `/CL` held as an indirect object is read as the line it is. +#[test] +fn a_hostile_callout_line_is_never_rewritten_into_another_shape() { + let reals = |v: &[f64]| Object::Array(v.iter().map(|x| Object::Real(*x)).collect()); + let hostile: Vec<(&str, Object)> = vec![ + ("two numbers", reals(&[200.0, 520.0])), + ("three numbers", reals(&[200.0, 520.0, 260.0])), + ("five numbers", reals(&[200.0, 520.0, 260.0, 620.0, 300.0])), + ("eight numbers", reals(&[200.0, 520.0, 260.0, 620.0, 280.0, 620.0, 300.0, 620.0])), + ("not a number", reals(&[200.0, f64::NAN, 260.0, 620.0, 300.0, 620.0])), + ("infinite", reals(&[f64::INFINITY, 520.0, 260.0, 620.0])), + ("empty", Object::Array(Vec::new())), + ("nested", Object::Array(vec![reals(&[200.0, 520.0]), reals(&[300.0, 620.0])])), + ("a name", Object::name("CL")), + ("a string", PdfString::text("200 520 300 620").into()), + ]; + for (what, cl) in hostile { + let (mut doc, c) = callout_fixture(); + let (_, r) = annot_ref(&mut doc, 0, c).unwrap(); + doc.update_dict(r, |d| d.set(b"CL".to_vec(), cl.clone())).unwrap(); + let len = cl.as_array().map(Vec::len); + for points in [&[[150.0, 400.0]][..], &[[150.0, 400.0], [260.0, 630.0]][..]] { + assert!(set_points(&mut doc, 0, c, points, &meta("p")).is_err(), "{what}: {points:?}"); + } + // Compared as written: a not-a-number value is never equal to itself. + let same = |doc: &Document| format!("{:?}", list(doc, 0)[c].get(b"CL")) == format!("{:?}", Some(&cl)); + assert!(same(&doc), "{what}: refused, so unchanged"); + let s = summaries(&doc).into_iter().find(|s| s.index == c).unwrap(); + assert_eq!((s.points.len(), s.text_box), (0, None), "{what}: nothing to drag"); + let _ = set_rect(&mut doc, 0, c, [300.0, 580.0, 520.0, 640.0], &meta("r")); + assert!(same(&doc), "{what}: resizing leaves it as it was"); + let _ = move_annotation(&mut doc, 0, c, 20.0, -15.0, &meta("m")); + assert_eq!(list(&doc, 0)[c].get(b"CL").and_then(Object::as_array).map(Vec::len), len, "{what}: moved in place"); + reopen(&doc); + } + // Held as an indirect object, a six-number line is a callout line like any other. + let (mut doc, c) = callout_fixture(); + let line = doc.add(reals(&[200.0, 520.0, 260.0, 620.0, 300.0, 620.0])); + let (_, r) = annot_ref(&mut doc, 0, c).unwrap(); + doc.update_dict(r, |d| d.set(b"CL".to_vec(), Object::Ref(line))).unwrap(); + set_points(&mut doc, 0, c, &[[150.0, 400.0]], &meta("p")).unwrap(); + assert_eq!(assert_callout_line(&list(&doc, 0)[c], 6, "indirect"), [150.0, 400.0, 260.0, 620.0, 300.0, 620.0]); +} + +/// A pop-up is opened, closed and moved through `set_popup` (#comment.popups): a note's own +/// pop-up follows, a comment without one gets one beside it, and the comment itself (its text, +/// its date) is left alone. Summaries report the pop-up. Text boxes, Fill & Sign marks and +/// replies have none of their own; a bad rectangle is refused. +#[test] +fn pop_ups_open_close_and_move_and_are_reported() { + let mut doc = fixture(); + let note = add_annotation(&mut doc, &new(0, Shape::Note { at: [20.0, 280.0], icon: NoteIcon::Comment }), &meta("n1")).unwrap(); + let before = summaries(&doc).into_iter().find(|s| s.index == note).unwrap(); + assert_eq!(before.popup.map(|p| p.open), Some(false), "new notes have a closed pop-up"); + set_popup(&mut doc, 0, note, None, Some(true), &meta("n1")).unwrap(); + let after = summaries(&doc).into_iter().find(|s| s.index == note).unwrap(); + assert_eq!(after.popup.map(|p| p.open), Some(true)); + assert_eq!(after.popup.map(|p| p.rect), before.popup.map(|p| p.rect), "opening does not move it"); + assert_eq!((after.modified, after.contents.as_deref()), (before.modified, Some("hello")), "the comment is untouched"); + let (_, r) = annot_ref(&mut doc, 0, note).unwrap(); + assert_eq!(annot_dict(&doc, r).get(b"Open"), Some(&Object::Bool(true)), "the note's own /Open follows"); + set_popup(&mut doc, 0, note, Some([150.0, 100.0, 290.0, 200.0]), Some(false), &meta("n1")).unwrap(); + let moved = summaries(&doc).into_iter().find(|s| s.index == note).unwrap(); + assert_eq!(moved.popup, Some(PopupInfo { rect: [150.0, 100.0, 290.0, 200.0], open: false })); + assert_eq!(annot_dict(&doc, r).get(b"Open"), Some(&Object::Bool(false))); + + // A highlight is created without a pop-up: the first open gives it one, beside it. + let mark = add_annotation( + &mut doc, + &new(0, Shape::TextMarkup { kind: Markup::Highlight, quads: vec![[20.0, 200.0, 120.0, 200.0, 20.0, 190.0, 120.0, 190.0]] }), + &meta("h1"), + ) + .unwrap(); + assert_eq!(summaries(&doc).into_iter().find(|s| s.index == mark).unwrap().popup, None); + set_popup(&mut doc, 0, mark, None, Some(true), &meta("h1")).unwrap(); + let given = summaries(&doc).into_iter().find(|s| s.index == mark).unwrap().popup.expect("a pop-up now"); + assert!(given.open); + assert!((given.rect[0] - 130.0).abs() < 0.01 && (given.rect[2] - 330.0).abs() < 0.01, "beside the mark: {:?}", given.rect); + + // #2137 review: a /Popup naming another annotation (here the note) isn't this comment's: the + // note is left alone and the mark gets a pop-up of its own. + let (_, mr) = annot_ref(&mut doc, 0, mark).unwrap(); + doc.update_dict(mr, |m| m.set(b"Popup".to_vec(), Object::Ref(r))).unwrap(); + let note_before = annot_dict(&doc, r); + set_popup(&mut doc, 0, mark, Some([10.0, 10.0, 110.0, 60.0]), Some(true), &meta("h1")).unwrap(); + assert_eq!(annot_dict(&doc, r), note_before, "the note isn't moved or opened"); + let own = annot_dict(&doc, mr).reference(b"Popup").expect("a pop-up of its own"); + assert_ne!(own, r); + assert_eq!(doc.get(own).as_dict().and_then(|p| p.reference(b"Parent")), Some(mr)); + // Its own pop-up, with /Parent naming it, is the one moved next time. + set_popup(&mut doc, 0, mark, Some([12.0, 10.0, 112.0, 60.0]), None, &meta("h1")).unwrap(); + assert_eq!(annot_dict(&doc, mr).reference(b"Popup"), Some(own), "the same pop-up"); + + // #2137 review: nor is it reported as the comment's (see the helper). + foreign_pop_ups_are_not_the_marks(&mut doc, r, mark, mr); + + // None of their own. + let text_box = add_annotation(&mut doc, &new(0, Shape::TextBox { rect: [10.0, 10.0, 110.0, 40.0], font_size: 12.0 }), &meta("t1")).unwrap(); + assert!(matches!(set_popup(&mut doc, 0, text_box, None, Some(true), &meta("t1")), Err(AnnotError::Invalid(_)))); + let reply = add_reply(&mut doc, 0, note, "Agreed", "Ada", &meta("r1")).unwrap(); + assert!(matches!(set_popup(&mut doc, 0, reply, None, Some(true), &meta("r1")), Err(AnnotError::Invalid(_)))); + assert!(matches!(set_popup(&mut doc, 0, note, Some([0.0, 0.0, 0.5, 0.5]), None, &meta("n1")), Err(AnnotError::Invalid(_)))); + assert!(matches!(set_popup(&mut doc, 0, note, Some([f64::NAN, 0.0, 10.0, 10.0]), None, &meta("n1")), Err(AnnotError::Invalid(_)))); + assert!(matches!(set_popup(&mut doc, 0, 99, None, Some(true), &meta("n1")), Err(AnnotError::NoSuchAnnotation { .. }))); + + // #2137 review: /Annots may list the note through an object that is only a reference to it. + // Its pop-up's /Parent names the note itself, so the pop-up is still the note's. + let p = page_ref(&doc, 0).unwrap(); + let via = doc.add(Object::Ref(r)); + let list: Vec = annots(&doc, p).into_iter().map(|o| if o.as_ref() == Some(r) { Object::Ref(via) } else { o }).collect(); + set_annots(&mut doc, p, list).unwrap(); + let listed = summaries(&doc).into_iter().find(|s| s.index == note).unwrap(); + assert_eq!(listed.popup, Some(PopupInfo { rect: [150.0, 100.0, 290.0, 200.0], open: false }), "listed through {via:?}"); +} + +/// #2137 review: a pop-up that is not the mark's own is neither reported as its pop-up nor +/// changed through it (part of `pop_ups_open_close_and_move_and_are_reported`). +fn foreign_pop_ups_are_not_the_marks(doc: &mut Document, note: ObjRef, mark: usize, mr: ObjRef) { + // #2137 review: nor is it reported as the comment's. A /Popup naming the note, or the note's + // own pop-up (its /Parent is the note), gives the mark no pop-up in the summaries. + let popup_of_mark = |doc: &Document| summaries(doc).into_iter().find(|s| s.index == mark).unwrap().popup; + let notes_popup = annot_dict(doc, note).reference(b"Popup").expect("the note's pop-up"); + for foreign in [note, notes_popup] { + doc.update_dict(mr, |m| m.set(b"Popup".to_vec(), Object::Ref(foreign))).unwrap(); + assert_eq!(popup_of_mark(doc), None, "{foreign:?} isn't the mark's"); + } + // A pop-up whose /Parent isn't a reference (it must be one) isn't the mark's either: it is + // neither reported nor changed, and the mark gets one of its own. + let mut bad = Dict::new(); + bad.set(b"Type".to_vec(), Object::name("Annot")); + bad.set(b"Subtype".to_vec(), Object::name("Popup")); + bad.set(b"Rect".to_vec(), Object::Array([0.0, 0.0, 50.0, 50.0].iter().map(|v| Object::Real(*v)).collect())); + bad.set(b"Parent".to_vec(), Object::Dict(Dict::new())); + let bad = doc.add(Object::Dict(bad)); + doc.update_dict(mr, |m| m.set(b"Popup".to_vec(), Object::Ref(bad))).unwrap(); + assert_eq!(popup_of_mark(doc), None, "a direct /Parent"); + let bad_before = doc.get(bad).as_dict().cloned(); + set_popup(doc, 0, mark, None, Some(true), &meta("h1")).unwrap(); + assert_eq!(doc.get(bad).as_dict().cloned(), bad_before, "left alone"); + assert_ne!(annot_dict(doc, mr).reference(b"Popup"), Some(bad)); + // A pop-up with no /Parent at all is taken as the comment's. + let mut orphan = Dict::new(); + orphan.set(b"Type".to_vec(), Object::name("Annot")); + orphan.set(b"Subtype".to_vec(), Object::name("Popup")); + orphan.set(b"Rect".to_vec(), Object::Array([0.0, 0.0, 50.0, 50.0].iter().map(|v| Object::Real(*v)).collect())); + let orphan = doc.add(Object::Dict(orphan)); + doc.update_dict(mr, |m| m.set(b"Popup".to_vec(), Object::Ref(orphan))).unwrap(); + assert_eq!(popup_of_mark(doc), Some(PopupInfo { rect: [0.0, 0.0, 50.0, 50.0], open: false }), "no /Parent"); +} + +/// #2137 review: a comment's own pop-up without a `/Rect` of four numbers reads as none; opening +/// it gives it the place a new pop-up gets, so the window shows again. +#[test] +fn opening_a_pop_up_without_a_rect_gives_it_one() { + let mut doc = fixture(); + let note = add_annotation(&mut doc, &new(0, Shape::Note { at: [20.0, 280.0], icon: NoteIcon::Comment }), &meta("n1")).unwrap(); + let (_, r) = annot_ref(&mut doc, 0, note).unwrap(); + let pr = annot_dict(&doc, r).reference(b"Popup").expect("new notes have a pop-up"); + doc.update_dict(pr, |p| { + p.remove(b"Rect"); + }) + .unwrap(); + assert_eq!(summaries(&doc).into_iter().find(|s| s.index == note).unwrap().popup, None, "no /Rect reads as no pop-up"); + set_popup(&mut doc, 0, note, None, Some(true), &meta("n1")).unwrap(); + let popup = summaries(&doc).into_iter().find(|s| s.index == note).unwrap().popup.expect("the pop-up reads again"); + assert!(popup.open); + assert!(popup.rect[2] - popup.rect[0] >= 100.0 && popup.rect[3] - popup.rect[1] >= 100.0, "a window's size: {:?}", popup.rect); + assert_eq!(annot_dict(&doc, r).reference(b"Popup"), Some(pr), "the same pop-up, not a new one"); +} + +/// #2137 review: a pop-up rectangle with its corners reversed is read, and saved, with them in +/// order (ISO 32000-2 §7.9.5: a reader normalises a rectangle). One whose numbers aren't finite +/// as `f32` (a file's huge number) reads as no pop-up, and opening it gives it a new place. +#[test] +fn a_pop_up_rect_with_reversed_corners_is_put_in_order_and_a_huge_one_is_none() { + let mut doc = fixture(); + let note = add_annotation(&mut doc, &new(0, Shape::Note { at: [20.0, 280.0], icon: NoteIcon::Comment }), &meta("n1")).unwrap(); + let (_, r) = annot_ref(&mut doc, 0, note).unwrap(); + let pr = annot_dict(&doc, r).reference(b"Popup").expect("new notes have a pop-up"); + let popup = |doc: &Document| summaries(doc).into_iter().find(|s| s.index == note).unwrap().popup; + let set_rect = |doc: &mut Document, rect: [f64; 4]| doc.update_dict(pr, |p| p.set(b"Rect".to_vec(), num_array(&rect))).unwrap(); + + // Reversed in the file: read in order. + set_rect(&mut doc, [290.0, 200.0, 150.0, 100.0]); + assert_eq!(popup(&doc), Some(PopupInfo { rect: [150.0, 100.0, 290.0, 200.0], open: false })); + // Reversed through `set_popup`: saved in order. + set_popup(&mut doc, 0, note, Some([300.0, 220.0, 160.0, 120.0]), None, &meta("n1")).unwrap(); + let saved: Vec = + doc.get(pr).as_dict().and_then(|p| p.get(b"Rect")).and_then(Object::as_array).unwrap().iter().filter_map(Object::as_f64).collect(); + assert_eq!(saved, [160.0, 120.0, 300.0, 220.0]); + + // A number too large for `f32` (or infinite): no pop-up, and opening it gives it a place. + for huge in [1e300, f64::INFINITY] { + set_rect(&mut doc, [0.0, 0.0, huge, 100.0]); + assert_eq!(popup(&doc), None, "{huge}"); + } + set_popup(&mut doc, 0, note, None, Some(true), &meta("n1")).unwrap(); + let given = popup(&doc).expect("the pop-up reads again"); + assert!(given.open && given.rect.iter().all(|v| v.is_finite()), "{given:?}"); +} + #[test] fn freehand_highlight_is_compatible_ink_with_marker_blending() { let shape = Shape::FreehandHighlight { strokes: vec![vec![[20.0, 100.0], [120.0, 100.0]]] }; @@ -2786,6 +4934,500 @@ fn indirect_typewriter_names_keep_outer_whitespace_in_summaries() { check(&reopen(&doc)); } +/// PDFBOX-2699: a `/C` written `[1 0 0 null]` (Table 166 asks for numbers) is still a four-entry +/// colour, CMYK with the null as 0 as PDFium, MuPDF and Poppler read it, so a shape without an +/// appearance is drawn instead of dropped. `/IC` is read the same way, and three numbers stay RGB. +/// The properties and the comment list report the colours as drawn. +#[test] +fn a_four_entry_colour_with_a_null_is_cmyk_with_zero() { + let doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 100 40] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R 5 0 R] >>", + b"<< /Type /Annot /Subtype /Square /Rect [5 5 45 35] /C [1 0 0 null] /IC [0 0 1 null] >>", + b"<< /Type /Annot /Subtype /Square /Rect [55 5 95 35] /C [1 0 0] >>", + ]); + let display = with_missing_appearances(&doc).expect("both squares get a fallback appearance"); + let squares = list(&display, 0); + let cmyk = ap_content(&display, &squares[0]); + assert!(cmyk.contains("0 1 1 RG") && cmyk.contains("1 1 0 rg"), "CMYK (1,0,0,0) stroke and (0,0,1,0) fill: {cmyk}"); + let rgb = ap_content(&display, &squares[1]); + assert!(rgb.contains("1 0 0 RG"), "three numbers stay RGB: {rgb}"); + // The properties read the stroke and the fill as they are drawn. + let p = props(&doc, 0, 0).expect("the square has properties"); + assert_eq!((p.color, p.fill), (Some([0.0, 1.0, 1.0]), Some([1.0, 1.0, 0.0]))); + assert_eq!(props(&doc, 0, 1).and_then(|p| p.color), Some([1.0, 0.0, 0.0])); + // The comment list reports the same colours. + let colors: Vec> = summaries(&doc).iter().map(|s| s.color).collect(); + assert_eq!(colors, [Some([0.0, 1.0, 1.0]), Some([1.0, 0.0, 0.0])]); +} + +/// A `/C` given by reference is followed for the comment list, as three numbers were, and read +/// like a direct one: `[0 0 1 null]` is CMYK yellow, and a referenced grey level is grey. +#[test] +fn a_referenced_colour_is_listed_like_a_direct_one() { + let doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 100 40] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R 5 0 R] >>", + b"<< /Type /Annot /Subtype /Square /Rect [5 5 45 35] /C 6 0 R >>", + b"<< /Type /Annot /Subtype /Circle /Rect [55 5 95 35] /C 7 0 R >>", + b"[0 0 1 null]", + b"[0.5]", + ]); + let colors: Vec> = summaries(&doc).iter().map(|s| s.color).collect(); + assert_eq!(colors, [Some([1.0, 1.0, 0.0]), Some([0.5, 0.5, 0.5])]); +} + +/// A reference among the four entries is not a non-number read as 0: `/C [1 0 0 6 0 R]` with +/// object 6 the number 1 is CMYK black (§7.3.10), so drawing it cyan would be wrong. The display +/// and the properties follow the reference (`draw::with_direct_values`) and draw and +/// read it black; the comment list, which follows only a `/C` given whole by reference, reads no +/// colour rather than cyan. +#[test] +fn a_four_entry_colour_with_a_reference_is_not_read_as_zero() { + let doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 100 40] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R 5 0 R] >>", + b"<< /Type /Annot /Subtype /Square /Rect [5 5 45 35] /C [1 0 0 6 0 R] >>", + b"<< /Type /Annot /Subtype /Square /Rect [55 5 95 35] /C [1 0 0] >>", + b"1", + ]); + let display = with_missing_appearances(&doc).expect("both squares get a fallback appearance"); + let squares = list(&display, 0); + let black = ap_content(&display, &squares[0]); + assert!(black.contains("0 0 0 RG") && !black.contains("0 1 1 RG"), "CMYK black, no cyan stand-in: {black}"); + assert_eq!(props(&doc, 0, 0).and_then(|p| p.color), Some([0.0, 0.0, 0.0])); + assert_ne!(summaries(&doc)[0].color, Some([0.0, 1.0, 1.0]), "nor cyan in the comment list"); +} + +/// A text markup annotation is drawn over its `/QuadPoints` (ISO 32000-2 Table 182), and an +/// appearance is mapped onto `/Rect` (§12.5.5). A highlight whose `/Rect` lies beside its text +/// (here written with its corners swapped, as a real file has it) got an appearance cut to that +/// `/Rect`, so nothing showed. The display copy's appearance and `/Rect` now take in the +/// quadrilaterals; a markup whose `/Rect` already holds them keeps it. +#[test] +fn text_markup_whose_quads_reach_past_its_rect_is_drawn_over_them() { + let doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R 5 0 R 6 0 R 7 0 R 8 0 R 9 0 R 10 0 R] >>", + b"<< /Type /Annot /Subtype /Highlight /Rect [335.1 370.83 324.6 471.78] /C [1 1 0] /QuadPoints [370 287 471 287 370 277 471 277] >>", + b"<< /Type /Annot /Subtype /Underline /Rect [10 10 20 20] /C [0 0 1] /QuadPoints [100 120 200 120 100 100 200 100] >>", + b"<< /Type /Annot /Subtype /Highlight /Rect [50 50 150 70] /C [1 1 0] /QuadPoints [60 68 140 68 60 52 140 52] >>", + b"<< /Type /Annot /Subtype /StrikeOut /Rect [10 10 20 20] /C [1 0 0] /QuadPoints [100 220 200 220 100 200 200 200] >>", + b"<< /Type /Annot /Subtype /Squiggly /Rect [10 10 20 20] /C [0 1 0] /QuadPoints [100 320 200 320 100 300 200 300] >>", + b"<< /Type /Annot /Subtype /Highlight /Rect [10 10 20 20] /C [1 1 0] /QuadPoints [-1e30 20 1e30 20 -1e30 10 1e30 10] >>", + b"<< /Type /Annot /Subtype /Highlight /Rect [-1e308 10 1e308 20] /C [1 1 0] /QuadPoints [100 120 200 120 100 100 200 100] >>", + ]); + let display = with_missing_appearances(&doc).expect("the markups need an appearance"); + let shown = list(&display, 0); + let rect = |d: &Dict| nums_of(d, b"Rect"); + let bbox = |d: &Dict| { + let n = d.get(b"AP").map(|a| display.resolve(a)).and_then(|a| a.as_dict().and_then(|a| a.reference(b"N"))).unwrap(); + let obj = display.get(n); + let Object::Stream(s) = &*obj else { panic!("AP is not a stream") }; + nums_of(&s.dict, b"BBox") + }; + assert_eq!(rect(&shown[0]), [324.6, 277.0, 471.0, 471.78], "the highlight's /Rect takes in its text"); + assert_eq!(bbox(&shown[0]), rect(&shown[0])); + assert!(ap_content(&display, &shown[0]).contains("370 287 m 471 287 l 471 277 l 370 277 l h f"), "the quad is filled"); + // Half the underline's width (0.07 × 20 / 2) past the quad. + assert_eq!(rect(&shown[1]), [10.0, 10.0, 200.7, 120.7], "an underline's too"); + assert_eq!(bbox(&shown[1]), rect(&shown[1])); + assert_eq!(rect(&shown[2]), [50.0, 50.0, 150.0, 70.0], "a /Rect that holds the quads stays as it is"); + assert_eq!(rect(&shown[3]), [10.0, 10.0, 200.7, 220.7], "a strikeout's too"); + assert_eq!(bbox(&shown[3]), rect(&shown[3])); + assert_eq!(rect(&shown[4]), [10.0, 10.0, 200.5, 320.5], "a squiggly's too"); + assert_eq!(bbox(&shown[4]), rect(&shown[4])); + assert_eq!(rect(&shown[5]), [0.0, 10.0, 612.0, 20.0], "quads far past the page reach only as far as it shows"); + // A /Rect too wide for its width to be a number keeps its size: the on-page quad above it + // doesn't make it taller. + assert_eq!(rect(&shown[6]), [-1e308, 10.0, 1e308, 20.0], "an overflowing /Rect is not widened"); + assert_eq!(rect(&list(&doc, 0)[0]), [335.1, 370.83, 324.6, 471.78], "the source document is unchanged"); +} + +/// The box widens over the quads the markup is drawn over, read as every `/QuadPoints` reader +/// reads them (`pdfcraft_geom::quad_points::usable_quads`): an indirect array or entry is +/// followed, and a malformed quad or trailing partial one is skipped while the others still widen +/// the box, so what is drawn is not cut to `/Rect`. +#[test] +fn text_markup_reach_reads_quads_like_the_drawing() { + let doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R 5 0 R 6 0 R] >>", + b"<< /Type /Annot /Subtype /Highlight /Rect [10 10 20 20] /C [1 1 0] /QuadPoints 7 0 R >>", + b"<< /Type /Annot /Subtype /Highlight /Rect [10 10 20 20] /C [1 1 0] /QuadPoints [100 120 /oops 120 100 100 200 100 100 220 200 220 100 200 200 200 1 2 3] >>", + b"<< /Type /Annot /Subtype /Highlight /Rect [10 10 20 20] /C [1 1 0] /QuadPoints [100 120 200 120 100 100 8 0 R 100] >>", + b"[100 120 200 120 100 100 200 100]", + b"200", + ]); + let display = with_missing_appearances(&doc).expect("the markups need an appearance"); + let shown = list(&display, 0); + let rect = |d: &Dict| nums_of(d, b"Rect"); + assert_eq!(rect(&shown[0]), [10.0, 10.0, 200.0, 120.0], "an indirect array widens the box"); + assert_eq!(ap_bbox(&display, &shown[0]), rect(&shown[0])); + assert_eq!(rect(&shown[1]), [10.0, 10.0, 200.0, 220.0], "the usable quad still widens it"); + assert_eq!(ap_bbox(&display, &shown[1]), rect(&shown[1])); + assert_eq!(rect(&shown[2]), [10.0, 10.0, 200.0, 120.0], "an indirect entry is followed"); +} + +/// `n` copies of the quadrilateral `[100 120 200 120 100 100 200 100]`, as `/QuadPoints`. +fn repeated_quads(n: usize) -> String { + format!("[{}]", vec!["100 120 200 120 100 100 200 100"; n].join(" ")) +} + +/// The `/BBox` of `d`'s normal appearance in `doc`. +fn ap_bbox(doc: &Document, d: &Dict) -> Vec { + let n = d.get(b"AP").map(|a| doc.resolve(a)).and_then(|a| a.as_dict().and_then(|a| a.reference(b"N"))).expect("has /AP /N"); + let obj = doc.get(n); + let Object::Stream(s) = &*obj else { panic!("AP is not a stream") }; + nums_of(&s.dict, b"BBox") +} + +/// The widened box stops at the page's visible region (its crop box, ISO 32000-2 §14.11.2), and +/// quadrilaterals wholly outside it widen nothing. A markup with more than +/// `appearance::MAX_MARKUP_QUADS` quadrilaterals keeps its `/Rect`. An annotation with an +/// appearance of its own, and one that isn't text markup, keep theirs whatever `/QuadPoints` say. +#[test] +fn text_markup_reach_stops_at_the_page_and_the_quad_cap() { + let cap = appearance::MAX_MARKUP_QUADS; + let at_cap = format!("<< /Type /Annot /Subtype /Highlight /Rect [10 10 20 20] /C [1 1 0] /QuadPoints {} >>", repeated_quads(cap)); + let past_cap = format!("<< /Type /Annot /Subtype /Highlight /Rect [10 10 20 20] /C [1 1 0] /QuadPoints {} >>", repeated_quads(cap + 1)); + let doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>", + b"<< /Type /Page /Parent 2 0 R /CropBox [0 0 300 400] /Annots [4 0 R 5 0 R 6 0 R 7 0 R 8 0 R 9 0 R] >>", + b"<< /Type /Annot /Subtype /Highlight /Rect [10 10 20 20] /C [1 1 0] /QuadPoints [100 380 500 380 100 360 500 360] >>", + b"<< /Type /Annot /Subtype /Highlight /Rect [10 10 20 20] /C [1 1 0] /QuadPoints [400 500 450 500 400 480 450 480] >>", + at_cap.as_bytes(), + past_cap.as_bytes(), + b"<< /Type /Annot /Subtype /Highlight /Rect [10 10 20 20] /C [1 1 0] /QuadPoints [100 120 200 120 100 100 200 100] /AP << /N 10 0 R >> >>", + b"<< /Type /Annot /Subtype /Square /Rect [10 10 20 20] /C [0 0 1] /QuadPoints [100 120 200 120 100 100 200 100] >>", + b"<< /Type /XObject /Subtype /Form /BBox [0 0 10 10] /Length 0 >>\nstream\n\nendstream", + ]); + let display = with_missing_appearances(&doc).expect("the markups need an appearance"); + let shown = list(&display, 0); + let rect = |d: &Dict| nums_of(d, b"Rect"); + assert_eq!(rect(&shown[0]), [10.0, 10.0, 300.0, 380.0], "the box stops at the crop box"); + assert_eq!(ap_bbox(&display, &shown[0]), rect(&shown[0])); + assert_eq!(rect(&shown[1]), [10.0, 10.0, 20.0, 20.0], "quads off the page widen nothing"); + assert_eq!(rect(&shown[2]), [10.0, 10.0, 200.0, 120.0], "as many quads as the cap widen the box"); + assert_eq!(rect(&shown[3]), [10.0, 10.0, 20.0, 20.0], "more quads than the cap keep the /Rect"); + assert_eq!(ap_bbox(&display, &shown[3]), rect(&shown[3])); + assert_eq!(rect(&shown[4]), [10.0, 10.0, 20.0, 20.0], "a markup with an appearance of its own keeps its /Rect"); + assert_eq!( + shown[4].get(b"AP").map(|a| display.resolve(a)).and_then(|a| a.as_dict().and_then(|a| a.reference(b"N"))), + Some(ObjRef::new(10, 0)), + "and its appearance" + ); + assert_eq!(rect(&shown[5]), [10.0, 10.0, 20.0, 20.0], "a square is not text markup"); + assert_eq!(ap_bbox(&display, &shown[5]), rect(&shown[5])); +} + +/// A restyle (the edit path, `set_appearance`) redraws a text markup over its quadrilaterals and +/// saves the `/Rect` its appearance covers, as far as the page shows, since the appearance is +/// mapped onto `/Rect` (ISO 32000-2 §12.5.5). A markup whose `/Rect` already holds its quads keeps +/// that `/Rect` as written, swapped corners and all. +#[test] +fn restyled_text_markup_saves_the_rect_its_appearance_covers() { + let mut doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R 5 0 R 6 0 R] >>", + b"<< /Type /Annot /Subtype /Highlight /Rect [335.1 370.83 324.6 471.78] /C [1 1 0] /QuadPoints [370 287 471 287 370 277 471 277] >>", + b"<< /Type /Annot /Subtype /Highlight /Rect [150 70 50 50] /C [1 1 0] /QuadPoints [60 68 140 68 60 52 140 52] >>", + b"<< /Type /Annot /Subtype /Underline /Rect [10 10 20 20] /C [0 0 1] /QuadPoints [500 120 700 120 500 100 700 100] >>", + ]); + for index in 0..3 { + set_style(&mut doc, 0, index, StyleChange::new(Some([1.0, 0.0, 0.0]), None, None, None), &meta("")).unwrap(); + } + let saved = reopen(&doc); + let shown = list(&saved, 0); + let rect = |d: &Dict| nums_of(d, b"Rect"); + assert_eq!(rect(&shown[0]), [324.6, 277.0, 471.0, 471.78], "the saved /Rect takes in the highlighted text"); + assert_eq!(ap_bbox(&saved, &shown[0]), rect(&shown[0])); + assert!(ap_content(&saved, &shown[0]).contains("370 287 m 471 287 l 471 277 l 370 277 l h f"), "the quad is filled"); + assert_eq!(rect(&shown[1]), [150.0, 70.0, 50.0, 50.0], "a /Rect that holds the quads is kept as written"); + assert_eq!(rect(&shown[2]), [10.0, 10.0, 612.0, 120.7], "the saved /Rect stops at the page"); + assert_eq!(ap_bbox(&saved, &shown[2]), rect(&shown[2])); +} + +/// A one-page document whose page lists `annots` (objects 4 on) and the display copy +/// `with_missing_appearances` draws for it. +fn displayed(annots: &[&[u8]]) -> (Document, Document) { + let kids: Vec = (0..annots.len()).map(|i| format!("{} 0 R", i + 4)).collect(); + let page = format!("<< /Type /Page /Parent 2 0 R /Annots [{}] >>", kids.join(" ")); + let mut objs: Vec<&[u8]> = + vec![b"<< /Type /Catalog /Pages 2 0 R >>", b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>", page.as_bytes()]; + objs.extend_from_slice(annots); + let doc = document_of(&objs); + let display = with_missing_appearances(&doc).expect("the annotations need an appearance"); + (doc, display) +} + +/// Ink strokes (`/InkList`, ISO 32000-2 Table 185) outside an ink annotation's `/Rect` are drawn: +/// the display copy's `/Rect` and `/BBox` take in the strokes and half their width. Shaped like +/// the pencil marks of `issue13447.pdf`, whose `/Rect` has x and y swapped. An ink annotation +/// whose `/Rect` holds its points keeps it, even where its wide stroke reaches past it. +#[test] +fn ink_whose_strokes_reach_past_its_rect_is_drawn_through_them() { + let (doc, display) = displayed(&[ + b"<< /Type /Annot /Subtype /Ink /Rect [109.25 260.01 21.25 313.51] /BS << /W 3 >> /C [1 0 0] /InkList [[260 502.75 300 537 330 520] [280 510]] >>", + b"<< /Type /Annot /Subtype /Ink /Rect [0 0 100 100] /BS << /W 30 >> /C [1 0 0] /InkList [[10 10 90 90]] >>", + ]); + let shown = list(&display, 0); + let rect = |d: &Dict| nums_of(d, b"Rect"); + // The smooth curve rises past its highest point (537) to 537.33; half the width (1.5) more. + assert_rect_near(&rect(&shown[0]), [21.25, 260.01, 331.5, 538.825], "the ink's /Rect takes in its strokes"); + assert_eq!(ap_bbox(&display, &shown[0]), rect(&shown[0])); + assert!(ap_content(&display, &shown[0]).contains("260 502.75 m"), "the stroke is drawn"); + assert_eq!(rect(&shown[1]), [0.0, 0.0, 100.0, 100.0], "a /Rect that holds the points stays as it is"); + assert_eq!(rect(&list(&doc, 0)[0]), [109.25, 260.01, 21.25, 313.51], "the source document is unchanged"); +} + +/// A polyline's `/Vertices` (Table 181) outside its `/Rect` are drawn, with its line endings +/// (`/LE`): the box takes in the arrow at the last vertex. +#[test] +fn polyline_whose_vertices_reach_past_its_rect_is_drawn_through_them() { + let (_, display) = displayed(&[ + b"<< /Type /Annot /Subtype /PolyLine /Rect [667.5 250.69 567.75 344.44] /BS << /W 2 >> /C [0 0 1] /Vertices [250 44 350 94 285 138] >>", + b"<< /Type /Annot /Subtype /PolyLine /Rect [10 10 20 20] /BS << /W 2 >> /C [0 0 1] /LE [/None /OpenArrow] /Vertices [100 100 200 100 200 200] >>", + ]); + let shown = list(&display, 0); + let rect = |d: &Dict| nums_of(d, b"Rect"); + assert_eq!(rect(&shown[0]), [249.0, 43.0, 667.5, 344.44], "the polyline's /Rect takes in its vertices"); + assert_eq!(ap_bbox(&display, &shown[0]), rect(&shown[0])); + assert!(ap_content(&display, &shown[0]).contains("250 44 m"), "the polyline is drawn"); + // Half the width (1) and the arrow (6 + 3 × 2) past the vertices. + assert_eq!(rect(&shown[1]), [10.0, 10.0, 213.0, 213.0], "and its arrow"); +} + +/// A polygon's `/Vertices` outside its `/Rect` are drawn, its cloudy border (§12.5.4) included. +#[test] +fn polygon_whose_vertices_reach_past_its_rect_is_drawn_through_them() { + let (_, display) = displayed(&[ + b"<< /Type /Annot /Subtype /Polygon /Rect [10 10 20 20] /BS << /W 1 >> /C [0 0 1] /IC [1 1 0] /Vertices [100 100 200 100 150 180] >>", + b"<< /Type /Annot /Subtype /Polygon /Rect [10 10 20 20] /BS << /W 2 >> /C [0 0 1] /BE << /S /C /I 1 >> /Vertices [100 100 200 100 150 180] >>", + ]); + let shown = list(&display, 0); + let rect = |d: &Dict| nums_of(d, b"Rect"); + assert_eq!(rect(&shown[0]), [10.0, 10.0, 200.5, 180.5], "the polygon's /Rect takes in its vertices"); + assert_eq!(ap_bbox(&display, &shown[0]), rect(&shown[0])); + assert!(ap_content(&display, &shown[0]).contains("100 100 m"), "the polygon is drawn"); + // Half the width (1) and the cloud's reach (`cloud_reach`: 1.5 bumps of 6 + 1.5 × 2 at + // intensity 1) past the vertices. + assert_eq!(rect(&shown[1]), [10.0, 10.0, 214.5, 194.5], "and its cloudy border"); +} + +/// A line's `/L` (Table 178) outside its `/Rect` is drawn, with its line ending and its leader +/// lines. Shaped like the arrows of `issue13447.pdf`. +#[test] +fn line_whose_points_reach_past_its_rect_is_drawn_through_them() { + let (_, display) = displayed(&[ + b"<< /Type /Annot /Subtype /Line /Rect [598.31 146.63 537 316.13] /BS << /W 6 >> /C [0 0 1] /LE [/None /ClosedArrow] /L [176 45 178 154] >>", + b"<< /Type /Annot /Subtype /Line /Rect [10 10 20 20] /BS << /W 2 >> /C [0 0 1] /L [100 100 200 100] /LL 20 /LLE 5 >>", + b"<< /Type /Annot /Subtype /Line /Rect [90 90 210 110] /BS << /W 2 >> /C [0 0 1] /LE [/OpenArrow /OpenArrow] /L [100 100 200 100] >>", + ]); + let shown = list(&display, 0); + let rect = |d: &Dict| nums_of(d, b"Rect"); + // Half the width (3) and the arrow (6 + 3 × 6) past `/L`. + assert_eq!(rect(&shown[0]), [149.0, 18.0, 598.31, 316.13], "the line's /Rect takes in /L and its arrow"); + assert_eq!(ap_bbox(&display, &shown[0]), rect(&shown[0])); + assert!(ap_content(&display, &shown[0]).contains("176 45 m"), "the line is drawn"); + // The leaders rise 20 + 5 above `/L`; half the width (1) past them. + assert_eq!(rect(&shown[1]), [10.0, 10.0, 201.0, 126.0], "and its leader lines"); + assert_eq!(rect(&shown[2]), [90.0, 90.0, 210.0, 110.0], "a /Rect that holds /L stays as it is"); +} + +/// Points past the caps (`appearance::MAX_REACH_POINTS`) leave `/Rect` as it is, as do points +/// that aren't numbers; a restyle (the edit path) saves the widened `/Rect` of an ink annotation. +#[test] +fn path_reach_stops_at_the_point_cap_and_is_saved_on_restyle() { + let cap = appearance::MAX_REACH_POINTS; + let stroke = |n: usize| format!("[{}]", vec!["100 120"; n].join(" ")); + let at_cap = format!("<< /Type /Annot /Subtype /Ink /Rect [10 10 20 20] /BS << /W 0 >> /C [1 0 0] /InkList [{} [200 200]] >>", stroke(cap - 1)); + let past_cap = format!("<< /Type /Annot /Subtype /Ink /Rect [10 10 20 20] /BS << /W 0 >> /C [1 0 0] /InkList [{} [200 200]] >>", stroke(cap)); + let (_, display) = displayed(&[at_cap.as_bytes(), past_cap.as_bytes()]); + let shown = list(&display, 0); + let rect = |d: &Dict| nums_of(d, b"Rect"); + assert_eq!(rect(&shown[0]), [10.0, 10.0, 200.0, 200.0], "as many points as the cap widen the box"); + assert_eq!(rect(&shown[1]), [10.0, 10.0, 20.0, 20.0], "more points than the cap keep the /Rect"); + + let mut doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>", + b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R] >>", + b"<< /Type /Annot /Subtype /Ink /Rect [109.25 260.01 21.25 313.51] /BS << /W 3 >> /C [1 0 0] /InkList [[260 502.75 300 537 330 520]] >>", + ]); + set_style(&mut doc, 0, 0, StyleChange::new(Some([0.0, 0.0, 1.0]), None, None, None), &meta("")).unwrap(); + let saved = reopen(&doc); + let shown = list(&saved, 0); + assert_rect_near(&rect(&shown[0]), [21.25, 260.01, 331.5, 538.825], "the saved /Rect takes in the strokes"); + assert_eq!(ap_bbox(&saved, &shown[0]), rect(&shown[0])); +} + +/// A squiggle's peaks stand 1.5 above a quad's bottom edge however low the quad is, so over a quad +/// 1 high they reach past it: the widened box takes them in, with half the line's width. +#[test] +fn squiggly_peaks_over_a_low_quad_are_inside_the_widened_box() { + let (_, display) = + displayed(&[b"<< /Type /Annot /Subtype /Squiggly /Rect [10 10 20 20] /C [0 1 0] /QuadPoints [100 301 200 301 100 300 200 300] >>"]); + let shown = list(&display, 0); + // Peak 300 + 1.5, half the 0.5 line width: 301.75; the sides reach 0.75 past the quad. + assert_eq!(nums_of(&shown[0], b"Rect"), [10.0, 10.0, 200.75, 301.75]); + assert_eq!(ap_bbox(&display, &shown[0]), nums_of(&shown[0], b"Rect")); +} + +/// A line's caption (`/Cap`, ISO 32000-2 Table 178) is drawn beside `/L`: the widened box takes +/// in the caption's text as well as the line. +#[test] +fn a_line_caption_is_inside_the_widened_box() { + let (_, display) = displayed(&[ + b"<< /Type /Annot /Subtype /Line /Rect [10 10 20 20] /BS << /W 2 >> /C [0 0 1] /L [100 100 200 100] /Cap true /CP /Top /Contents (Caption) >>", + ]); + let shown = list(&display, 0); + let rect = nums_of(&shown[0], b"Rect"); + // The caption sits on top of the line: its 10-point text rises well past the stroke's 101. + assert!(rect[3] > 108.0, "{rect:?}"); + assert_eq!(ap_bbox(&display, &shown[0]), rect); +} + +/// A line just above the crop box whose wide stroke reaches down onto the page is drawn as far as +/// the page shows, not cut to its `/Rect`. +#[test] +fn a_stroke_that_reaches_onto_the_page_from_points_off_it_widens_the_box() { + let doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>", + b"<< /Type /Page /Parent 2 0 R /CropBox [0 0 300 400] /Annots [4 0 R] >>", + b"<< /Type /Annot /Subtype /Line /Rect [10 10 20 20] /BS << /W 6 >> /C [0 0 1] /L [100 401 200 401] >>", + ]); + let display = with_missing_appearances(&doc).expect("the line needs an appearance"); + let shown = list(&display, 0); + // The stroke spans 398 to 404 across and 97 to 203 along; the page shows up to 400. + assert_eq!(nums_of(&shown[0], b"Rect"), [10.0, 10.0, 203.0, 400.0]); +} + +/// A cached display appearance was drawn for its page's crop box: once the crop box grows, the +/// next refresh draws the annotation again for the larger region. +#[test] +fn a_cached_appearance_is_redrawn_for_a_changed_crop_box() { + let mut doc = document_of(&[ + b"<< /Type /Catalog /Pages 2 0 R >>", + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>", + b"<< /Type /Page /Parent 2 0 R /CropBox [0 0 300 400] /Annots [4 0 R] >>", + b"<< /Type /Annot /Subtype /Highlight /Rect [10 10 20 20] /C [1 1 0] /QuadPoints [100 380 500 380 100 360 500 360] >>", + ]); + let mut cache = AppearanceCache::default(); + let display = with_missing_appearances_cached(&doc, &mut cache).expect("drawn"); + assert_eq!(nums_of(&list(&display, 0)[0], b"Rect"), [10.0, 10.0, 300.0, 380.0]); + doc.update_dict(ObjRef::new(3, 0), |d| d.set(b"CropBox".to_vec(), Object::Array([0, 0, 612, 792].map(Object::Int).to_vec()))).unwrap(); + let display = with_missing_appearances_cached(&doc, &mut cache).expect("drawn"); + let shown = list(&display, 0); + assert_eq!(nums_of(&shown[0], b"Rect"), [10.0, 10.0, 500.0, 380.0], "drawn for the larger crop box"); + assert_eq!(ap_bbox(&display, &shown[0]), nums_of(&shown[0], b"Rect")); +} + +/// Redrawing a comment drawn through its points finds its page through `/P` when that page +/// lists it, reading that one page's `/Annots` rather than every page's: here the last of 300. +#[test] +fn restyling_a_comment_drawn_through_its_points_reads_only_its_page() { + let pages = 300; + let mut objs: Vec = vec!["<< /Type /Catalog /Pages 2 0 R >>".into()]; + let kids: Vec = (0..pages).map(|i| format!("{} 0 R", 3 + 2 * i)).collect(); + objs.push(format!("<< /Type /Pages /Kids [{}] /Count {pages} /MediaBox [0 0 612 792] >>", kids.join(" "))); + for i in 0..pages { + let (page, annot) = (3 + 2 * i, 4 + 2 * i); + objs.push(format!("<< /Type /Page /Parent 2 0 R /Annots [{annot} 0 R] >>")); + objs.push(format!( + "<< /Type /Annot /Subtype /Highlight /P {page} 0 R /Rect [10 10 20 20] /C [1 1 0] /QuadPoints [100 120 200 120 100 100 200 100] >>" + )); + } + let bytes: Vec<&[u8]> = objs.iter().map(|o| o.as_bytes()).collect(); + let mut doc = document_of(&bytes); + let last = pages - 1; + ANNOTS_ENTRIES_COPIED.with(|c| c.set(0)); + set_style(&mut doc, last, 0, StyleChange::new(Some([1.0, 0.0, 0.0]), None, None, None), &meta("")).unwrap(); + let copied = ANNOTS_ENTRIES_COPIED.with(std::cell::Cell::get); + assert!(copied < 20, "{copied} annotation entries copied for one restyle"); + assert_eq!(nums_of(&list(&doc, last)[0], b"Rect"), [10.0, 10.0, 200.0, 120.0], "drawn for its page"); +} + +/// `got` is `want` to within 0.001 in every coordinate. +fn assert_rect_near(got: &[f64], want: [f64; 4], what: &str) { + assert!(got.len() == 4 && got.iter().zip(want).all(|(g, w)| (g - w).abs() < 1e-3), "{what}: {got:?}"); +} + +fn nums_of(d: &Dict, key: &[u8]) -> Vec { + d.get(key).and_then(|a| a.as_array()).map(|a| a.iter().filter_map(|o| o.as_f64()).collect()).unwrap_or_default() +} + +/// The `y` of every `1 0 0 1 x y Tm` in `content`: where each piece of text sits. +fn baselines(content: &str) -> Vec { + content.split(" Tm").filter_map(|s| s.rsplit(' ').next()?.parse().ok()).collect() +} + +/// Thai typed into Fill & Sign text or a text box is shaped and drawn from the Thai face, as Add +/// text draws it: Type 3 glyphs (their streams objects of their own) with a ToUnicode map, never +/// `?`. The box grows to hold the ink of its stacked marks and descenders, which reach past the +/// 1.2 em lines boxes are sized by. A build without a Thai face refuses the text, saying why. +#[test] +fn thai_in_fill_sign_text_and_text_boxes_is_drawn_from_the_thai_face() { + let mut doc = fixture(); + // Each line has the face's highest stack (น้ำ) and deepest descender (ฎู). + let thai = "น้ำฎู บุญ\nปั้นน้ำฎู"; + let item = |shape: Shape| NewAnnotation { page: 0, style: Style::default_for(&shape), shape, contents: thai.into(), author: "Ada".into() }; + let shapes = [ + Shape::Typewriter { rect: [100.0, 700.0, 300.0, 716.0], font_size: 24.0 }, + Shape::TextBox { rect: [100.0, 400.0, 400.0, 430.0], font_size: 30.0 }, + ]; + if pdfcraft_fonts::document_thai_font().is_none() { + for shape in shapes { + let err = add_annotation(&mut doc, &item(shape), &meta("t")).unwrap_err(); + assert!(matches!(&err, AnnotError::Undrawable(m) if m.contains("Thai") && m.contains("CRAFT_FONTS_DIR")), "{err:?}"); + } + assert_eq!(list(&doc, 0).len(), 1, "nothing was added"); + let t = add_annotation(&mut doc, &new(0, Shape::TextBox { rect: [100.0, 600.0, 300.0, 650.0], font_size: 12.0 }), &meta("b")).unwrap(); + assert!(matches!(set_contents(&mut doc, 0, t, thai, &meta("")), Err(AnnotError::Undrawable(_)))); + assert_eq!(text(&list(&doc, 0)[t], b"Contents"), "hello"); + return; + } + let reach = pdfcraft_fonts::shaped_text::thai_reach; + let (lo, hi) = (reach("ฎู").unwrap()[0], reach("น้ำ").unwrap()[1]); + for (shape, size) in shapes.into_iter().zip([24.0, 30.0]) { + let t = add_annotation(&mut doc, &item(shape), &meta("t")).unwrap(); + let doc = reopen(&doc); + let d = &list(&doc, 0)[t]; + assert_eq!((text(d, b"Contents").as_str(), pdfcraft_fonts::undrawn_mark(d)), (thai, None)); + let content = ap_content(&doc, d); + assert!(!content.contains('?') && content.contains("/PCTh1 "), "{content}"); + // The glyphs and the text they stand for are objects of the saved file. + let n = d.get(b"AP").and_then(|a| a.as_dict()).and_then(|a| a.reference(b"N")).unwrap(); + let ap = doc.get(n); + let fonts = ap.as_dict().and_then(|s| s.get(b"Resources")).and_then(|r| r.as_dict()).and_then(|r| r.get(b"Font")).and_then(|f| f.as_dict()); + let font = fonts.and_then(|f| f.get(b"PCTh1")).and_then(|f| f.as_dict()).unwrap().clone(); + assert_eq!(font.name(b"Subtype"), Some(&b"Type3"[..])); + let cmap = doc.get(font.reference(b"ToUnicode").unwrap()); + let Object::Stream(cmap) = &*cmap else { panic!("ToUnicode is a stream object") }; + let cmap = String::from_utf8_lossy(&cmap.decoded().unwrap()).into_owned(); + assert!(cmap.contains("<0E190E490E33>"), "น้ำ copies as typed: {cmap}"); + let procs = font.get(b"CharProcs").and_then(|p| p.as_dict()).unwrap(); + assert!(procs.iter().all(|(_, g)| matches!(g, Object::Ref(_))), "glyph programs are stream objects"); + // Every line's ink lies inside the box: the appearance's BBox and its clip. + let (r, ys) = (rect(d), baselines(&content)); + let (top, bottom) = (ys.iter().copied().fold(f64::MIN, f64::max), ys.iter().copied().fold(f64::MAX, f64::min)); + assert!(top + hi * size <= r[3] + 1e-3 && bottom + lo * size >= r[1] - 1e-3, "{ys:?} at {size} in {r:?}"); + } + // New text refits the box to its ink too: the stack on its first line, the descender on its last. + set_contents(&mut doc, 0, 1, "น้ำ\nฎู", &meta("")).unwrap(); + let d = &list(&doc, 0)[1]; + let (r, ys) = (rect(d), baselines(&ap_content(&doc, d))); + assert!(ys[0] + hi * 24.0 <= r[3] + 1e-3 && ys[ys.len() - 1] + lo * 24.0 >= r[1] - 1e-3, "{ys:?} in {r:?}"); +} + const QUAD: [f64; 8] = [20.0, 110.0, 40.0, 110.0, 20.0, 95.0, 40.0, 95.0]; fn reals(v: &[f64]) -> Vec { diff --git a/crates/annot/src/text_style.rs b/crates/annot/src/text_style.rs new file mode 100644 index 0000000000..dfa4c8c1fa --- /dev/null +++ b/crates/annot/src/text_style.rs @@ -0,0 +1,480 @@ +//! The face, size and colour a text box's text is drawn in when PdfCraft draws its appearance: +//! `/DA` (ISO 32000-2 Table 177; the font named there is looked up in the interactive form's +//! `/DR` font resources, §12.7.4.3) and `/DS`, the default style string (Table 177), which is +//! read for its CSS `font`, `font-family`, `font-size`, `font-weight`, `font-style` and `color`. +//! +//! Only the twelve standard text faces are drawn (Helvetica, Times and Courier, each regular, +//! bold, italic and bold italic, in WinAnsiEncoding): a font that isn't one of them is drawn in +//! Helvetica, as before. Symbol and ZapfDingbats are not text faces for this purpose. + +use pdfcraft_cos::{Dict, Document, Object}; +use pdfcraft_fonts::{Std14, Std14Match as Match}; + +use crate::Rgb; +use crate::appearance::parse_da; + +/// `/DS` past this many bytes is read only this far (a style string is a line or two). +const MAX_DS: usize = 4096; + +/// Font sizes a text box is drawn at, as for `/DA`. +const MAX_SIZE: f64 = 400.0; + +/// How a text box's text is drawn. +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct TextStyle { + pub face: Std14, + pub size: f64, + pub color: Rgb, +} + +/// The style of `d`'s text: `/DA`, with what `/DS` sets taking its place. `dr_face` is the face +/// the `/DA` font resource names in the form's `/DR` ([`dr_face`]); `None` when it isn't there, +/// and the common resource names (`/Helv`, `/Cour`, `/TiRo`, …) are read instead. +pub fn text_style(d: &Dict, dr_face: Option) -> TextStyle { + let (da_color, da_size) = parse_da(d); + let da_face = dr_face.or_else(|| da_font_name(d).as_deref().and_then(resource_face)).unwrap_or(Std14::Helvetica); + let mut style = TextStyle { face: da_face, size: da_size, color: da_color }; + let Some(ds) = d.get(b"DS").and_then(Object::as_string).map(|s| s.to_text()) else { return style }; + let ds = parse_ds(&ds); + if let Some(size) = ds.size { + style.size = size; + } + // `/DS` colours are 8-bit (`#RRGGBB`): one that rounds to the same as `/DA`'s keeps `/DA`'s. + if let Some(color) = ds.color + && bytes(color) != bytes(da_color) + { + style.color = color; + } + let base = ds.face.unwrap_or(style.face); + style.face = styled(base, ds.bold.unwrap_or(is_bold(base)), ds.italic.unwrap_or(is_italic(base))); + style +} + +/// The face the `/DA` font of `d` names in the document's `/AcroForm /DR /Font` (§12.7.4.3): +/// `Some` when that resource is there — the standard text face its `/BaseFont` names, or +/// Helvetica for any other font — and `None` when it isn't. +pub fn dr_face(doc: &Document, d: &Dict) -> Option { + let name = da_font_name(d)?; + let root = doc.trailer().get(b"Root").map(|r| doc.resolve(r))?; + let form = root.as_dict()?.get(b"AcroForm").map(|f| doc.resolve(f))?; + let dr = form.as_dict()?.get(b"DR").map(|r| doc.resolve(r))?; + let fonts = dr.as_dict()?.get(b"Font").map(|f| doc.resolve(f))?; + let font = fonts.as_dict()?.get(name.as_bytes()).map(|f| doc.resolve(f))?; + let font = font.as_dict()?; + let base = font.name(b"BaseFont").map(String::from_utf8_lossy); + Some(base.as_deref().and_then(base_font_face).unwrap_or(Std14::Helvetica)) +} + +/// The font resource name of the last `Tf` in `/DA` (without the slash). `Tf` takes a name and +/// a size; a name given without its size (`/TiRo Tf`) is still read, and the size then falls back +/// as [`parse_da`] says. +pub fn da_font_name(d: &Dict) -> Option { + let da = d.get(b"DA").and_then(Object::as_string)?; + let da = String::from_utf8_lossy(&da.bytes); + let toks: Vec<&str> = da.split_whitespace().collect(); + let tf = toks.iter().rposition(|t| *t == "Tf")?; + let name = match toks.get(..tf)? { + [.., name, size] if !size.starts_with('/') => name, + [.., name] => name, + [] => return None, + }; + let name = name.strip_prefix('/')?; + (!name.is_empty()).then(|| name.to_string()) +} + +/// The face a font resource name stands for: the names PDF writers commonly give the standard +/// fonts in `/DR` (`Helv`, `HeBo`, `Cour`, `TiRo`, …), or a standard face's own name. +pub fn resource_face(name: &str) -> Option { + Some(match name { + "Helv" => Std14::Helvetica, + "HeBo" => Std14::HelveticaBold, + "HeOb" | "HeIt" => Std14::HelveticaOblique, + "HeBO" | "HeBI" => Std14::HelveticaBoldOblique, + "Cour" => Std14::Courier, + "CoBo" => Std14::CourierBold, + "CoOb" | "CoIt" => Std14::CourierOblique, + "CoBO" | "CoBI" => Std14::CourierBoldOblique, + "TiRo" => Std14::TimesRoman, + "TiBo" => Std14::TimesBold, + "TiIt" => Std14::TimesItalic, + "TiBI" => Std14::TimesBoldItalic, + _ => return base_font_face(name), + }) +} + +/// The standard text face a `/BaseFont` (or family) name asks for: the spec's names and their +/// usual aliases (Arial, Times New Roman, Courier New), not a guess from a family hint. +fn base_font_face(name: &str) -> Option { + match Std14::recognise(name)? { + (Std14::Symbol | Std14::ZapfDingbats, _) | (_, Match::FamilyHint) => None, + (face, Match::Alias) => Some(face), + } +} + +/// `CSS` family names: the generic ones, else a standard face's name or alias. +fn family_face(name: &str) -> Option { + let name = name.trim().trim_matches(|c| c == '"' || c == '\'').trim(); + match name.to_ascii_lowercase().as_str() { + "serif" => Some(Std14::TimesRoman), + "monospace" => Some(Std14::Courier), + "sans-serif" => Some(Std14::Helvetica), + _ => base_font_face(name), + } +} + +fn is_bold(f: Std14) -> bool { + matches!( + f, + Std14::HelveticaBold + | Std14::HelveticaBoldOblique + | Std14::TimesBold + | Std14::TimesBoldItalic + | Std14::CourierBold + | Std14::CourierBoldOblique + ) +} + +fn is_italic(f: Std14) -> bool { + matches!( + f, + Std14::HelveticaOblique + | Std14::HelveticaBoldOblique + | Std14::TimesItalic + | Std14::TimesBoldItalic + | Std14::CourierOblique + | Std14::CourierBoldOblique + ) +} + +/// `base`'s family in the given weight and slant. +fn styled(base: Std14, bold: bool, italic: bool) -> Std14 { + use Std14::*; + match (base, bold, italic) { + (TimesRoman | TimesBold | TimesItalic | TimesBoldItalic, false, false) => TimesRoman, + (TimesRoman | TimesBold | TimesItalic | TimesBoldItalic, true, false) => TimesBold, + (TimesRoman | TimesBold | TimesItalic | TimesBoldItalic, false, true) => TimesItalic, + (TimesRoman | TimesBold | TimesItalic | TimesBoldItalic, true, true) => TimesBoldItalic, + (Courier | CourierBold | CourierOblique | CourierBoldOblique, false, false) => Courier, + (Courier | CourierBold | CourierOblique | CourierBoldOblique, true, false) => CourierBold, + (Courier | CourierBold | CourierOblique | CourierBoldOblique, false, true) => CourierOblique, + (Courier | CourierBold | CourierOblique | CourierBoldOblique, true, true) => CourierBoldOblique, + (_, false, false) => Helvetica, + (_, true, false) => HelveticaBold, + (_, false, true) => HelveticaOblique, + (_, true, true) => HelveticaBoldOblique, + } +} + +/// The resource name an appearance stream gives `face` (the usual `/DR` names). +pub fn resource_name(face: Std14) -> &'static str { + match face { + Std14::HelveticaBold => "HeBo", + Std14::HelveticaOblique => "HeOb", + Std14::HelveticaBoldOblique => "HeBO", + Std14::TimesRoman => "TiRo", + Std14::TimesBold => "TiBo", + Std14::TimesItalic => "TiIt", + Std14::TimesBoldItalic => "TiBI", + Std14::Courier => "Cour", + Std14::CourierBold => "CoBo", + Std14::CourierOblique => "CoOb", + Std14::CourierBoldOblique => "CoBO", + _ => "Helv", + } +} + +/// `face`'s `CSS` family, weight and style, for writing `/DS` and `/RC`. +pub fn css(face: Std14) -> (&'static str, &'static str, &'static str) { + let family = match styled(face, false, false) { + Std14::TimesRoman => "Times", + Std14::Courier => "Courier", + _ => "Helvetica", + }; + (family, if is_bold(face) { "bold" } else { "normal" }, if is_italic(face) { "italic" } else { "normal" }) +} + +fn bytes(c: Rgb) -> [u8; 3] { + // Clamped to 0..=1 first, so the product is 0..=255 and the cast exact. + c.map(|x| (x.clamp(0.0, 1.0) * 255.0).round() as u8) +} + +/// What a `/DS` string sets. +#[derive(Debug, Default, PartialEq)] +struct Ds { + face: Option, + bold: Option, + italic: Option, + size: Option, + color: Option, +} + +/// Read a default style string: `;`-separated `CSS` declarations. Unknown or malformed ones are +/// skipped; later ones win. +fn parse_ds(ds: &str) -> Ds { + let end = ds.char_indices().map(|(i, _)| i).find(|i| *i >= MAX_DS).unwrap_or(ds.len()); + let ds = ds.get(..end).unwrap_or_default(); + let mut out = Ds::default(); + for decl in ds.split(';') { + let Some((prop, value)) = decl.split_once(':') else { continue }; + let value = value.trim(); + match prop.trim().to_ascii_lowercase().as_str() { + "font" => font_shorthand(value, &mut out), + "font-family" => { + if let Some(face) = value.split(',').find_map(family_face) { + out.face = Some(face); + } + } + "font-size" => { + if let Some(size) = font_size(value, true) { + out.size = Some(size); + } + } + "font-weight" => { + if let Some(bold) = weight(value) { + out.bold = Some(bold); + } + } + "font-style" => { + if let Some(italic) = slant(value) { + out.italic = Some(italic); + } + } + "color" => { + if let Some(c) = css_color(value) { + out.color = Some(c); + } + } + _ => {} + } + } + out +} + +/// `font: [style] [weight] size[/line-height] family[, family…]`, read leniently: the size is +/// the token with a unit, keywords set weight and slant, and the rest is the family list (some +/// writers put the family first). +fn font_shorthand(value: &str, out: &mut Ds) { + let mut family = Vec::new(); + for tok in value.split_whitespace() { + if let Some(size) = font_size(tok, false) { + out.size = Some(size); + } else if let Some(bold) = weight(tok).filter(|_| !tok.eq_ignore_ascii_case("normal")) { + out.bold = Some(bold); + } else if let Some(italic) = slant(tok).filter(|_| !tok.eq_ignore_ascii_case("normal")) { + out.italic = Some(italic); + } else if !matches!(tok.to_ascii_lowercase().as_str(), "normal" | "small-caps") { + family.push(tok); + } + } + if let Some(face) = family.join(" ").split(',').find_map(family_face) { + out.face = Some(face); + } +} + +/// A size in points: `14pt` (with a line height, `14pt/16pt`), or a bare number where +/// `unitless` allows it. Positive and finite only; capped as `/DA` sizes are. +fn font_size(tok: &str, unitless: bool) -> Option { + let tok = tok.split('/').next()?.trim(); + let num = match tok.strip_suffix("pt").or_else(|| tok.strip_suffix("PT")) { + Some(n) => n, + None if unitless => tok, + None => return None, + }; + num.parse::().ok().filter(|s| s.is_finite() && *s > 0.0).map(|s| s.min(MAX_SIZE)) +} + +fn weight(v: &str) -> Option { + match v.to_ascii_lowercase().as_str() { + "bold" | "bolder" | "600" | "700" | "800" | "900" => Some(true), + "normal" | "lighter" | "100" | "200" | "300" | "400" | "500" => Some(false), + _ => None, + } +} + +fn slant(v: &str) -> Option { + match v.to_ascii_lowercase().as_str() { + "italic" | "oblique" => Some(true), + "normal" => Some(false), + _ => None, + } +} + +/// `#RRGGBB`, `#RGB` or `rgb(r, g, b)` (0–255 or percentages). +fn css_color(v: &str) -> Option { + let v = v.trim(); + if let Some(hex) = v.strip_prefix('#') { + let digit = |c: char| c.to_digit(16).map(f64::from); + let ds: Vec = hex.chars().map(digit).collect::>()?; + return match ds.as_slice() { + [r, g, b] => Some([r * 17.0 / 255.0, g * 17.0 / 255.0, b * 17.0 / 255.0]), + [r1, r0, g1, g0, b1, b0] => Some([(r1 * 16.0 + r0) / 255.0, (g1 * 16.0 + g0) / 255.0, (b1 * 16.0 + b0) / 255.0]), + _ => None, + }; + } + let lower = v.to_ascii_lowercase(); + let inner = lower.strip_prefix("rgb(")?.strip_suffix(')')?; + let parts: Vec = inner + .split(',') + .map(|p| { + let p = p.trim(); + let x = match p.strip_suffix('%') { + Some(pc) => pc.trim().parse::().ok()? / 100.0, + None => p.parse::().ok()? / 255.0, + }; + x.is_finite().then(|| x.clamp(0.0, 1.0)) + }) + .collect::>()?; + match parts.as_slice() { + [r, g, b] => Some([*r, *g, *b]), + _ => None, + } +} + +#[cfg(test)] +mod tests { + use pdfcraft_cos::PdfString; + + use super::*; + + fn dict(da: &str, ds: Option<&str>) -> Dict { + let mut d = Dict::new(); + d.set(b"DA".to_vec(), PdfString::literal(da.as_bytes().to_vec())); + if let Some(ds) = ds { + d.set(b"DS".to_vec(), PdfString::text(ds)); + } + d + } + + #[test] + fn da_resource_names_pick_the_standard_faces() { + assert_eq!(text_style(&dict("0 g /Cour 12 Tf", None), None).face, Std14::Courier); + assert_eq!(text_style(&dict("1 0 0 rg /TiRo 16 Tf", None), None).face, Std14::TimesRoman); + assert_eq!(text_style(&dict("/HeBo 9 Tf 0 g", None), None).face, Std14::HelveticaBold); + assert_eq!(text_style(&dict("/Courier-Bold 9 Tf", None), None).face, Std14::CourierBold); + // Unknown, symbolic and family-hint-only names keep Helvetica. + for da in ["/F1 12 Tf", "/ZaDb 12 Tf", "/Symbol 12 Tf", "/MySerif 12 Tf", "12 Tf", "", "/ Tf"] { + assert_eq!(text_style(&dict(da, None), None).face, Std14::Helvetica, "{da:?}"); + } + // A `/DR` lookup outranks the name. + assert_eq!(text_style(&dict("/Cour 12 Tf", None), Some(Std14::TimesRoman)).face, Std14::TimesRoman); + } + + #[test] + fn ds_font_size_and_colour_win_over_da() { + let s = text_style(&dict("0 g /Helv 12 Tf", Some("font: Times 14pt; color: #008000")), None); + assert_eq!((s.face, s.size), (Std14::TimesRoman, 14.0)); + assert!((s.color[1] - 128.0 / 255.0).abs() < 1e-9 && s.color[0] == 0.0 && s.color[2] == 0.0, "{:?}", s.color); + let s = text_style( + &dict( + "1 0 0 rg /Cour 9 Tf", + Some("font-family: 'Helvetica', sans-serif; font-size: 20; font-weight: bold; font-style: italic; color: rgb(0, 0, 255)"), + ), + None, + ); + assert_eq!((s.face, s.size, s.color), (Std14::HelveticaBoldOblique, 20.0, [0.0, 0.0, 1.0])); + // CSS order, a line height, a #RGB colour and a generic family. + let s = text_style(&dict("0 g /Helv 12 Tf", Some("font: italic bold 10pt/12pt serif; color:#f00")), None); + assert_eq!((s.face, s.size, s.color), (Std14::TimesBoldItalic, 10.0, [1.0, 0.0, 0.0])); + // Bold from the style keeps the DA family. + assert_eq!(text_style(&dict("/Cour 12 Tf", Some("font-weight:700")), None).face, Std14::CourierBold); + } + + /// PdfCraft writes `/DS` from `/DA` (8-bit colour): `/DA`'s exact colour, size and face stay. + #[test] + fn pdfcraft_written_ds_matches_da() { + let s = text_style(&dict("0.1 0.2 0.3 rg /Helv 12.5 Tf", Some("font: Helvetica 12.5pt; text-align:left; color:#1A334D")), None); + assert_eq!(s, TextStyle { face: Std14::Helvetica, size: 12.5, color: [0.1, 0.2, 0.3] }); + } + + #[test] + fn malformed_ds_falls_back_to_da() { + let da = TextStyle { face: Std14::Courier, size: 9.0, color: [1.0, 0.0, 0.0] }; + let huge = format!("color: #00ff00;{}", "x".repeat(1 << 20)); + let past_cap = format!("{}; font-size: 30pt", " ".repeat(MAX_DS)); + for ds in [ + "font-size: NaN", + "font-size: inf", + "font-size: -3pt", + "font-size: 0", + "font: NaNpt Zzz", + "font: 1e999pt", + "color: #12", + "color: #zzzzzz", + "color: rgb(1, 2)", + "color: rgb(NaN, 0, 0)", + "color: #ff0000é", + ":::;;;", + "font-family: Wingdings, 'Comic Sans'", + "é: ü; ;", + past_cap.as_str(), + ] { + assert_eq!(text_style(&dict("1 0 0 rg /Cour 9 Tf", Some(ds)), None), da, "{ds:?}"); + } + // Read up to the cap: what comes first still counts, and nothing past it is scanned. + assert_eq!(text_style(&dict("1 0 0 rg /Cour 9 Tf", Some(&huge)), None).color, [0.0, 1.0, 0.0]); + // Sizes are capped as /DA's are. + assert_eq!(text_style(&dict("/Helv 9 Tf", Some("font-size: 1e300pt")), None).size, MAX_SIZE); + } + + /// `/DA` strings as files give them: a `Tf` without a size, a zero size (auto-size in a form + /// field, which a text box doesn't have), and hostile operands. None panics, each keeps what it + /// can read, and nothing that isn't a finite number reaches the style. + #[test] + fn hostile_da_strings_keep_what_they_can() { + let style = |da: &str| text_style(&dict(da, None), None); + let black12 = |face| TextStyle { face, size: 12.0, color: [0.0; 3] }; + for (da, want) in [ + // A missing size keeps the named face at the default size. + ("/TiRo Tf", black12(Std14::TimesRoman)), + ("0 g /Cour Tf", black12(Std14::Courier)), + // Zero, negative, NaN, infinite or not a number: the default size. + ("/Cour 0 Tf", black12(Std14::Courier)), + ("/Cour -0 Tf", black12(Std14::Courier)), + ("/Cour -12 Tf", black12(Std14::Courier)), + ("/Cour NaN Tf", black12(Std14::Courier)), + ("/Cour inf Tf", black12(Std14::Courier)), + ("/Cour -inf Tf", black12(Std14::Courier)), + ("/Cour 1e999 Tf", black12(Std14::Courier)), + ("/Cour twelve Tf", black12(Std14::Courier)), + // No Tf at all, or one with nothing before it. + ("1 0 0 rg", TextStyle { color: [1.0, 0.0, 0.0], ..black12(Std14::Helvetica) }), + ("Tf", black12(Std14::Helvetica)), + ("Tf Tf Tf", black12(Std14::Helvetica)), + ("", black12(Std14::Helvetica)), + // Colour operands that aren't finite numbers leave the colour black. + ("NaN NaN NaN rg /Helv 9 Tf", TextStyle { size: 9.0, ..black12(Std14::Helvetica) }), + ("NaN g /Helv 9 Tf", TextStyle { size: 9.0, ..black12(Std14::Helvetica) }), + ("inf 0 0 rg /Helv 9 Tf", TextStyle { size: 9.0, ..black12(Std14::Helvetica) }), + ("rg g /Helv 9 Tf", TextStyle { size: 9.0, ..black12(Std14::Helvetica) }), + // Out-of-range colour components are clamped. + ("2 -1 0.5 rg /Helv 9 Tf", TextStyle { size: 9.0, color: [1.0, 0.0, 0.5], ..black12(Std14::Helvetica) }), + // Huge sizes are capped. + ("/Helv 1e300 Tf", TextStyle { size: MAX_SIZE, ..black12(Std14::Helvetica) }), + ("/Helv 100000 Tf", TextStyle { size: MAX_SIZE, ..black12(Std14::Helvetica) }), + // Garbage around a valid Tf: the last Tf is read. + ("\u{0} ] [ << /Helv 9 Tf >> /Cour 14 Tf )", TextStyle { size: 14.0, ..black12(Std14::Courier) }), + ] { + assert_eq!(style(da), want, "{da:?}"); + let s = style(da); + assert!(s.size.is_finite() && s.size > 0.0 && s.color.iter().all(|c| (0.0..=1.0).contains(c)), "{da:?}: {s:?}"); + } + // A font name that is no standard face and that no `/DR` lists keeps Helvetica. + assert_eq!(style("0 g /F9 11 Tf"), TextStyle { size: 11.0, ..black12(Std14::Helvetica) }); + // Binary bytes that aren't UTF-8 don't panic. + let mut d = Dict::new(); + d.set(b"DA".to_vec(), PdfString::literal(vec![0xFF, 0xFE, b' ', b'/', 0xC3, b' ', b'9', b' ', b'T', b'f'])); + assert_eq!(text_style(&d, None).size, 9.0); + assert_eq!(da_font_name(&dict("/TiRo Tf", None)).as_deref(), Some("TiRo")); + assert_eq!(da_font_name(&dict("12 Tf", None)), None); + assert_eq!(da_font_name(&dict("0 g Tf", None)), None); + } + + #[test] + fn css_round_trips_through_the_parser() { + for face in [Std14::Helvetica, Std14::TimesBold, Std14::CourierOblique, Std14::HelveticaBoldOblique, Std14::TimesBoldItalic] { + let (family, weight, style) = css(face); + let ds = format!("font-family:{family};font-weight:{weight};font-style:{style}"); + assert_eq!(text_style(&dict("/Helv 9 Tf", Some(&ds)), None).face, face); + } + } +} diff --git a/crates/annot/src/types.rs b/crates/annot/src/types.rs index f6bcc68a7e..d9dcba7678 100644 --- a/crates/annot/src/types.rs +++ b/crates/annot/src/types.rs @@ -417,13 +417,17 @@ pub enum Shape { contours: Vec>, }, /// A custom stamp: a picture already in the document (an image XObject, or a form XObject - /// whose `/Matrix` maps it to `size` points) filling `rect`, named `name`. + /// whose `/Matrix` maps it to `size` points) filling `rect`, named `name`. With `line`, a + /// line of text under the picture (#1741): a dynamic stamp's name, date and time, filled in + /// when it was placed; the picture then fills the rectangle above it + /// ([`CUSTOM_LINE_BAND`](crate::CUSTOM_LINE_BAND)). CustomStamp { rect: [f64; 4], name: String, picture: ObjRef, image: bool, size: (f64, f64), + line: Option, }, /// A redaction mark (§12.5.6.23) over quadrilaterals (text) or one rectangle as a quad /// (areas, pages). `overlay` is the text shown on the box once applied, drawn with `look`. diff --git a/crates/audit/README.md b/crates/audit/README.md index cfa2979c21..8b487db8d3 100644 --- a/crates/audit/README.md +++ b/crates/audit/README.md @@ -3,7 +3,7 @@ Layer L4 (may use `edit`): verification tooling, the faux-redaction audit. ```rust -let findings: Vec = audit_redactions(&doc)?; // page, kind, rect, covered_text +let findings: Vec = audit_redactions(&doc)?; // page, kind, rect, covered_text ``` It reports two kinds of finding: diff --git a/crates/automation/README.md b/crates/automation/README.md index 937dfecb7d..e25ff9b8b2 100644 --- a/crates/automation/README.md +++ b/crates/automation/README.md @@ -61,7 +61,8 @@ do not keep an old PDF byte buffer alive. This bounds retained text-cache data, not total process memory: active extraction batches, renderers, document buffers and the requested output have separate lifetimes. Search still reads -growing batches (8 to 128 pages), with one lookahead match to determine continuation. +growing batches (8 to 128 pages), with one lookahead match to determine continuation, unless +`stop_at_limit` is set (below), which starts at one page and looks no further than its limit. `text_find` keeps matches from readable pages when another reached page fails extraction. It returns `warnings` with 1-based `page`, `error` and `error_truncated`, plus `warning_count` and @@ -75,6 +76,16 @@ that reaches the end with warnings therefore has `complete: false` **without** ` should follow `next` when present and separately inspect warnings. A capped search can return both. The existing `query`, `count`, `matches` and rectangle semantics are unchanged. +That lookahead means a search with exactly `limit` matches reads on to the end of the document to +learn that no further match exists. `stop_at_limit: true` skips it: the search ends at the +`limit`th match. It reads one page first, then batches of 2, 4, … up to 128 pages, so nothing past +the batch holding that match is read or can add warnings (a match on the first page reads only that +page). The answer carries +`stopped_at_limit`. When that is `true`, `complete` is `false` and `next` is only the position just +past the last match, not a promise of another one; following it may return no matches and +`complete: true`. When it is `false`, the search ran out of pages first and `complete` means what +it always does. + `text_extract`, comment placement and redaction keep strict all-requested-pages-must-succeed semantics; they never silently act on partial text. @@ -146,7 +157,10 @@ claude mcp add pdfcraft -- pdfcraft-cli mcp --root `object_list {doc, page}` inventories recognised paragraphs, Image/Form artwork and added items with generation-scoped kind/index references. `object_move {doc, page, objects, offset, generation?}` moves 1–1000 references atomically in displayed points (right/down), preserving source glyph bytes -and fonts. A refusal changes nothing; success is one undo step. See +and fonts. A refusal changes nothing; success is one undo step. `all: true` takes every object on the +page instead, as ⌘A does. `object_copy {doc, page, objects | all, cut?}` copies them with every +resource they use and `object_paste {doc, page}` pastes them as page content that draws the same, +into the same or another open document, stepped as the app steps a paste. See [the selection and movement guide](../../docs/editing-multiple-objects.md) for limits and examples. ## Accessibility review record @@ -285,7 +299,14 @@ Example: `form_set_props doc=1 field=agree export_value=Accepted widget=1`. reference) to a PNG, without the viewer. `model` picks the page's Nth 3D model (default 1); `width` and `height` are 1 to 4096 (default 512); `direction` is the view direction from the eye (default `[-1, 1, -1]`, with Z up unless the view looks along Z); `perspective` defaults to true. -The model is framed to fill the image and shaded by a headlight on white. U3D models are refused +The model is framed to fill the image and shaded by a headlight on white. `orbit`, instead of +`direction`, frames it as a viewer's orbit control would: `{view, up, front, turn, zoom, +zoom_at, pan}`, all optional. `view` is `iso` (default), `front`, `right`, `top` or `back` +about the `up` axis (default `+z`) and the `front` axis (perpendicular to `up`); `turn` is +`[yaw, pitch]` in degrees, pitch down; `zoom` is a positive factor, the result held within 0.05 to 50, at `zoom_at` `[right, up]` +in picture heights from the centre, where the point under it stays still; `pan` `[right, up]` +moves the target, in model radii, so the model moves the other way. Every view frames the model +at the same size, whatever its direction. U3D models are refused with an error, and the annotation's own views, lighting and render modes are not applied yet. ## Viewer navigation history diff --git a/crates/automation/src/a11y.rs b/crates/automation/src/a11y.rs index 1f0c9cb8a5..b17192aab0 100644 --- a/crates/automation/src/a11y.rs +++ b/crates/automation/src/a11y.rs @@ -6,10 +6,10 @@ use serde_json::{Value, json}; use sha2::{Digest, Sha256}; use std::collections::BTreeSet; -use crate::{Args, Automation, Result, ToolError, child, failed, unused, write_atomic}; +use crate::{Args, Automation, Result, ToolError, child, failed, write_atomic}; mod documents; -mod form_tools; +pub(crate) mod form_tools; mod ocr; fn status_id(s: Status) -> &'static str { @@ -213,25 +213,25 @@ const MAX_TAG_PAGE: usize = 1000; impl Automation { pub(crate) fn accessibility_autotag_apply(&mut self, a: &Args) -> Result { let id = self.doc(a)?.id; - self.session.apply(id, pdfcraft_engine::Edit::AutoTagText).map_err(failed)?; + let warnings = self.apply_with_warnings(id, pdfcraft_engine::Edit::AutoTagText)?; let tags = self.accessibility_tags(a)?; - Ok(json!({ + let out = json!({ "tags": tags["tags"], "count": tags["count"], "note": "Text objects were tagged as paragraphs. Review heading roles, reading order and remaining accessibility findings manually.", - })) + }); + Ok(with_warnings(out, warnings)) } pub(crate) fn accessibility_autotag_form_fields(&mut self, a: &Args) -> Result { let id = self.doc(a)?.id; - self.session.apply(id, pdfcraft_engine::Edit::AutoTagFormFields).map_err(failed)?; + let warnings = self.apply_with_warnings(id, pdfcraft_engine::Edit::AutoTagFormFields)?; let report = self .doc(a)? .accessibility_check(&Options { rules: [Rule::TaggedFormFields].into(), pages: None }) .ok_or_else(|| failed("the document can't be read"))?; let result = report.result(Rule::TaggedFormFields).ok_or_else(|| failed("the form field check is unavailable"))?; - Ok( - json!({ "status": status_id(result.status), "remaining_findings": result.findings.len(), "note": "Only form widgets were tagged. Review text, figures, links, reading order, and other document content separately." }), - ) + let out = json!({ "status": status_id(result.status), "remaining_findings": result.findings.len(), "note": "Only form widgets were tagged. Review text, figures, links, reading order, and other document content separately." }); + Ok(with_warnings(out, warnings)) } pub(crate) fn accessibility_autotag_preview(&self, a: &Args) -> Result { @@ -432,10 +432,11 @@ impl Automation { let rule = Rule::from_id(id).ok_or_else(|| ToolError::InvalidArgs(format!("unknown rule {id:?}")))?; let edit = self.doc(a)?.accessibility_fix(rule, a.opt_str("value")?).map_err(failed)?; let doc = self.doc(a)?.id; - self.session.apply(doc, edit).map_err(failed)?; + let warnings = self.apply_with_warnings(doc, edit)?; let r = self.doc(a)?.accessibility_check(&Options { rules: [rule].into(), pages: None }).ok_or_else(|| failed("the document can't be read"))?; - Ok(json!({ "rule": id, "status": status_id(r.results.iter().find(|x| x.rule == rule).map_or(Status::Skipped, |x| x.status)) })) + let out = json!({ "rule": id, "status": status_id(r.results.iter().find(|x| x.rule == rule).map_or(Status::Skipped, |x| x.status)) }); + Ok(with_warnings(out, warnings)) } pub(crate) fn accessibility_figures(&self, a: &Args) -> Result { @@ -466,8 +467,8 @@ impl Automation { pdfcraft_engine::Edit::SetAltText { figure, alt: a.opt_str("alt")?.map(str::to_owned) } }; let id = self.doc(a)?.id; - self.session.apply(id, edit).map_err(failed)?; - self.accessibility_figures(a) + let warnings = self.apply_with_warnings(id, edit)?; + Ok(with_warnings(self.accessibility_figures(a)?, warnings)) } /// The structure tags, one page at a time (`offset`, `limit`), optionally only some types, pages @@ -551,7 +552,22 @@ impl Automation { let generation = u16::try_from(a.int("generation")?) .map_err(|_| ToolError::InvalidArgs("generation must be a PDF object generation from accessibility_tags".into()))?; let id = self.doc(a)?.id; - self.session.apply(id, pdfcraft_engine::Edit::MarkTagArtifact { object: pdfcraft_cos::ObjRef::new(object, generation) }).map_err(failed)?; + let warnings = + self.apply_with_warnings(id, pdfcraft_engine::Edit::MarkTagArtifact { object: pdfcraft_cos::ObjRef::new(object, generation) })?; + Ok(with_warnings(self.accessibility_tags(a)?, warnings)) + } + + pub(crate) fn accessibility_layout_table_to_blocks(&mut self, a: &Args) -> Result { + let object = u32::try_from(a.int("object")?).map_err(|_| ToolError::InvalidArgs("object must be a positive tag object number".into()))?; + if object == 0 { + return Err(ToolError::InvalidArgs("object must be a positive tag object number".into())); + } + let generation = u16::try_from(a.int("generation")?) + .map_err(|_| ToolError::InvalidArgs("generation must be a PDF object generation from accessibility_tags".into()))?; + let id = self.doc(a)?.id; + self.session + .apply(id, pdfcraft_engine::Edit::LayoutTableToBlocks { object: pdfcraft_cos::ObjRef::new(object, generation) }) + .map_err(failed)?; self.accessibility_tags(a) } @@ -573,9 +589,32 @@ impl Automation { other => return Err(ToolError::InvalidArgs(format!("placement must be before or after, not {other:?}"))), }; let id = self.doc(a)?.id; - self.session.apply(id, pdfcraft_engine::Edit::MoveTag { object, anchor, placement }).map_err(failed)?; - self.accessibility_tags(a) + let warnings = self.apply_with_warnings(id, pdfcraft_engine::Edit::MoveTag { object, anchor, placement })?; + Ok(with_warnings(self.accessibility_tags(a)?, warnings)) } + + /// Apply `edit` to document `id` and take out the errors it added to the document's script + /// output, such as the values of an XFA form that could not be written to its data. + fn apply_with_warnings(&mut self, id: DocId, edit: pdfcraft_engine::Edit) -> Result> { + let ((), out) = self.session.with_own_js_output(id, |s| s.apply(id, edit)).map_err(failed)?; + Ok(out.errors) + } +} + +/// `out` with `warnings` added to its `warnings` list (made when it has none). +fn with_warnings(mut out: Value, warnings: Vec) -> Value { + if warnings.is_empty() { + return out; + } + if let Some(map) = out.as_object_mut() { + match map.get_mut("warnings").and_then(Value::as_array_mut) { + Some(list) => list.extend(warnings.into_iter().map(Value::from)), + None => { + map.insert("warnings".into(), json!(warnings)); + } + } + } + out } #[cfg(test)] diff --git a/crates/automation/src/a11y/documents.rs b/crates/automation/src/a11y/documents.rs index 5ae3d4fe46..137c98c5d4 100644 --- a/crates/automation/src/a11y/documents.rs +++ b/crates/automation/src/a11y/documents.rs @@ -1,4 +1,4 @@ -use super::{Args, Automation, Result, ToolError, Value, child, failed, json, unused, write_atomic}; +use super::{Args, Automation, Result, ToolError, Value, child, failed, json, with_warnings, write_atomic}; impl Automation { pub(crate) fn doc_export_office(&self, a: &Args) -> Result { @@ -103,8 +103,8 @@ impl Automation { pub(crate) fn pdfa_convert(&mut self, a: &Args) -> Result { let level = self.pdfa_level(a)?; let id = self.doc(a)?.id; - self.session.apply(id, pdfcraft_engine::Edit::ConvertPdfA { level }).map_err(failed)?; - self.pdfa_verify(a) + let warnings = self.apply_with_warnings(id, pdfcraft_engine::Edit::ConvertPdfA { level })?; + Ok(with_warnings(self.pdfa_verify(a)?, warnings)) } pub(crate) fn action_list(&self) -> Result { @@ -148,18 +148,7 @@ impl Automation { .and_then(|bytes| run_on(&action, &name, std::sync::Arc::new(bytes), |_, _| {})); out.push(match result { Ok(r) => { - let target = child(&folder, &name); - let target = if target.exists() { - let stem = - std::path::Path::new(&name).file_stem().map(|s| s.to_string_lossy().into_owned()).unwrap_or_else(|| "document".into()); - unused(&folder, &stem) - } else { - target - }; - if target.exists() { - return Err(failed(format!("{}: no unused output name available", target.display()))); - } - write_atomic(&target, &r.bytes)?; + let target = crate::batch::publish(&child(&folder, &name), &r.bytes)?; json!({ "path": p, "output": target.to_string_lossy(), "log": r.log }) } Err(e) => json!({ "path": p, "error": e }), @@ -220,7 +209,7 @@ impl Automation { pub(crate) fn doc_compare_mark(&mut self, a: &Args) -> Result { let (old, new) = self.compare_ids(a)?; - let n = self.session.mark_differences(old, new).map_err(failed)?; - Ok(json!({ "comments": n })) + let (n, out) = self.session.with_own_js_output(new, |s| s.mark_differences(old, new)).map_err(failed)?; + Ok(with_warnings(json!({ "comments": n }), out.errors)) } } diff --git a/crates/automation/src/a11y/form_tools.rs b/crates/automation/src/a11y/form_tools.rs index 2420a44372..ae9b624035 100644 --- a/crates/automation/src/a11y/form_tools.rs +++ b/crates/automation/src/a11y/form_tools.rs @@ -1,4 +1,4 @@ -use super::{Args, Automation, Result, ToolError, Value, failed, json, write_atomic}; +use super::{Args, Automation, Result, ToolError, Value, failed, json, with_warnings, write_atomic}; /// A script's request to the viewer as the JSON that running a script reports; a side panel /// goes by its fixed id. @@ -12,6 +12,7 @@ fn request_json(r: &pdfcraft_engine::js::Request) -> Value { R::Submit(u) => json!({ "submit": u }), R::Focus(f) => json!({ "focus": f }), R::Beep => json!({ "beep": true }), + R::Highlight(on) => json!({ "highlight": on }), R::SaveAs => json!({ "save_as": true }), R::ShowPanel(p) => json!({ "panel": p.id() }), } @@ -19,7 +20,7 @@ fn request_json(r: &pdfcraft_engine::js::Request) -> Value { /// A message a script showed, as `js_run` reports it: its text, title (null when it gave none), /// icon and buttons. -fn alert_json(m: &pdfcraft_engine::js::Message) -> Value { +pub(crate) fn alert_json(m: &pdfcraft_engine::js::Message) -> Value { json!({ "message": m.text, "title": m.title, "icon": m.icon.name(), "buttons": m.buttons.name() }) } @@ -28,10 +29,18 @@ impl Automation { /// messages, console lines, requests, error, the failing document scripts' errors and result. pub(crate) fn js_run(&mut self, a: &Args) -> Result { let id = self.doc(a)?.id; - let o = self.session.run_javascript(id, a.str("script")?, a.opt_str("field")?).map_err(failed)?; + let (script, field) = (a.str("script")?, a.opt_str("field")?); + // Applying the script's field changes runs the field scripts again; their output is reported with the script's. + let (mut o, edit) = self.session.with_own_js_output(id, |s| s.run_javascript(id, script, field)).map_err(failed)?; + o.alerts.extend(edit.alerts); + o.console.extend(edit.console); + o.requests.extend(edit.requests); let reqs: Vec = o.requests.iter().map(request_json).collect(); let alerts: Vec = o.alerts.iter().map(alert_json).collect(); - Ok(json!({ "alerts": alerts, "console": o.console, "requests": reqs, "error": o.error, "document_errors": o.doc_errors, "result": o.result })) + Ok(with_warnings( + json!({ "alerts": alerts, "console": o.console, "requests": reqs, "error": o.error, "document_errors": o.doc_errors, "result": o.result }), + edit.errors, + )) } pub(crate) fn js_document_scripts(&self, a: &Args) -> Result { @@ -39,13 +48,17 @@ impl Automation { Ok(json!({ "scripts": list })) } + /// `js_set_document_script`: set the document-level script `name` to `script`, or remove it + /// when `script` is left out, as one undoable edit; returns the document's scripts. pub(crate) fn js_set_document_script(&mut self, a: &Args) -> Result { let edit = pdfcraft_engine::Edit::SetDocumentScript { name: a.str("name")?.into(), script: a.opt_str("script")?.map(str::to_string) }; let id = self.doc(a)?.id; - self.session.apply(id, edit).map_err(failed)?; - self.js_document_scripts(a) + let warnings = self.apply_with_warnings(id, edit)?; + Ok(with_warnings(self.js_document_scripts(a)?, warnings)) } + /// `form_set_script`: set field `field`'s JavaScript for `event` to `script`, or remove it when + /// `script` is left out, as one undoable edit; returns the console output and errors. pub(crate) fn form_set_script(&mut self, a: &Args) -> Result { let edit = pdfcraft_engine::Edit::SetFieldScript { name: a.str("field")?.into(), @@ -53,11 +66,41 @@ impl Automation { script: a.opt_str("script")?.map(str::to_string), }; let id = self.doc(a)?.id; - self.session.apply(id, edit).map_err(failed)?; - let out = self.session.take_js_output(id); + let ((), out) = self.session.with_own_js_output(id, |s| s.apply(id, edit)).map_err(failed)?; Ok(json!({ "field": a.str("field")?, "event": a.str("event")?, "console": out.console, "errors": out.errors })) } + /// `form_focus`: field `field` (its 1-based `widget`) gains (`enter`) or loses (`exit`) + /// focus, running its XFA scripts for that event; returns whether the form changed and what + /// the scripts showed or asked for. + pub(crate) fn form_focus(&mut self, a: &Args) -> Result { + use pdfcraft_engine::js::FieldFocus; + let focus = match a.str("event")? { + "enter" => FieldFocus::Enter, + "exit" => FieldFocus::Exit, + other => return Err(ToolError::InvalidArgs(format!("event must be enter or exit, not {other:?}"))), + }; + let name = a.str("field")?; + let doc = self.doc(a)?; + let field = doc.form.iter().find(|f| f.name == name).ok_or_else(|| failed(format!("there is no field named {name:?} (see form_fields)")))?; + // A field with no widget (one only scripts or calculations use) has nothing to focus. + let widgets = field.widgets.len(); + let widget = a.opt_int("widget")?.unwrap_or(1); + let widget = usize::try_from(widget).ok().and_then(|w| w.checked_sub(1)).filter(|w| *w < widgets).ok_or_else(|| match widgets { + 0 => ToolError::InvalidArgs(format!("{name:?} has no widget to focus")), + n => ToolError::InvalidArgs(format!("widget must be 1 to {n} for {name:?}")), + })?; + let id = doc.id; + let changed = self.session.field_focus(id, name, widget, focus).map_err(failed)?; + let out = self.session.take_js_output(id); + let reqs: Vec = out.requests.iter().map(request_json).collect(); + Ok( + json!({ "changed": changed, "alerts": out.alerts.iter().map(alert_json).collect::>(), "console": out.console, "requests": reqs, "errors": out.errors }), + ) + } + + /// `form_merge_data`: merge the form data files at `paths` into one CSV at `path`, a row per + /// file; returns the path and the row and column counts. pub(crate) fn form_merge_data(&mut self, a: &Args) -> Result { let mut files = Vec::new(); for p in a.strs("paths")? { @@ -90,10 +133,12 @@ impl Automation { v }) .collect(); + let mut warnings = Vec::new(); if a.opt_bool("add")?.unwrap_or(true) && !found.is_empty() { - self.session.auto_detect_fields(id, &pages).map_err(failed)?; + let (_, out) = self.session.with_own_js_output(id, |s| s.auto_detect_fields(id, &pages)).map_err(failed)?; + warnings = out.errors; } - Ok(json!({ "fields": list })) + Ok(with_warnings(json!({ "fields": list }), warnings)) } pub(crate) fn form_actions(&self, a: &Args) -> Result { @@ -157,8 +202,8 @@ impl Automation { let edit = pdfcraft_engine::Edit::SetFieldProps { name, props: Box::new(pdfcraft_engine::FieldProps { actions: Some(acts), ..Default::default() }) }; let id = self.doc(a)?.id; - self.session.apply(id, edit).map_err(failed)?; - self.form_actions(a) + let warnings = self.apply_with_warnings(id, edit)?; + Ok(with_warnings(self.form_actions(a)?, warnings)) } pub(crate) fn js_enabled(&mut self, a: &Args) -> Result { diff --git a/crates/automation/src/a11y/ocr.rs b/crates/automation/src/a11y/ocr.rs index 2846aa44a6..6d1eaaa389 100644 --- a/crates/automation/src/a11y/ocr.rs +++ b/crates/automation/src/a11y/ocr.rs @@ -1,4 +1,4 @@ -use super::{Args, Automation, Result, ToolError, Value, child, failed, json, write_atomic}; +use super::{Args, Automation, Result, ToolError, Value, child, failed, json, with_warnings, write_atomic}; impl Automation { /// The settings every OCR tool shares: dpi, language, the engine/strategy choice, the four @@ -258,9 +258,10 @@ impl Automation { let recognizers = pdfcraft_engine::ocr::recognizers(&settings).map_err(failed)?; match out { None => { - let found = self.session.recognize_text(id, &pages, settings).map_err(failed)?; + let (found, out) = self.session.with_own_js_output(id, |s| s.recognize_text(id, &pages, settings)).map_err(failed)?; + let warnings = out.errors; let words = found.iter().map(|p| p.words.len()).sum::(); - Ok(json!({ "words": words, "pages": Self::ocr_pages_json(&found, copy) })) + Ok(with_warnings(json!({ "words": words, "pages": Self::ocr_pages_json(&found, copy) }), warnings)) } Some(out) => { let job = self.session.ocr_job(id, &pages, settings).ok_or_else(|| failed("the document could not be read"))?; diff --git a/crates/automation/src/assemble.rs b/crates/automation/src/assemble.rs index deb2efa18a..5fd9d9b0be 100644 --- a/crates/automation/src/assemble.rs +++ b/crates/automation/src/assemble.rs @@ -36,8 +36,9 @@ impl Automation { } (None, None) => return Err(ToolError::InvalidArgs("text needs `text` or `path`".into())), }; - // The page font is Helvetica/WinAnsi: text it would turn into `?` is refused (with - // its line) by `pdfcraft_create::from_text`, as for every other caller. + // The page font is Helvetica/WinAnsi, or the bundled Inter face for Latin, Greek and + // Cyrillic letters WinAnsi lacks: text neither can draw is refused (with its line) + // by `pdfcraft_create::from_text`, as for every other caller. let bytes = self.session.create_from_text(&title, &text).map_err(|e| match e { pdfcraft_engine::EditError::Create(pdfcraft_engine::CreateError::Invalid(m)) => ToolError::InvalidArgs(m), e => failed(e), diff --git a/crates/automation/src/batch.rs b/crates/automation/src/batch.rs index f650e565b7..14a98f23e2 100644 --- a/crates/automation/src/batch.rs +++ b/crates/automation/src/batch.rs @@ -281,7 +281,7 @@ impl Automation { } /// Publish atomically without replacing any existing name, even if another writer wins a race. -fn publish(target: &Path, bytes: &[u8]) -> Result { +pub(crate) fn publish(target: &Path, bytes: &[u8]) -> Result { let dir = target.parent().ok_or_else(|| failed("output has no parent"))?; std::fs::create_dir_all(dir).map_err(failed)?; let mut temp = tempfile::NamedTempFile::new_in(dir).map_err(failed)?; diff --git a/crates/automation/src/bates.rs b/crates/automation/src/bates.rs index 5267716e85..71ae6b3b91 100644 --- a/crates/automation/src/bates.rs +++ b/crates/automation/src/bates.rs @@ -8,6 +8,8 @@ use super::json::Args; use super::paths::read_input; use super::{Arc, Automation, Path, PathBuf}; +const MAX_BATES_BYTES: u64 = 512 * 1024 * 1024; + struct FilePlan { input: String, source: PathBuf, @@ -26,19 +28,23 @@ impl Automation { } let start = match args.0.get("start_number") { None | Some(Value::Null) => 1, - Some(value) => value - .as_u64() - .filter(|n| *n >= 1) - .ok_or_else(|| ToolError::InvalidArgs("start_number must be a whole number of at least 1".into()))?, + Some(value) => { + value.as_u64().filter(|n| *n >= 1).ok_or_else(|| ToolError::InvalidArgs("start_number must be an integer of at least 1".into()))? + } }; let digits = args.opt_int("digits")?.unwrap_or(6); let digits = u8::try_from(digits).ok().filter(|d| (1..=12).contains(d)).ok_or_else(|| ToolError::InvalidArgs("digits must be from 1 to 12".into()))?; let prefix = args.opt_str("prefix")?.unwrap_or(""); let suffix = args.opt_str("suffix")?.unwrap_or(""); - // `#` separates the token's fields and `>>` ends it: either would cut the token short. - if prefix.len() > 128 || suffix.len() > 128 || [prefix, suffix].iter().any(|s| s.contains('#') || s.contains(">>")) { - return Err(ToolError::InvalidArgs("prefix and suffix must be at most 128 bytes and cannot contain # or >>".into())); + if prefix.chars().count() > 128 + || suffix.chars().count() > 128 + || prefix.contains('#') + || suffix.contains('#') + || prefix.contains(">>") + || suffix.contains(">>") + { + return Err(ToolError::InvalidArgs("prefix and suffix must be at most 128 characters and cannot contain # or >>".into())); } let dry_run = args.opt_bool("dry_run")?.unwrap_or(false); @@ -70,16 +76,7 @@ impl Automation { targets.push((output.to_string(), target)); passwords.push(entry.get("password").and_then(Value::as_str).map(str::to_owned)); } - let input_paths: std::collections::HashSet<_> = sources.iter().map(|(_, path)| path.clone()).collect(); - let mut output_paths = std::collections::HashSet::new(); - for (name, path) in &targets { - if input_paths.contains(path) { - return Err(failed(format!("output {name} names an input file; choose a separate output path"))); - } - if !output_paths.insert(path.clone()) { - return Err(failed(format!("two files use the same output path: {name}"))); - } - } + check_unique_outputs(&targets, dry_run)?; let mut plans = Vec::with_capacity(files.len()); let mut next = start; @@ -88,13 +85,13 @@ impl Automation { let size = std::fs::metadata(input_path).map_err(failed)?.len(); input_bytes = input_bytes .checked_add(size) - .filter(|total| *total <= 512 * 1024 * 1024) + .filter(|total| *total <= MAX_BATES_BYTES) .ok_or_else(|| failed("combined Bates input exceeds the 512 MiB safety limit"))?; let bytes = read_input(input_path)?; input_bytes = input_bytes .saturating_sub(size) .checked_add(u64::try_from(bytes.len()).unwrap_or(u64::MAX)) - .filter(|total| *total <= 512 * 1024 * 1024) + .filter(|total| *total <= MAX_BATES_BYTES) .ok_or_else(|| failed("combined Bates input exceeds the 512 MiB safety limit"))?; let display_name = input_path.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default(); let password = passwords.get(index).and_then(Option::as_deref); @@ -106,7 +103,8 @@ impl Automation { return Err(failed(format!("{input_name} has no pages to number"))); } let count = u64::try_from(pages).map_err(|_| failed(format!("page count is too large for {input_name}")))?; - let last = next.checked_add(count.saturating_sub(1)).ok_or_else(|| failed(format!("Bates number range overflows for {input_name}")))?; + let last_offset = count.checked_sub(1).ok_or_else(|| failed(format!("{input_name} has no pages to number")))?; + let last = next.checked_add(last_offset).ok_or_else(|| failed(format!("Bates number range overflows for {input_name}")))?; plans.push(FilePlan { input: input_name.clone(), source: input_path.clone(), @@ -116,7 +114,7 @@ impl Automation { first: next, last, }); - if index + 1 < sources.len() { + if index + 1 < files.len() { next = last.checked_add(1).ok_or_else(|| failed(format!("Bates sequence overflows after {input_name}")))?; } } @@ -163,12 +161,14 @@ impl Automation { let result = applied.and_then(|()| self.session.save_full_bytes(id)); match result { Ok(bytes) => { - let total = output_bytes.saturating_add(u64::try_from(bytes.len()).unwrap_or(u64::MAX)); - if total > 512 * 1024 * 1024 { - results.push(json!({"input":plan.input,"output":plan.output,"status":"error","pages":plan.pages,"first":plan.first,"last":plan.last,"error":"combined Bates output exceeds the 512 MiB safety limit"})); - } else { + let candidate = output_bytes + .checked_add(u64::try_from(bytes.len()).unwrap_or(u64::MAX)) + .filter(|total| *total <= MAX_BATES_BYTES); + if let Some(total) = candidate { output_bytes = total; staged.push((plan.output.clone(), bytes)); + } else { + results.push(json!({"input":plan.input,"output":plan.output,"status":"error","pages":plan.pages,"first":plan.first,"last":plan.last,"error":"combined Bates output exceeds the 512 MiB safety limit"})); } } Err(error) => results.push(json!({"input":plan.input,"output":plan.output,"status":"error","pages":plan.pages,"first":plan.first,"last":plan.last,"error":error.to_string()})), @@ -198,6 +198,123 @@ impl Automation { } } +fn check_unique_outputs(targets: &[(String, PathBuf)], dry_run: bool) -> Result<()> { + if targets.len() < 2 { + return Ok(()); + } + + // Probe exact basenames in a directory under each canonical parent, so the filesystem applies its own name rules. + let mut groups: std::collections::HashMap)> = std::collections::HashMap::new(); + for (name, target) in targets { + let parent = target.parent().ok_or_else(|| failed(format!("output has no parent directory: {name}")))?; + let identity = parent.canonicalize().map_err(failed)?; + let probe_name = output_probe_name(target).map_err(failed)?; + groups.entry(identity.clone()).or_insert_with(|| (identity, Vec::new())).1.push((name.clone(), probe_name)); + } + + if dry_run { + // Dry runs cannot probe the volume: Windows uses a conservative case-insensitive comparison, + // while Unix preserves case, so unusual case-sensitive Windows or case-insensitive Unix volumes can differ. + for (_, (_, names)) in groups { + for (index, (name, candidate)) in names.iter().enumerate() { + if names.iter().take(index).any(|(_, prior)| same_output_name(prior, candidate)) { + return Err(failed(format!("two files use the same output path: {name}"))); + } + } + } + return Ok(()); + } + + for (_, (parent, names)) in groups { + match tempfile::tempdir_in(&parent) { + Ok(probes) => { + for (name, probe_name) in names { + let probe = probes.path().join(probe_name); + match std::fs::OpenOptions::new().write(true).create_new(true).open(&probe) { + Ok(file) => drop(file), + Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => { + return Err(failed(format!("two files use the same output path: {name}"))); + } + Err(error) => return Err(failed(format!("could not validate output path {name}: {error}"))), + } + } + } + #[cfg(windows)] + Err(error) if error.kind() == std::io::ErrorKind::PermissionDenied => probe_files_for_resolved_outputs(&parent, names)?, + Err(error) => return Err(failed(error)), + } + } + Ok(()) +} + +#[cfg(windows)] +fn probe_files_for_resolved_outputs(parent: &Path, names: Vec<(String, std::ffi::OsString)>) -> Result<()> { + use std::os::windows::fs::OpenOptionsExt; + + const FILE_FLAG_DELETE_ON_CLOSE: u32 = 0x0400_0000; + let mut probes = Vec::with_capacity(names.len()); + for (name, probe_name) in names { + let probe = parent.join(probe_name); + let mut options = std::fs::OpenOptions::new(); + options.write(true).create_new(true).share_mode(0).custom_flags(FILE_FLAG_DELETE_ON_CLOSE); + match options.open(&probe) { + Ok(file) => probes.push(file), + Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => { + return Err(failed(format!("two files use the same output path: {name}"))); + } + Err(error) => return Err(failed(format!("could not validate output path {name}: {error}"))), + } + } + drop(probes); + Ok(()) +} + +fn same_output_name(left: &std::ffi::OsStr, right: &std::ffi::OsStr) -> bool { + #[cfg(windows)] + { + use std::os::windows::ffi::OsStrExt; + fn key(name: &std::ffi::OsStr) -> Vec { + std::char::decode_utf16(name.encode_wide()) + .map(|decoded| match decoded { + Ok(character) => { + let mut uppercase = character.to_uppercase(); + match (uppercase.next(), uppercase.next()) { + (Some(single), None) => single as u32, + _ => character as u32, + } + } + Err(error) => 0x11_0000 + u32::from(error.unpaired_surrogate()), + }) + .collect() + } + key(left) == key(right) + } + #[cfg(not(windows))] + { + left == right + } +} + +fn output_probe_name(path: &Path) -> std::io::Result { + let name = path.file_name().ok_or_else(|| std::io::Error::new(std::io::ErrorKind::InvalidInput, "output has no file name"))?; + #[cfg(windows)] + { + use std::os::windows::ffi::{OsStrExt, OsStringExt}; + let is_verbatim = path.components().any(|component| matches!(component, std::path::Component::Prefix(prefix) if matches!(prefix.kind(), std::path::Prefix::Verbatim(_) | std::path::Prefix::VerbatimUNC(_, _) | std::path::Prefix::VerbatimDisk(_)))); + let mut wide: Vec = name.encode_wide().collect(); + if !is_verbatim { + while wide.last().is_some_and(|unit| *unit == u16::from(b'.') || *unit == u16::from(b' ')) { + wide.pop(); + } + } + Ok(std::ffi::OsString::from_wide(&wide)) + } + #[cfg(not(windows))] + { + Ok(name.to_os_string()) + } +} + /// Publish a PDF atomically while refusing a target created by another process after preflight. pub(super) fn publish_new(path: &Path, bytes: &[u8]) -> Result<()> { let parent = path.parent().ok_or_else(|| failed("output has no parent directory"))?; @@ -212,3 +329,104 @@ pub(super) fn publish_new(path: &Path, bytes: &[u8]) -> Result<()> { } }) } + +#[cfg(test)] +mod tests { + use super::check_unique_outputs; + + #[test] + fn dry_run_path_check_creates_no_probe_files() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("same.pdf"); + let targets = vec![("first.pdf".into(), path.clone()), ("second.pdf".into(), path)]; + assert!(check_unique_outputs(&targets, true).is_err()); + assert_eq!(std::fs::read_dir(dir.path()).unwrap().count(), 0); + } + + #[cfg(unix)] + #[test] + fn dry_run_detects_output_parent_aliases_without_writing() { + let dir = tempfile::tempdir().unwrap(); + let alias = dir.path().join("alias"); + std::os::unix::fs::symlink(dir.path(), &alias).unwrap(); + let first = dir.path().join("same.pdf"); + let second = alias.join("same.pdf"); + let targets = vec![("first.pdf".into(), first), ("second.pdf".into(), second)]; + assert!(check_unique_outputs(&targets, true).is_err()); + assert_eq!(std::fs::read_dir(dir.path()).unwrap().count(), 1); + } + + #[test] + fn long_output_names_fit_in_probe_directory() { + let dir = tempfile::tempdir().unwrap(); + let first = dir.path().join(format!("{}.pdf", "a".repeat(220))); + let second = dir.path().join(format!("{}.pdf", "b".repeat(220))); + let targets = vec![("first.pdf".into(), first), ("second.pdf".into(), second)]; + assert!(check_unique_outputs(&targets, false).is_ok()); + assert_eq!(std::fs::read_dir(dir.path()).unwrap().count(), 0); + } + + #[cfg(unix)] + #[test] + fn output_probe_paths_preserve_non_utf8_names() { + use std::os::unix::ffi::OsStringExt; + let dir = tempfile::tempdir().unwrap(); + let first = dir.path().join(std::ffi::OsString::from_vec(vec![b'o', 0xff])); + let second = dir.path().join(std::ffi::OsString::from_vec(vec![b'o', 0xfe])); + let targets = vec![("first.pdf".into(), first), ("second.pdf".into(), second)]; + assert!(check_unique_outputs(&targets, false).is_ok()); + assert_eq!(std::fs::read_dir(dir.path()).unwrap().count(), 0); + } + + #[cfg(windows)] + #[test] + fn windows_probe_names_trim_only_non_verbatim_paths() { + use std::ffi::OsString; + use std::path::Path; + + let normal = Path::new(r"C:\root\report.pdf. "); + assert_eq!(super::output_probe_name(normal).unwrap(), OsString::from("report.pdf")); + + let verbatim = Path::new(r"\\?\C:\root\report.pdf. "); + assert_eq!(super::output_probe_name(verbatim).unwrap(), OsString::from("report.pdf. ")); + } + + #[cfg(windows)] + #[test] + fn dry_run_detects_case_only_output_collisions_without_writing() { + let dir = tempfile::tempdir().unwrap(); + let first = dir.path().join("Case.pdf"); + let second = dir.path().join("CASE.pdf"); + let targets = vec![("first.pdf".into(), first), ("second.pdf".into(), second)]; + assert!(check_unique_outputs(&targets, true).is_err()); + assert_eq!(std::fs::read_dir(dir.path()).unwrap().count(), 0); + } + + #[cfg(windows)] + #[test] + fn permission_denied_child_directory_uses_delete_on_close_file_probes() { + use std::process::Command; + + let dir = tempfile::tempdir().unwrap(); + let identity = Command::new("whoami.exe").args(["/user", "/fo", "csv", "/nh"]).output().unwrap(); + assert!(identity.status.success()); + let identity = String::from_utf8(identity.stdout).unwrap(); + let sid = identity.split('"').nth(3).unwrap(); + let acl = Command::new("icacls.exe").arg(dir.path()).arg("/deny").arg(format!("*{sid}:(AD)")).status().unwrap(); + assert!(acl.success()); + + // Where the deny entry does not bind the test's own account (it did not on CI's Windows image), + // there is no denied folder to probe. + if !matches!(tempfile::tempdir_in(dir.path()), Err(error) if error.kind() == std::io::ErrorKind::PermissionDenied) { + return; + } + let direct_probe = dir.path().join("direct-probe"); + let file = std::fs::OpenOptions::new().write(true).create_new(true).open(&direct_probe).unwrap(); + drop(file); + std::fs::remove_file(&direct_probe).unwrap(); + + let targets = vec![("first.pdf".into(), dir.path().join("first.pdf")), ("second.pdf".into(), dir.path().join("second.pdf"))]; + assert!(check_unique_outputs(&targets, false).is_ok()); + assert_eq!(std::fs::read_dir(dir.path()).unwrap().count(), 0); + } +} diff --git a/crates/automation/src/comments.rs b/crates/automation/src/comments.rs index 78f4cefbe0..e4da78ab09 100644 --- a/crates/automation/src/comments.rs +++ b/crates/automation/src/comments.rs @@ -179,7 +179,21 @@ impl Automation { let doc = self.doc(a)?; let only = a.opt_int("page")?.map(|p| (p.max(1) - 1) as usize); let all = self.comments(a)?; + // A group (`/RT /Group`, ISO 32000-2 §12.5.6.2) lists as its primary, with the primary's + // attributes: the subordinates are neither comments nor replies, only its `group`. + let mut members: HashMap<(usize, usize), Vec<&Annotation>> = HashMap::new(); + for (m, p) in all.iter().zip(pdfcraft_render::group_primaries(&all)) { + if let Some(p) = p.and_then(|p| all.get(p)) { + members.entry((p.page, p.index)).or_default().push(m); + } + } let view = |c: &Annotation| -> Value { + let group: Vec = members + .get(&(c.page, c.index)) + .into_iter() + .flatten() + .map(|m| json!({ "id": m.name, "page": m.page + 1, "index": m.index + 1, "type": m.subtype })) + .collect(); let mut v = json!({ "id": c.name, "page": c.page + 1, @@ -191,6 +205,9 @@ impl Automation { "modified": c.modified, "rect": crate::view_rect(&doc.info, c.page, c.shown_rect(doc.info.pages.get(c.page).map_or(0, |p| p.rotation))), "color": c.color.map(hex), + "group": group, + // Unchanged on a page the viewer could not read, as `rect` is. + "popup": c.popup.map(|w| json!({ "rect": crate::view_rect(&doc.info, c.page, w.rect), "open": w.open })), }); // A text box's or typed text's oval or circle (a rectangle is not reported). if c.subtype == "FreeText" @@ -211,8 +228,8 @@ impl Automation { v["text_opacity"] = json!(t); } } - // A line's ends, or the corners of connected lines, a polygon or a cloud, as - // comment_edit's `points` takes them. + // A line's ends, the corners of connected lines, a polygon or a cloud, or a callout's + // arrow point and knee, as comment_edit's `points` takes them. // Unchanged on a page the viewer could not read, as `rect` is. if !c.points.is_empty() { let round = |v: f32| (v * 100.0).round() / 100.0; @@ -220,28 +237,36 @@ impl Automation { v["points"] = json!(c.points.iter().map(|q| page.map_or([q[0], q[1]], |p| p.user_to_view(q[0], q[1])).map(round)).collect::>()); } + // A callout's text box, as comment_edit's `rect` takes it (`rect` also holds the line). + if let Some(tb) = c.text_box { + v["text_box"] = json!(crate::view_rect(&doc.info, c.page, tb)); + } v }; // Every reply under the comment its thread starts from, at any depth, in document order: // a reply to a reply keeps its `/IRT` on the direct parent, so following one level would - // drop it. Parents are found by reference, as `/NM` need not be unique. A status or - // checkmark belongs to the annotation its `/IRT` names (ISO 32000-2 §12.5.6.2), as in the - // Comments panel: one set on a reply is not the comment's. + // drop it. Parents are found by reference, as `/NM` need not be unique. Status and + // checkmark replies are read through `review` instead. A group subordinate is in no + // thread, and a reply to one is the group's, its primary's. let tops = pdfcraft_render::thread_tops(&all); let mut threads: HashMap<(usize, usize), Vec<&Annotation>> = HashMap::new(); - for r in &all { + for r in all.iter().filter(|r| r.state.is_none() && !r.is_group_member()) { let Some(&top) = tops.get(&(r.page, r.index)) else { continue }; - let key = match r.parent { - Some(parent) if r.state.is_some() => parent, - _ => top, - }; - threads.entry(key).or_default().push(r); + threads.entry(top).or_default().push(r); } - // An annotation's review status and checkmark: the latest of each set on it. + // An annotation's review states. A state reply's `/IRT` refers to the annotation whose + // state it sets, and a reviewer's later change replies to their previous one (ISO 32000-2 + // §12.5.6.3), so a status on a reply is that reply's, as in the Comments panel. `status` + // is the latest set by anyone; `statuses` has each reviewer's latest (a state is "for the + // specified user"). Checkmarks are each reviewer's own too: `marked_by` names who has one + // on, and `marked` whether anyone has. + let review = pdfcraft_render::ReviewStates::new(&all); let states = |c: &Annotation, v: &mut Value| { - let set = threads.get(&(c.page, c.index)).map(Vec::as_slice).unwrap_or(&[]); - v["status"] = json!(set.iter().rev().filter(|r| !r.is_mark()).find_map(|r| r.state.clone())); - v["marked"] = json!(set.iter().rev().find(|r| r.is_mark()).is_some_and(|r| r.state.as_deref() == Some("Marked"))); + v["status"] = json!(review.latest_status(c)); + v["statuses"] = json!(review.statuses(c).iter().map(|(who, state)| json!({ "author": who, "status": state })).collect::>()); + let marked_by = review.marked(c); + v["marked"] = json!(!marked_by.is_empty()); + v["marked_by"] = json!(marked_by); }; let roots: Vec = all .iter() @@ -253,7 +278,6 @@ impl Automation { let thread = threads.get(&(c.page, c.index)).map(Vec::as_slice).unwrap_or(&[]); v["replies"] = thread .iter() - .filter(|r| r.state.is_none()) .map(|r| { let mut reply = view(r); states(r, &mut reply); @@ -406,18 +430,8 @@ impl Automation { let point = to_user(&info, t[0], t[1]); let knee = match a.nums::<2>("knee")? { Some(k) => to_user(&info, k[0], k[1]), - // Halfway between the point and the box, level with the box's middle. - None => { - let mid = (rect[1] + rect[3]) / 2.0; - let side = if point[0] < rect[0] { - rect[0] - } else if point[0] > rect[2] { - rect[2] - } else { - (rect[0] + rect[2]) / 2.0 - }; - [(point[0] + side) / 2.0, mid] - } + // Acrobat's: 12 pt out from the middle of the side facing the point. + None => pdfcraft_engine::callout_knee(rect, point), }; Shape::Callout { rect, @@ -490,8 +504,12 @@ impl Automation { if contents.trim().is_empty() { return Err(ToolError::InvalidArgs("replace needs `contents`: the replacement text".into())); } + // `/C` is a group attribute, taken from the primary (ISO 32000-2 §12.5.6.2), so the + // caret carries the colour too. Only the colour: opacity and the rest are the + // caret's own. + let mut caret = Style::default_for(&Shape::Caret { rect: [0.0; 4] }); + caret.color = style.color; let strike = style; - let caret = Style::default_for(&Shape::Caret { rect: [0.0; 4] }); let mut out = self.apply(a, Edit::ReplaceText { page, quads, text: contents, author, strike, caret })?; out["comment"] = json!({ "page": page + 1, "type": "replace" }); return Ok(out); @@ -694,6 +712,17 @@ impl Automation { self.apply(a, Edit::MarkAnnotation { page, index, marked, author }) } + pub(crate) fn comment_popup(&mut self, a: &Args) -> Result { + let (page, index) = self.comment_target(a)?; + let info = self.doc(a)?.info.pages[page].clone(); + let open = a.opt_bool("open")?; + let rect = a.nums::<4>("rect")?.map(|r| rect_to_user(&info, r)); + if open.is_none() && rect.is_none() { + return Err(ToolError::InvalidArgs("nothing to change: pass open and/or rect".into())); + } + self.apply(a, Edit::SetAnnotationPopup { page, index, rect, open }) + } + pub(crate) fn comment_lock(&mut self, a: &Args) -> Result { let (page, index) = self.comment_target(a)?; let locked = a.opt_bool("locked")?.unwrap_or(true); @@ -867,6 +896,7 @@ impl Automation { let bytes = crate::read_input(&path)?; let file = path.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default(); let name = a.opt_str("name")?.map(str::to_owned).unwrap_or_else(|| file.rsplit_once('.').map_or(file.clone(), |(s, _)| s.to_owned())); + let author = a.opt_str("author")?.unwrap_or(DEFAULT_AUTHOR).to_string(); let edit = Edit::AddCustomStamp { page, rect, @@ -876,7 +906,16 @@ impl Automation { bytes: std::sync::Arc::new(bytes), page: a.opt_int("file_page")?.unwrap_or(1).max(1) as usize - 1, }, - author: a.opt_str("author")?.unwrap_or(DEFAULT_AUTHOR).to_string(), + line: None, + author: author.clone(), + }; + // A dynamic line (#1741): `{name}`, `{date}` and `{time}` filled in now. + let edit = match (edit, a.opt_str("line")?) { + (Edit::AddCustomStamp { page, rect, name, file, author, .. }, Some(t)) if !t.trim().is_empty() => { + let line = Some(self.session().stamp_line(t, &author)); + Edit::AddCustomStamp { page, rect, name, file, author, line } + } + (edit, _) => edit, }; self.apply(a, edit) } diff --git a/crates/automation/src/content.rs b/crates/automation/src/content.rs index 1f26dcc26a..a77ef98b8b 100644 --- a/crates/automation/src/content.rs +++ b/crates/automation/src/content.rs @@ -354,14 +354,32 @@ impl Automation { Ok(json!({"page":page+1,"generation":doc.edit_generation(),"count":items.len(),"objects":items})) } - pub(crate) fn object_move(&mut self, a: &Args) -> Result { + /// The references `objects` names (from object_list), or with `all` every object on `page`, + /// as ⌘A selects them in Edit a PDF; refused when `generation` is not the document's. + fn object_targets(&self, a: &Args, page: usize) -> Result> { use pdfcraft_engine::{MAX_MOVE_OBJECTS, ObjectKind, ObjectTarget}; - let page = self.page(a)?; + let doc = self.doc(a)?; + if let Some(g) = a.get("generation") { + let g = g.as_u64().ok_or_else(|| bad("generation must be a non-negative integer from object_list"))?; + if g != doc.edit_generation() { + return Err(failed("the document changed; call object_list again")); + } + } + if a.opt_bool("all")?.unwrap_or(false) { + if a.get("objects").is_some() { + return Err(bad("give objects or all, not both")); + } + let all: Vec<_> = doc.editable_objects(page).map_err(failed)?.into_iter().map(|o| o.target).collect(); + if all.is_empty() || all.len() > MAX_MOVE_OBJECTS { + return Err(failed(format!("page {} has {} objects; select 1–1000", page + 1, all.len()))); + } + return Ok(all); + } let rows = a .get("objects") .and_then(Value::as_array) .filter(|r| !r.is_empty() && r.len() <= MAX_MOVE_OBJECTS) - .ok_or_else(|| bad("objects must contain 1–1000 references from object_list"))?; + .ok_or_else(|| bad("objects must contain 1–1000 references from object_list (or pass all: true)"))?; let mut targets = Vec::with_capacity(rows.len()); for row in rows { let row = row.as_object().ok_or_else(|| bad("each object must contain kind and index"))?; @@ -382,6 +400,12 @@ impl Automation { .ok_or_else(|| bad("object index must be a positive integer from object_list"))?; targets.push(ObjectTarget { kind, index }); } + Ok(targets) + } + + pub(crate) fn object_move(&mut self, a: &Args) -> Result { + let page = self.page(a)?; + let targets = self.object_targets(a, page)?; let delta = a .get("offset") .and_then(Value::as_array) @@ -389,17 +413,46 @@ impl Automation { .and_then(|v| Some([v.first()?.as_f64()?, v.get(1)?.as_f64()?])) .filter(|v| v.iter().all(|n| n.is_finite() && n.abs() <= 1e9)) .ok_or_else(|| bad("offset must contain two finite numbers [dx, dy] in displayed points"))?; - let doc = self.doc(a)?; - if let Some(g) = a.get("generation") { - let g = g.as_u64().ok_or_else(|| bad("generation must be a non-negative integer from object_list"))?; - if g != doc.edit_generation() { - return Err(failed("the document changed; call object_list again before moving")); - } - } - let offset = doc.object_move_offset(page, delta).map_err(failed)?; + let offset = self.doc(a)?.object_move_offset(page, delta).map_err(failed)?; let count = targets.len(); let mut result = self.apply(a, Edit::MoveObjects { page, objects: targets, offset })?; result["moved"] = json!(count); Ok(result) } + + /// Edit ▸ Copy (or with `cut`, Cut) of objects selected together, kept for object_paste into + /// any open document. A cut then deletes them as one undo step. + pub(crate) fn object_copy(&mut self, a: &Args) -> Result { + let page = self.page(a)?; + let targets = self.object_targets(a, page)?; + let cut = a.opt_bool("cut")?.unwrap_or(false); + let doc = self.doc(a)?; + if cut && !doc.allows_modification() { + return Err(failed(pdfcraft_engine::EditError::NotPermitted("changes"))); + } + let clip = doc.copy_objects(page, &targets).map_err(failed)?; + let out = json!({ "copied": clip.objects().len(), "text": clip.text() }); + self.objects_clipboard = Some(pdfcraft_engine::ObjectClipboard::new(clip)); + if !cut { + return Ok(out); + } + let delete = Edit::delete_objects(page, targets).ok_or_else(|| failed("nothing to cut"))?; + let mut result = self.apply(a, delete)?; + result["copied"] = out["copied"].clone(); + result["text"] = out["text"].clone(); + Ok(result) + } + + /// Edit ▸ Paste of the objects object_copy copied: onto `page`, where they were, or on the + /// page they came from (and on each further paste onto a page) 18 points down and right. + pub(crate) fn object_paste(&mut self, a: &Args) -> Result { + let page = self.page(a)?; + let mut clipboard = self.objects_clipboard.take().ok_or_else(|| failed("copy objects first (object_copy)"))?; + let count = clipboard.clip().objects().len(); + let edit = self.doc(a).and_then(|doc| clipboard.paste(doc, page).map_err(failed)); + self.objects_clipboard = Some(clipboard); + let mut result = self.apply(a, edit?)?; + result["pasted"] = json!(count); + Ok(result) + } } diff --git a/crates/automation/src/dispatch.rs b/crates/automation/src/dispatch.rs index a838241705..d3850d1d8f 100644 --- a/crates/automation/src/dispatch.rs +++ b/crates/automation/src/dispatch.rs @@ -700,7 +700,7 @@ impl Automation { summary(self.session.get(new).ok_or_else(|| failed("the document vanished"))?) } "doc_export_images" | "doc_export_text" | "doc_export_all_images" => self.export(name, &a)?, - "doc_header_footer" | "doc_watermark" | "doc_background" | "doc_remove_marks" => self.marks(name, &a)?, + "doc_header_footer" | "doc_watermark" | "doc_background" | "doc_printer_marks" | "doc_remove_marks" => self.marks(name, &a)?, "doc_backdrops" => self.backdrops(&a)?, "doc_unprotect" => { let mut out = self.apply(&a, Edit::RemoveProtection)?; @@ -709,6 +709,7 @@ impl Automation { } "form_fields" => self.form_fields(&a)?, "form_fill" => self.form_fill(&a)?, + "form_focus" => self.form_focus(&a)?, "form_set_image" => { let path = self.resolve(a.str("path")?, false)?; let bytes = read_input(&path)?; @@ -738,6 +739,8 @@ impl Automation { "links_remove" => self.links_remove(&a)?, "object_list" => self.object_list(&a)?, "object_move" => self.object_move(&a)?, + "object_copy" => self.object_copy(&a)?, + "object_paste" => self.object_paste(&a)?, "content_list" => self.content_list(&a)?, "page_add_text" => self.page_add_text(&a)?, "page_add_image" => self.page_add_image(&a)?, @@ -745,6 +748,7 @@ impl Automation { "content_delete" => self.content_delete(&a)?, "content_copy" => self.content_copy(&a)?, "doc_print" => self.doc_print(&a)?, + "doc_fix_hairlines" => self.fix_hairlines(&a)?, "attachment_add" => { let path = self.resolve(a.str("path")?, false)?; let bytes = read_input(&path)?; @@ -764,6 +768,49 @@ impl Automation { let edit = Edit::DeleteAttachment { key: a.str("name")?.into() }; self.apply(&a, edit)? } + "doc_convert_colours" => self.convert_colours(&a)?, + "attachment_edit" => { + let key = a.opt_str("key")?; + let name = a.opt_str("name")?; + if key.is_some() == name.is_some() { + return Err(ToolError::InvalidArgs("provide exactly one of key or name".into())); + } + let keys: Vec = self + .doc(&a)? + .info + .attachments + .iter() + .filter_map(|attachment| match &attachment.source { + pdfcraft_render::AttachmentSource::Document { key: attachment_key } + if key.is_some_and(|key| key == attachment_key) || name.is_some_and(|name| name == attachment.name) => + { + Some(attachment_key.clone()) + } + pdfcraft_render::AttachmentSource::Document { .. } => None, + pdfcraft_render::AttachmentSource::Annotation { .. } => None, + }) + .collect(); + let key = match keys.as_slice() { + [key] => key.clone(), + [] if key.is_some() => { + return Err(ToolError::InvalidArgs(format!("there is no document attachment with key {:?}", key.unwrap_or_default()))); + } + [] => { + return Err(ToolError::InvalidArgs(format!( + "there is no document attachment named {:?}; use a name from doc_info", + name.unwrap_or_default() + ))); + } + _ => { + return Err(ToolError::InvalidArgs(format!( + "more than one document attachment is named {:?}; provide its key from doc_info", + name.unwrap_or_default() + ))); + } + }; + let edit = Edit::EditAttachmentDescription { key, description: Some(a.str("description")?.into()) }; + self.apply(&a, edit)? + } "doc_remove_hidden" => self.doc_remove_hidden(&a)?, "doc_audit_redactions" => self.doc_audit_redactions(&a)?, "fill_sign_add" => self.fill_sign_add(&a)?, @@ -832,6 +879,7 @@ impl Automation { "accessibility_tag_content" => self.accessibility_tag_content(&a)?, "accessibility_set_tag_property" => self.accessibility_set_tag_property(&a)?, "accessibility_mark_tag_artifact" => self.accessibility_mark_tag_artifact(&a)?, + "accessibility_layout_table_to_blocks" => self.accessibility_layout_table_to_blocks(&a)?, "accessibility_move_tag" => self.accessibility_move_tag(&a)?, "sign_id_create" => self.sign_id_create(&a)?, "sign_document" => self.sign_document(&a)?, @@ -869,6 +917,7 @@ impl Automation { "sign_trust" => self.sign_trust(&a)?, "comment_mark" => self.comment_mark(&a)?, "comment_lock" => self.comment_lock(&a)?, + "comment_popup" => self.comment_popup(&a)?, "comments_hide" => self.comments_hide(&a)?, "layer_set" => self.layer_set(&a)?, "view_reading_colors" => self.view_reading_colors(&a)?, diff --git a/crates/automation/src/documents.rs b/crates/automation/src/documents.rs index 95f94f3b88..e1439aa89f 100644 --- a/crates/automation/src/documents.rs +++ b/crates/automation/src/documents.rs @@ -1,7 +1,7 @@ use super::error::{Result, ToolError, failed}; use super::json::{Args, nest_rows, security, summary}; use super::paths::{child, read_input, write_atomic}; -use super::{Arc, Automation, BlockKey, DocId, Document, Edit, Path, PathBuf, TextBlock, Value, comments, file_access, json}; +use super::{Arc, Automation, BlockKey, DocId, Document, Edit, Path, TextBlock, Value, comments, json}; /// How far a watermark may be moved from the page centre, in points (a page is at most 14,400). const WATERMARK_MAX_OFFSET: f64 = 20_000.0; @@ -135,18 +135,18 @@ impl Automation { pub(crate) fn doc_save_with(&mut self, a: &Args, write: impl FnOnce(&Path, &[u8]) -> Result<()>) -> Result { let doc = self.doc(a)?; let id = doc.id; - let target = match a.opt_str("path")? { + let explicit = a.opt_str("path")?; + let target = match explicit { Some(p) => self.resolve(p, true)?, None => { // Saving in place writes where the file was opened from, which the read root - // allows and the write root may not. + // allows and the write root may not. It also goes through --root again: a + // folder on the remembered path may have become a link out since (#1791). let own = doc.path.clone().ok_or_else(|| failed("the document has never been saved: pass a path"))?; - let own_path = PathBuf::from(&own); - self.check_roots(&own, &own_path, file_access::Access::Write)?; - own_path + self.resolve(&own, true)? } }; - let same_file = doc.path.as_deref().is_some_and(|p| Path::new(p) == target); + let same_file = explicit.is_none() || doc.path.as_deref().is_some_and(|p| Path::new(p) == target); // Saving to a new file is a full rewrite unless asked otherwise, like Save As. let full = a.opt_bool("full")?.unwrap_or(!same_file); let flatten_fill_sign = a.opt_bool("flatten_fill_sign")?.unwrap_or(false); @@ -360,12 +360,15 @@ impl Automation { ("discard_thumbnails", &mut settings.discard_thumbnails), ("discard_alternate_images", &mut settings.discard_alternate_images), ("discard_tags", &mut settings.discard_tags), + ("discard_search_index", &mut settings.discard_search_index), ("discard_print_settings", &mut settings.discard_print_settings), + ("discard_form_actions", &mut settings.discard_form_actions), ("flate_unencoded", &mut settings.flate_unencoded), ("object_streams", &mut settings.object_streams), ("lzw_to_flate", &mut settings.lzw_to_flate), ("remove_invalid_links", &mut settings.remove_invalid_links), ("remove_unreferenced_dests", &mut settings.remove_unreferenced_dests), + ("discard_external_refs", &mut settings.discard_external_refs), ] { if let Some(b) = a.opt_bool(key)? { *flag = b; @@ -470,12 +473,17 @@ impl Automation { "image_bytes_after": o.image_bytes_after, "thumbnails": o.thumbnails, "alternate_images": o.alternate_images, + "search_indexes": o.search_indexes, "tags_removed": o.tags_removed, + "print_settings": o.print_settings, + "form_actions": o.form_actions, "streams_compressed": o.streams_compressed, "lzw_streams": o.lzw_streams, "invalid_links": o.invalid_links, "invalid_bookmarks": o.invalid_bookmarks, "unreferenced_dests": o.unreferenced_dests, + "external_links": o.external_links, + "external_bookmarks": o.external_bookmarks, "unused_xobjects": o.unused_xobjects, "merged_objects": r.merged, "discarded": r.discarded.iter().map(|(h, n)| json!({ "category": h.id(), "count": n })).collect::>(), @@ -623,11 +631,33 @@ impl Automation { replace, file: self.mark_file(a)?, }, + "doc_printer_marks" => { + use pdfcraft_engine::PrinterMarks; + let flag = |k: &str, default: bool| -> Result { Ok(a.opt_bool(k)?.unwrap_or(default)) }; + let (y, m, d) = self.session.today(); + let settings = PrinterMarks { + trim: flag("trim", true)?, + bleed: flag("bleed", true)?, + registration: flag("registration", true)?, + color_bars: flag("color_bars", true)?, + page_info: flag("page_info", true)?, + line_weight: a.opt_num("line_weight")?.unwrap_or(0.25), + make_room: match a.opt_num("make_room")? { + Some(r) if r < 0.0 => None, + Some(r) => Some(r), + None => Some(36.0), + }, + label: a.opt_str("label")?.map(str::to_string).unwrap_or_else(|| self.doc(a).map(|d| d.name.clone()).unwrap_or_default()), + date: a.opt_str("date")?.map(str::to_string).unwrap_or_else(|| format!("{y:04}-{m:02}-{d:02}")), + }; + Edit::AddPrinterMarks { pages, settings, replace: a.opt_bool("replace")?.unwrap_or(true) } + } _ => { let kind = match a.str("kind")? { "header_footer" => MarkKind::HeaderFooter, "watermark" => MarkKind::Watermark, "background" => MarkKind::Background, + "printer_marks" => MarkKind::PrinterMarks, other => return Err(ToolError::InvalidArgs(format!("unknown kind {other:?}"))), }; Edit::RemoveMarks { kind } diff --git a/crates/automation/src/file_access.rs b/crates/automation/src/file_access.rs index dab2760f5d..9ffdea083a 100644 --- a/crates/automation/src/file_access.rs +++ b/crates/automation/src/file_access.rs @@ -5,22 +5,24 @@ //! holds the two folders. Every path a tool reads is checked with [`Access::Read`] and every path //! it writes with [`Access::Write`], in `Automation::resolve`, which all of them go through: //! -//! - The path is made absolute against the working directory, as the file system reads it. Its -//! links are followed: for a file that doesn't exist yet, the links of its nearest existing -//! folder, the rest of the path being plain names (`..` below a missing folder is refused, and so -//! is a link that leads nowhere). +//! - The path is made absolute against the working directory, then followed the way the file +//! system reads it, one name at a time from the top, its links followed. A name is looked up +//! only while the folder it is in is the root, inside it, or one of the root's own parent +//! folders: a path that leaves those is refused before anything beyond is touched (looking up +//! `/net/` mounts it), and every such refusal reads the same, so it can't tell what exists +//! out there. Inside the root, a name that doesn't exist yet ends the walk and the rest is taken +//! as plain names (`..` below a missing folder is refused, and so is a link that leads nowhere). //! - A path on another network share or in a device namespace (`\\host\share`, `\\.\pipe\x`) //! than the root's is refused by name, before anything is resolved: resolving it would contact //! the host. -//! - The result must lie inside the root, compared by whole components (`/work2` isn't inside -//! `/work`), without regard to case on Windows. +//! - The result must lie inside the root, compared by whole canonical components (`/work2` isn't +//! inside `/work`), with the drive letter compared without regard to case on Windows. //! - Read and write access are separate: a root given alone grants only its own kind of access. //! //! Without roots nothing is checked. A path is checked, then opened: a process that can change the //! folders in between (swapping in a link) can still win that race. The roots keep an agent to the //! files it was given; they don't guard against other programs on the computer. -use std::ffi::OsString; use std::path::{Component, Path, PathBuf}; use std::sync::Arc; @@ -100,11 +102,10 @@ impl AutomationRoots { Access::Write => Err(format!("automation filesystem access is not granted: write authority is absent: {shown}")), }; }; - let absolute = absolute(shown, target)?; - // Another network share or a device is refused by name, as `--root` refuses it: even - // resolving the path would contact the host (#4003). - if !foreign_share(&absolute, folder) && inside(&resolve(shown, &absolute)?, folder) { - return Ok(()); + match follow(shown, &absolute(shown, target)?, folder) { + Toward::Inside => return Ok(()), + Toward::Refused(why) => return Err(why), + Toward::Outside => {} } let root = root_shown.display(); match access { @@ -141,60 +142,85 @@ fn absolute(shown: &str, path: &Path) -> Result { std::path::absolute(path).map_err(|e| format!("automation path rejected: cannot resolve {shown}: {e}")) } -/// `absolute` with its links followed; for a path that doesn't exist (yet), its nearest existing -/// folder's links followed and the rest appended. -fn resolve(shown: &str, absolute: &Path) -> Result { - // The names below the nearest existing folder, last first. - let mut missing: Vec = vec![]; - let mut at = absolute; - loop { - match std::fs::canonicalize(at) { - Ok(mut found) => { - found.extend(missing.iter().rev()); - return Ok(found); +/// Where a followed path ends up. +pub(crate) enum Toward { + /// Inside the root, with its links followed (a missing tail taken as names). + Inside, + /// Anywhere else. Said the same way whatever is out there. + Outside, + /// Inside the root, but it can't be followed, and why. + Refused(String), +} + +/// Follow the absolute `path` the way the file system reads it, one name at a time from the top, +/// looking a name up only while the folder it is in is `root`, inside it, or one of `root`'s own +/// parent folders (all with their links followed already, so `..` means what it means to the file +/// system). +pub(crate) fn follow(shown: &str, path: &Path, root: &Path) -> Toward { + // Another network share or a device is refused by name, as `--root` refuses it: even looking + // it up would contact the host (#4003). + if foreign_share(path, root) { + return Toward::Outside; + } + let near = |at: &Path| inside(at, root) || inside(root, at); + let mut parts = path.components().peekable(); + let top: PathBuf = std::iter::from_fn(|| parts.next_if(|c| matches!(c, Component::Prefix(_) | Component::RootDir))).collect(); + let mut at = match std::fs::canonicalize(&top) { + Ok(at) if near(&at) => at, + _ => return Toward::Outside, + }; + while let Some(part) = parts.next() { + let name = match part { + Component::Normal(name) => name, + Component::CurDir => continue, + Component::ParentDir => { + at.pop(); + continue; } - Err(e) => match std::fs::symlink_metadata(at) { - Ok(m) if m.file_type().is_symlink() => return Err(format!("automation path rejected: a link that leads nowhere: {shown}")), - // There, yet its real path can't be had (no permission): nothing to compare. - Ok(_) => return Err(format!("automation path rejected: cannot resolve {shown}: {e}")), - Err(e) if e.kind() == std::io::ErrorKind::NotFound => {} - Err(e) => return Err(format!("automation path rejected: cannot resolve {shown}: {e}")), - }, - } - let parent = at.parent(); - match (at.components().next_back(), parent) { - (Some(Component::Normal(name)), Some(parent)) => { - missing.push(name.to_os_string()); - at = parent; - } - (Some(Component::ParentDir), _) => { - return Err(format!("automation path rejected: parent traversal below a folder that doesn't exist: {shown}")); - } - // Down to the root (or drive) and it doesn't exist either. - _ => { - let e = std::io::Error::from(std::io::ErrorKind::NotFound); - return Err(format!("automation path rejected: cannot resolve {shown}: {e}")); + Component::Prefix(_) | Component::RootDir => return Toward::Outside, + }; + let next = at.join(name); + match std::fs::canonicalize(&next) { + Ok(found) if near(&found) => at = found, + Ok(_) => return Toward::Outside, + // Not even there yet: only inside the root does it matter why. + Err(_) if !inside(&at, root) => return Toward::Outside, + Err(e) => { + return match std::fs::symlink_metadata(&next) { + Ok(m) if m.file_type().is_symlink() => Toward::Refused(format!("automation path rejected: a link that leads nowhere: {shown}")), + // There, yet its real path can't be had (no permission): nothing to compare. + Ok(_) => Toward::Refused(format!("automation path rejected: cannot resolve {shown}: {e}")), + Err(e) if e.kind() == std::io::ErrorKind::NotFound => { + if parts.any(|part| !matches!(part, Component::Normal(_) | Component::CurDir)) { + return Toward::Refused(format!("automation path rejected: parent traversal below a folder that doesn't exist: {shown}")); + } + Toward::Inside + } + Err(e) => Toward::Refused(format!("automation path rejected: cannot resolve {shown}: {e}")), + }; } } } + if inside(&at, root) { Toward::Inside } else { Toward::Outside } } -/// Is `path` (resolved) `root` or inside it, component by component (without regard to case on -/// Windows, whose file systems ignore it)? +/// Is `path` (resolved) `root` or inside it, component by component (the drive letter without +/// regard to case on Windows)? fn inside(path: &Path, root: &Path) -> bool { let mut path = path.components(); root.components().all(|r| path.next().is_some_and(|p| same_component(p, r))) } fn same_component(a: Component, b: Component) -> bool { + use std::path::Prefix; if a == b { return true; } - if !cfg!(windows) { - return false; - } - match (a.as_os_str().to_str(), b.as_os_str().to_str()) { - (Some(a), Some(b)) => a.to_lowercase() == b.to_lowercase(), + match (a, b) { + (Component::Prefix(a), Component::Prefix(b)) => match (a.kind(), b.kind()) { + (Prefix::Disk(a) | Prefix::VerbatimDisk(a), Prefix::Disk(b) | Prefix::VerbatimDisk(b)) => a.eq_ignore_ascii_case(&b), + _ => false, + }, _ => false, } } @@ -306,6 +332,62 @@ mod tests { let _ = std::fs::remove_dir_all(base); } + /// Outside the root, the refusal comes before anything out there is looked up, so it can't + /// tell whether a folder out there exists. + #[test] + fn paths_out_of_the_root_are_refused_alike_without_looking() { + let (base, roots) = folders("follow"); + let sep = std::path::MAIN_SEPARATOR; + let root = s(base.join("in")); + let refusal = |p: &str| roots.check(p, Access::Read).unwrap_err().replace(p, ""); + // Through a folder outside that exists, and through one that doesn't. + let there = format!("{root}{sep}..{sep}outside{sep}..{sep}x.pdf"); + let missing = format!("{root}{sep}..{sep}nowhere{sep}..{sep}x.pdf"); + assert_eq!(refusal(&there), refusal(&missing)); + assert!(refusal(&missing).starts_with("automation path rejected: outside the read root"), "{}", refusal(&missing)); + // Out through a missing folder and back in. (Windows folds `..` by name when it makes a path + // absolute, as it does when it opens one: that path is `in\a.pdf` there.) + #[cfg(not(windows))] + assert!(refusal(&format!("{root}/../nowhere/../in/a.pdf")).contains("outside the read root")); + // A `\\?\` path is taken as written, `..` and all, so there the walk itself refuses it. + #[cfg(windows)] + { + let verbatim = |rest: &str| format!(r"\\?\{root}\{rest}"); + assert_eq!(refusal(&verbatim(r"..\outside\..\x.pdf")), refusal(&verbatim(r"..\nowhere\..\x.pdf"))); + assert!( + refusal(&verbatim(r"..\nowhere\x.pdf")).starts_with("automation path rejected: outside the read root"), + "{}", + refusal(&verbatim(r"..\nowhere\x.pdf")) + ); + } + let _ = std::fs::remove_dir_all(base); + } + + /// The root can be named as it was given or as it resolves (`/var/…` and `/private/var/…` on + /// macOS, where the temporary folder is reached through a link). + #[test] + fn a_root_may_be_named_as_given_or_as_resolved() { + let (base, roots) = folders("given"); + let resolved = std::fs::canonicalize(base.join("in")).unwrap(); + assert_eq!(roots.check(&s(base.join("in/a.pdf")), Access::Read), Ok(())); + assert_eq!(roots.check(&s(resolved.join("a.pdf")), Access::Read), Ok(())); + let _ = std::fs::remove_dir_all(base); + } + + /// A link beside the root, in one of its parent folders, is followed: to the root it leads in, + /// anywhere else it leads out. + #[cfg(unix)] + #[test] + fn links_beside_the_root_are_followed() { + let (base, roots) = folders("beside"); + std::os::unix::fs::symlink(base.join("in"), base.join("to-in")).unwrap(); + std::os::unix::fs::symlink(base.join("outside"), base.join("to-outside")).unwrap(); + assert_eq!(roots.check(&s(base.join("to-in/a.pdf")), Access::Read), Ok(())); + assert_eq!(roots.check(&s(base.join("to-in/sub/new.pdf")), Access::Read), Ok(())); + assert!(roots.check(&s(base.join("to-outside/secret.pdf")), Access::Read).unwrap_err().contains("outside the read root")); + let _ = std::fs::remove_dir_all(base); + } + #[test] fn relative_paths_resolve_against_the_working_directory() { let (base, roots) = folders("relative"); @@ -385,6 +467,37 @@ mod tests { let _ = std::fs::remove_dir_all(base); } + /// #4003: canonical path components are compared as the file system gave them, with only the + /// Windows drive letter matched ASCII case-insensitively, so a sibling whose name `to_lowercase` + /// folds onto the root's (`K` U+212A → `k`) or a differently-cased canonical component is not + /// taken as inside the root. + #[test] + fn canonical_components_are_compared_exactly() { + let (base, _) = folders("canonical-exact"); + std::fs::create_dir_all(base.join("ink")).unwrap(); + let kelvin = base.join("in\u{212A}"); + std::fs::create_dir_all(&kelvin).unwrap(); + std::fs::write(kelvin.join("secret.pdf"), "x").unwrap(); + // U+212A folds to K. A case-insensitive volume stores both names as one directory, so + // `check` would be asked to reject a path inside its own root. The component comparison + // itself does not need that volume. + let distinct = base.join("ink").canonicalize().ok() != kelvin.canonicalize().ok(); + if distinct { + let roots = AutomationRoots::new(Some(&base.join("ink")), None).unwrap().unwrap(); + let e = roots.check(&s(kelvin.join("secret.pdf")), Access::Read).unwrap_err(); + assert!(e.starts_with("automation path rejected: outside the read root"), "{e}"); + } + assert!(!inside(Path::new("/work/in\u{212A}/secret.pdf"), Path::new("/work/ink"))); + assert!(!inside(Path::new(r"\\?\C:\work\IN\a.pdf"), Path::new(r"\\?\C:\work\in"))); + #[cfg(windows)] + { + assert!(inside(Path::new(r"\\?\c:\work\in\a.pdf"), Path::new(r"\\?\C:\work\in"))); + assert!(inside(Path::new(r"c:\work\in\a.pdf"), Path::new(r"\\?\C:\work\in"))); + assert!(!inside(Path::new(r"\\?\D:\work\in\a.pdf"), Path::new(r"\\?\C:\work\in"))); + } + let _ = std::fs::remove_dir_all(base); + } + /// Every file a tool reads or writes goes through `Automation::resolve`, which holds it to the /// roots. A function that reaches for the file system without resolving a path first would /// bypass them, so each use of `std::fs` or `File::open` must come after a `resolve` call in @@ -393,7 +506,8 @@ mod tests { fn tools_do_not_touch_the_file_system_without_resolving_the_path() { // Helpers that are only called with a resolved path. `doc_batch` (batch.rs) resolves its // inputs and out_dir, walks only inside them (skipping links) and resolves every - // discovered file again before `process` reads it or `publish` writes it. + // discovered file again before `process` reads it or `publish` writes it. `doc_bates_multi` + // resolves its paths before checking output collisions and using the probe helpers. const RESOLVED_BY_CALLER: &[&str] = &[ "write_atomic", "write_atomic_with", @@ -405,6 +519,8 @@ mod tests { "linked", "walk", "publish", + "check_unique_outputs", + "probe_files_for_resolved_outputs", ]; let mut dirs = vec![Path::new(env!("CARGO_MANIFEST_DIR")).join("src")]; let mut files = vec![]; diff --git a/crates/automation/src/forms.rs b/crates/automation/src/forms.rs index adf0bd0884..2fd51038de 100644 --- a/crates/automation/src/forms.rs +++ b/crates/automation/src/forms.rs @@ -208,13 +208,58 @@ fn kind_name(k: FormFieldKind) -> &'static str { } } +/// A JSON number as the radio option it names: its integer spelling when it is a whole number +/// (`1`, `1.0` and `1e0` are all `1`; `-0` is `0`), otherwise JSON's own spelling (`1.5`), which +/// names only an option exported as exactly that text. A whole number at or past 2^53, where a +/// float no longer holds every integer, keeps JSON's spelling too (`1e300`), so it names no option +/// by accident. +fn number_option(n: &serde_json::Number) -> String { + // Integers as parsed (u64, i64) print exactly. + if n.is_u64() || n.is_i64() { + return n.to_string(); + } + const EXACT: f64 = 9_007_199_254_740_992.0; // 2^53 + match n.as_f64() { + // Finite, whole and below 2^53 in magnitude, so the cast is exact and in range. + Some(x) if x.is_finite() && x.fract() == 0.0 && x.abs() < EXACT => (x as i64).to_string(), + _ => n.to_string(), + } +} + /// A JSON value → the value for this field (booleans for check boxes, arrays for lists…). fn value_for(f: &FormField, v: &Value) -> Result { let wrong = |what: &str| ToolError::InvalidArgs(format!("{}: {what}", f.name)); Ok(match (f.kind, v) { (FormFieldKind::CheckBox, Value::Bool(b)) => FieldValue::Check(*b), + // {"widget": n} turns on button n (1-based, as form_fields numbers them) and never turns it + // off: radio buttons that share an on state stay exclusive (ISO 32000-2 Table 229). + (FormFieldKind::CheckBox | FormFieldKind::Radio, Value::Object(o)) => { + let n = o.get("widget").and_then(Value::as_u64).filter(|&n| n >= 1).ok_or_else(|| wrong("use {\"widget\": n} with n from 1"))?; + let i = usize::try_from(n - 1).map_err(|_| wrong("there is no such widget"))?; + if f.widgets.get(i).and_then(|w| w.on_state.as_ref()).is_none() { + return Err(wrong(&format!("there is no button {n} with an on state (the field has {})", f.widgets.len()))); + } + FieldValue::Widget(i) + } (FormFieldKind::Radio, Value::Null) => FieldValue::Radio(None), - (FormFieldKind::Radio, Value::String(s)) => FieldValue::Radio(Some(s.clone())), + // An option as `form_fields` lists it (an export value) names its button's on state. The + // export values are matched first and the on-state names after them + // (`Field::state_for_export`): with `/Opt [(1) (2)]` and states `/0` `/1`, `1` is the first + // button, not the second one's state. A value matching neither is refused when written, + // with the options listed. + (FormFieldKind::Radio, Value::String(s)) => FieldValue::Radio(Some(f.state_for_export(s).unwrap_or(s).to_owned())), + // A number is the same option written without quotes (`1` for the export value `1`). + // Below 2^53 it is compared as a number with the export values first, so `1.0` names the button + // exported as `1.0` (`/Opt [(1.0) (2.0)]`, states `/0 /1`) and not the one whose state is + // `/1`; then as text, as a string would be. + (FormFieldKind::Radio, Value::Number(n)) => { + if let Some(state) = n.as_f64().and_then(|x| f.state_for_number(x)) { + FieldValue::Radio(Some(state.to_owned())) + } else { + let s = number_option(n); + FieldValue::Radio(Some(f.state_for_export(&s).map_or_else(|| s.clone(), str::to_owned))) + } + } (FormFieldKind::Combo | FormFieldKind::List, Value::Array(a)) => FieldValue::Choice( a.iter().map(|x| x.as_str().map(str::to_owned).ok_or_else(|| wrong("list values must be strings"))).collect::>()?, ), @@ -294,9 +339,41 @@ impl Automation { } FormFieldKind::Radio => { o.insert("options".into(), json!((0..f.widgets.len()).filter_map(|i| f.export_of(i)).collect::>())); + // Buttons sharing an on state have the same value: which is on (1-based). + o.insert( + "on_widgets".into(), + json!((0..f.widgets.len()).filter(|&i| f.widget_on(i)).map(|i| i.saturating_add(1)).collect::>()), + ); + } + // A check box whose kids have different on states (ISO 32000-2 §12.7.5.2.3): + // which one is ticked, and the names form_fill takes to tick each. One option + // per distinct on state: kids sharing a state tick together (EXAMPLE 2), so + // their `/Opt` exports are not separate choices. Exports name the options + // unless two states share one (Table 230 allows it) or an export would tick + // another state (form_fill reads a name as an on state first); then the + // states do. + FormFieldKind::CheckBox => { + let mut seen = std::collections::HashSet::new(); + let states: Vec<&str> = f.widgets.iter().filter_map(|w| w.on_state.as_deref()).filter(|s| seen.insert(*s)).collect(); + if states.len() > 1 { + let mut exports = std::collections::HashSet::new(); + let by_export = states.iter().all(|s| { + let e = f.export_for_state(s); + exports.insert(e) && f.state_for(e) == Some(*s) + }); + let name = |s: &str| if by_export { f.export_for_state(s).to_owned() } else { s.to_owned() }; + // The ticked kid is the one whose `/AS` is its on state: §12.7.5.2.3 + // makes `/AS` decide where it and `/V` differ. + let on = (0..f.widgets.len()).find(|&i| f.widget_on(i)).and_then(|i| f.widgets.get(i)?.on_state.as_deref()); + o.insert("options".into(), json!(states.iter().map(|s| name(s)).collect::>())); + o.insert("checked".into(), json!(on.map(name))); + } } FormFieldKind::Combo | FormFieldKind::List => { o.insert("options".into(), f.options.iter().map(|(e, d)| json!({ "value": e, "label": d })).collect()); + // The selected options as 1-based positions in `options`: two options can share + // an export value, and `value` alone does not tell them apart. + o.insert("selected".into(), json!(f.selected.iter().map(|i| i.saturating_add(1)).collect::>())); o.insert("multi_select".into(), json!(f.has(field_flags::MULTI_SELECT))); o.insert("editable".into(), json!(f.has(field_flags::EDIT))); } diff --git a/crates/automation/src/hairlines.rs b/crates/automation/src/hairlines.rs new file mode 100644 index 0000000000..60e9dfe475 --- /dev/null +++ b/crates/automation/src/hairlines.rs @@ -0,0 +1,40 @@ +//! Print production ▸ Fix hairlines as a tool: strokes narrower than a threshold on the page get a +//! width (`doc_fix_hairlines`). + +use pdfcraft_engine::{Edit, Hairlines}; +use serde_json::{Value, json}; + +use crate::{Args, Automation, Result, ToolError}; + +impl Automation { + /// Strokes thinner than `narrower_than` points on the page (after every transformation; + /// zero-width lines always) are given `width` points; Type 3 glyphs and tiling patterns on + /// request. Nothing to fix is not an error: `fixed` is 0 and no edit is made. `elsewhere` + /// counts the glyphs and patterns changed that something outside `pages` also uses. + pub(crate) fn fix_hairlines(&mut self, a: &Args) -> Result { + let n = self.doc(a)?.info.pages.len(); + let pages = match a.opt_ints("pages")? { + Some(_) => self.pages(a, "pages")?, + None => (0..n).collect(), + }; + let settings = Hairlines { + narrower_than: a.opt_num("narrower_than")?.unwrap_or(0.003 * 72.0), + width: a.opt_num("width")?.unwrap_or(0.01 * 72.0), + type3: a.opt_bool("type3")?.unwrap_or(false), + patterns: a.opt_bool("patterns")?.unwrap_or(false), + }; + if !settings.in_range() { + return Err(ToolError::InvalidArgs(format!( + "narrower_than must be 0 or more and width above 0, both at most {} (points)", + Hairlines::MAX_POINTS + ))); + } + let id = self.doc(a)?.id; + let found = self.session.count_hairlines(id, &pages, &settings); + if found.lines == 0 { + return Ok(json!({ "fixed": 0, "pages": 0, "elsewhere": 0 })); + } + let r = self.apply(a, Edit::FixHairlines { pages, settings })?; + Ok(json!({ "fixed": found.lines, "pages": found.pages, "elsewhere": found.elsewhere, "undo": r["undo"] })) + } +} diff --git a/crates/automation/src/json.rs b/crates/automation/src/json.rs index c950e72f05..e6ca95e905 100644 --- a/crates/automation/src/json.rs +++ b/crates/automation/src/json.rs @@ -279,6 +279,8 @@ pub(crate) fn summary(d: &Document) -> Value { }) } +/// What `doc_info` returns for an open document: its metadata, security, pages, bookmarks, +/// comments, fields, links, layers, transitions, attachments, fonts, warnings and repairs. pub(crate) fn info(d: &Document) -> Value { let i = &d.info; let page1 = |p: usize| p + 1; @@ -313,7 +315,7 @@ pub(crate) fn info(d: &Document) -> Value { "pages": i.pages.iter().enumerate().map(|(n, p)| json!({ "page": n + 1, "label": p.label, "width": p.width, "height": p.height, "rotation": p.rotation, })).collect::>(), - "outline": outline(&i.outline), + "outline": outline(i, &i.outline), "outline_more": i.outline_more, // Rectangles use the tools' convention (top-left of the displayed page, like // comment_list and link_list), not raw PDF user space, so they can be fed back to @@ -329,34 +331,15 @@ pub(crate) fn info(d: &Document) -> Value { })).collect::>(), "links": i.links.iter().map(|l| json!({ "page": page1(l.page), "rect": view_rect(i, l.page, l.rect), - "target": match &l.target { - pdfcraft_render::LinkTarget::Page(p, _) => json!({ "page": page1(*p) }), - pdfcraft_render::LinkTarget::Uri(u) => json!({ "uri": u }), - pdfcraft_render::LinkTarget::SetLayers { changes, preserve_rb } => json!({ - "layers": changes.iter().map(|(op, ocg)| json!({ - "layer": i.layers.iter().find(|l| l.id == *ocg).map(|l| l.name.as_str()), - "state": match op { - pdfcraft_render::LayerOp::On => "on", - pdfcraft_render::LayerOp::Off => "off", - pdfcraft_render::LayerOp::Toggle => "toggle", - }, - })).collect::>(), - "preserve_rb": preserve_rb, - }), - pdfcraft_render::LinkTarget::RemoteGoTo { launch, file, destination, new_window } => json!({ - "action": if *launch { "Launch" } else { "GoToR" }, - "remote_document": file, - "destination": match destination { - pdfcraft_render::RemoteDestination::Default => json!({ "kind": "default" }), - pdfcraft_render::RemoteDestination::Page { page, view } => json!({ "kind": "page", "page": page1(*page), "view": format!("{view:?}") }), - pdfcraft_render::RemoteDestination::Named(name) => json!({ "kind": "named", "name": name }), - }, - "new_window": new_window, - }), - pdfcraft_render::LinkTarget::Other(o) => json!({ "other": o }), - }, + "target": link_target(i, &l.target), })).collect::>(), "layers": i.layers.iter().map(|l| json!({ "name": l.name, "visible": l.visible, "initial": l.initial })).collect::>(), + // The Layers panel's rows, the default configuration's /Order (ISO 32000-2 Table 99): a + // layer by its 1-based position in `layers`, or a label; `depth` 0 at the top. + "layer_order": i.layer_order.iter().map(|r| match &r.item { + pdfcraft_render::LayerItem::Layer(n) => json!({ "depth": r.depth, "layer": n.saturating_add(1), "name": i.layers.get(*n).map(|l| l.name.as_str()) }), + pdfcraft_render::LayerItem::Label(s) => json!({ "depth": r.depth, "label": s }), + }).collect::>(), // Full-screen presentation settings a page carries (ISO 32000-2 §12.4.4): only pages with // a `/Trans` or `/Dur` appear; `style` is null for a display time alone. "transitions": i.transitions.iter().map(|t| json!({ @@ -364,7 +347,12 @@ pub(crate) fn info(d: &Document) -> Value { "horizontal": t.horizontal, "inward": t.inward, "direction": t.direction, "scale": t.scale, "opaque": t.opaque, "display": t.display, })).collect::>(), - "attachments": i.attachments.iter().map(|a| json!({ "name": a.name, "description": a.description, "size": a.size })).collect::>(), + "attachments": i.attachments.iter().map(|a| json!({ "name": a.name, "key": match &a.source { pdfcraft_render::AttachmentSource::Document { key } => Some(key), pdfcraft_render::AttachmentSource::Annotation { .. } => None }, "description": a.description, "size": a.size })).collect::>(), + // A PDF portfolio (`/Collection`, ISO 32000-2 §7.11.6): its pages are a cover sheet and its + // documents are `attachments`; `initial` names the one presented first, null for the cover. + "portfolio": i.portfolio.as_ref().map(|p| json!({ + "initial": p.initial.and_then(|n| i.attachments.get(n)).map(|a| &a.name), "view": p.view.name(), + })), // `substitute` names the face standing in for a program the document does not embed, and // `substitution` says why. Both are null for an embedded font: nothing stands in. A null // `substitute` with a non-null `substitution` means no face was available at all, so @@ -389,6 +377,50 @@ pub fn view_rect(i: &pdfcraft_render::DocInfo, page: usize, rect: [f32; 4]) -> [ i.pages.get(page).map_or(rect, |p| comments::rect_to_view(p, rect)) } -fn outline(items: &[pdfcraft_render::OutlineItem]) -> Value { - Value::Array(items.iter().map(|o| json!({ "title": o.title, "page": o.page.map(|p| p + 1), "children": outline(&o.children) })).collect()) +/// Bookmarks with the page each goes to, or the action it runs instead (as a link's `target`). +fn outline(i: &pdfcraft_render::DocInfo, items: &[pdfcraft_render::OutlineItem]) -> Value { + Value::Array( + items + .iter() + .map(|o| { + json!({ + "title": o.title, "page": o.page.map(|p| p + 1), + "action": o.action.as_ref().map(|a| link_target(i, a)), "children": outline(i, &o.children), + }) + }) + .collect(), + ) +} + +/// Where a link goes, or what a bookmark's action does: a 1-based page, a web address, layer +/// changes, another PDF, or the name of an action that isn't run. +fn link_target(i: &pdfcraft_render::DocInfo, target: &pdfcraft_render::LinkTarget) -> Value { + // A remote page number is the file's, however large. + let page1 = |p: usize| p.saturating_add(1); + match target { + pdfcraft_render::LinkTarget::Page(p, _) => json!({ "page": page1(*p) }), + pdfcraft_render::LinkTarget::Uri(u) => json!({ "uri": u }), + pdfcraft_render::LinkTarget::SetLayers { changes, preserve_rb } => json!({ + "layers": changes.iter().map(|(op, ocg)| json!({ + "layer": i.layers.iter().find(|l| l.id == *ocg).map(|l| l.name.as_str()), + "state": match op { + pdfcraft_render::LayerOp::On => "on", + pdfcraft_render::LayerOp::Off => "off", + pdfcraft_render::LayerOp::Toggle => "toggle", + }, + })).collect::>(), + "preserve_rb": preserve_rb, + }), + pdfcraft_render::LinkTarget::RemoteGoTo { launch, file, destination, new_window } => json!({ + "action": if *launch { "Launch" } else { "GoToR" }, + "remote_document": file, + "destination": match destination { + pdfcraft_render::RemoteDestination::Default => json!({ "kind": "default" }), + pdfcraft_render::RemoteDestination::Page { page, view } => json!({ "kind": "page", "page": page1(*page), "view": format!("{view:?}") }), + pdfcraft_render::RemoteDestination::Named(name) => json!({ "kind": "named", "name": name }), + }, + "new_window": new_window, + }), + pdfcraft_render::LinkTarget::Other(o) => json!({ "other": o }), + } } diff --git a/crates/automation/src/lib.rs b/crates/automation/src/lib.rs index 7e4443d244..7fa63d1a1d 100644 --- a/crates/automation/src/lib.rs +++ b/crates/automation/src/lib.rs @@ -22,6 +22,7 @@ mod content; mod conventions; pub mod file_access; mod forms; +mod hairlines; mod layers; mod links; #[cfg(feature = "mcp")] @@ -48,7 +49,6 @@ use serde_json::{Value, json}; pub use pdfcraft_render::Pixels; pub use tools::{ToolDef, tools}; -use file_access::foreign_share; use text_cache::TextCache; mod assemble; @@ -71,7 +71,7 @@ use pages::{DEFAULT_DPI, MAX_DPI, encode_png, rendered_content}; use pages::{FIND_FIRST_BATCH, FIND_MAX_WARNING_BYTES, FIND_MAX_WARNINGS, extract_parallel}; #[cfg(test)] pub(crate) use paths::MAX_INPUT; -use paths::{child, lexical, unused, write_atomic}; +use paths::{child, lexical, write_atomic}; pub(crate) use paths::{read_input, read_input_bytes}; #[cfg(test)] use paths::{write_atomic_synced, write_atomic_with}; @@ -101,6 +101,8 @@ pub struct Automation { /// Review sessions held for `ocr_words` (a guard and the pages read), by document. ocr_reviews: HashMap, next_ocr_review: u64, + /// What object_copy copied, for object_paste into any open document. + objects_clipboard: Option, } impl Default for Automation { @@ -121,6 +123,7 @@ impl Automation { evicted: HashSet::new(), ocr_reviews: HashMap::new(), next_ocr_review: 0, + objects_clipboard: None, } } diff --git a/crates/automation/src/links.rs b/crates/automation/src/links.rs index 41fda9b7db..a6228cc5bc 100644 --- a/crates/automation/src/links.rs +++ b/crates/automation/src/links.rs @@ -2,7 +2,7 @@ //! all links. Rectangles are points from the top-left of the displayed page. use pdfcraft_engine::{ - Edit, LinkAction, LinkHighlight, LinkStyle, read_remote_document, remote_document_path_is_absolute, resolve_remote_document_path, + Edit, LinkAction, LinkHighlight, LinkStyle, LinkZoom, read_remote_document, remote_document_path_is_absolute, resolve_remote_document_path, }; use pdfcraft_render::{DestView, LinkTarget, RemoteDestination}; use serde_json::{Value, json}; @@ -25,18 +25,30 @@ impl Automation { Ok(Some([u0[0].min(u1[0]) as f64, u0[1].min(u1[1]) as f64, u0[0].max(u1[0]) as f64, u0[1].max(u1[1]) as f64])) } - fn link_action(&self, a: &Args) -> Result> { + /// The action `url`, `to_page` and `zoom` ask for, if any. `current` is the link's action now + /// (link_edit): a new page keeps the link's zoom, and a zoom alone keeps its page. + fn link_action(&self, a: &Args, current: Option<&LinkAction>) -> Result> { + let zoom = link_zoom(a)?; + let (current_page, current_zoom) = match current { + Some(LinkAction::Page { page, zoom }) => (Some(*page), Some(*zoom)), + _ => (None, None), + }; match (a.opt_str("url")?, a.opt_int("to_page")?) { (Some(_), Some(_)) => Err(bad("pass url or to_page, not both")), + (Some(_), None) if zoom.is_some() => Err(bad("zoom goes with to_page, not url")), (Some(u), None) => Ok(Some(LinkAction::Uri(u.to_string()))), (None, Some(p)) => { let n = self.doc(a)?.info.pages.len() as i64; if p < 1 || p > n { return Err(bad(format!("to_page must be 1–{n}"))); } - Ok(Some(LinkAction::Page(p as usize - 1))) + Ok(Some(LinkAction::Page { page: p as usize - 1, zoom: zoom.or(current_zoom).unwrap_or(LinkZoom::FitPage) })) } - (None, None) => Ok(None), + (None, None) => match (zoom, current_page) { + (None, _) => Ok(None), + (Some(zoom), Some(page)) => Ok(Some(LinkAction::Page { page, zoom })), + (Some(_), None) => Err(bad("zoom needs to_page: the link doesn't go to a page")), + }, } } @@ -65,7 +77,8 @@ impl Automation { pub(crate) fn link_list(&self, a: &Args) -> Result { let doc = self.doc(a)?; - let targets: std::collections::HashMap<(usize, usize), &LinkTarget> = doc.info.links.iter().map(|t| ((t.page, t.index), &t.target)).collect(); + let viewed: std::collections::HashMap<(usize, usize), &pdfcraft_render::Link> = + doc.info.links.iter().map(|t| ((t.page, t.index), t)).collect(); let items: Vec = doc .links() .iter() @@ -73,30 +86,104 @@ impl Automation { // The link's page may be one the viewer could not read (#2660): rect unchanged then. let rect = crate::view_rect(&doc.info, l.page, l.rect.map(|x| x as f32)); let mut v = json!({ "page": l.page + 1, "index": l.index + 1, "rect": rect, "visible": l.style.visible }); + let link = viewed.get(&(l.page, l.index)); + // Where the link is active when that isn't its whole rect: its /QuadPoints + // (ISO 32000-2 Table 176), as the bounding box of each quadrilateral. + if let Some(link) = link.filter(|k| !k.quads.is_empty()) { + let areas: Vec = link + .quads + .iter() + .map(|q| { + let (xs, ys) = ([q[0], q[2], q[4], q[6]], [q[1], q[3], q[5], q[7]]); + let fold = |v: [f32; 4], f: fn(f32, f32) -> f32, start: f32| v.into_iter().fold(start, f); + let box_ = [ + fold(xs, f32::min, f32::MAX), + fold(ys, f32::min, f32::MAX), + fold(xs, f32::max, f32::MIN), + fold(ys, f32::max, f32::MIN), + ]; + json!(crate::view_rect(&doc.info, l.page, box_)) + }) + .collect(); + v["active_areas"] = json!(areas); + } // Where an in-document link goes, as the viewer resolves it: named destinations // included, with the position on the page. - let target = targets.get(&(l.page, l.index)).and_then(|t| match t { + let target = link.and_then(|t| match &t.target { LinkTarget::Page(page, view) => Some((*page, view)), _ => None, }); match (&l.action, target) { - (LinkAction::Uri(u), _) => v["url"] = json!(u), + (LinkAction::Uri(u), _) => { + v["url"] = json!(u); + // A relative address, resolved against the catalog's /URI /Base + // (ISO 32000-2 Table 211): what the viewer offers to open. That is the + // address inspection kept, which stays raw once its text budget is spent; + // resolve afresh only for a link inspection did not list. + let opened = match link.map(|t| &t.target) { + Some(LinkTarget::Uri(kept)) => Some(kept.clone()), + Some(_) => None, + None => pdfcraft_render::resolve_uri(doc.info.uri_base.as_deref(), u), + }; + if let Some(opened) = opened.filter(|o| o != u) { + v["resolved_url"] = json!(opened); + } + } (action, Some((page, view))) => { v["to_page"] = json!(page + 1); if let Some(d) = doc.info.pages.get(page).and_then(|p| crate::destination(view, p)) { v["destination"] = d; } - if matches!(action, LinkAction::Other(_)) { - v["named"] = json!(true); + match action { + LinkAction::Page { zoom, .. } => v["zoom"] = zoom_value(*zoom), + LinkAction::Other(_) => v["named"] = json!(true), + LinkAction::Uri(_) => {} } } - (LinkAction::Page(p), None) => v["to_page"] = json!(p + 1), + (LinkAction::Page { page, zoom }, None) => { + v["to_page"] = json!(page + 1); + v["zoom"] = zoom_value(*zoom); + } (LinkAction::Other(o), None) => v["other"] = json!(o), } v }) .collect(); - Ok(json!({ "count": items.len(), "links": items })) + let mut out = json!({ "count": items.len(), "links": items }); + // Addresses written as plain page text, which the viewer makes clickable (#4207). Reading + // the pages' text costs more than the list above, so only on request. + if a.opt_bool("text_links")?.unwrap_or(false) { + let id = doc.id; + // One share of the pages at a time (`MAX_TEXT_LINK_PAGES_PER_SCAN`), from the 1-based + // `text_links_from`; `text_links_next_page` says where the next share starts. + let pages = doc.info.pages.len().max(1); + let from = match a.opt_int("text_links_from")? { + None => 0, + Some(p) => usize::try_from(p) + .ok() + .filter(|p| (1..=pages).contains(p)) + .map(|p| p - 1) + .ok_or_else(|| bad(format!("text_links_from must be a page from 1 to {pages}")))?, + }; + let scan = self.session.text_links(id, from); + if let Some(next) = scan.next_page { + out["text_links_next_page"] = json!(next.saturating_add(1)); + } + let found: Vec = scan + .links + .into_iter() + .map(|(page, l)| { + let mut v = json!({ "page": page + 1, "rects": l.rects, "url": l.uri }); + // What the viewer does on a click: the same check every document link gets. + if let Err(e) = pdfcraft_engine::links::document_url(&l.uri) { + v["blocked"] = json!(e.to_string()); + } + v + }) + .collect(); + out["text_links"] = json!(found); + } + Ok(out) } pub(crate) fn link_open(&mut self, a: &Args) -> Result { @@ -168,12 +255,13 @@ impl Automation { pub(crate) fn link_add(&mut self, a: &Args) -> Result { let page = self.page(a)?; let rect = self.view_rect(a, page)?.ok_or_else(|| bad("link_add needs rect"))?; - let action = self.link_action(a)?.ok_or_else(|| bad("pass url or to_page"))?; + let action = self.link_action(a, None)?.ok_or_else(|| bad("pass url or to_page"))?; let style = self.link_style(a, LinkStyle::default())?; self.apply(a, Edit::AddLink { page, rect, action, style }) } - fn link_index(&self, a: &Args) -> Result<(usize, usize, LinkStyle)> { + /// The link `page` and `index` name: its page, index in `/Annots`, appearance and action. + fn link_index(&self, a: &Args) -> Result<(usize, usize, LinkStyle, LinkAction)> { let page = self.page(a)?; let index = a.int("index")?; let doc = self.doc(a)?; @@ -182,19 +270,19 @@ impl Automation { .iter() .find(|l| l.page == page && l.index as i64 + 1 == index) .ok_or_else(|| failed(format!("page {} has no link {index} (see link_list)", page + 1)))?; - Ok((page, l.index, l.style)) + Ok((page, l.index, l.style, l.action.clone())) } pub(crate) fn link_edit(&mut self, a: &Args) -> Result { - let (page, index, style) = self.link_index(a)?; + let (page, index, style, current) = self.link_index(a)?; let rect = self.view_rect(a, page)?; - let action = self.link_action(a)?; + let action = self.link_action(a, Some(¤t))?; let new_style = self.link_style(a, style)?; self.apply(a, Edit::SetLink { page, index, rect, action, style: (new_style != style).then_some(new_style) }) } pub(crate) fn link_delete(&mut self, a: &Args) -> Result { - let (page, index, _) = self.link_index(a)?; + let (page, index, ..) = self.link_index(a)?; self.apply(a, Edit::DeleteLink { page, index }) } @@ -225,6 +313,40 @@ impl Automation { } } +/// The zoom a `zoom` argument names: fit_page, fit_width, fit_height, fit_visible, actual +/// (100%), inherit (keep the reader's zoom), or a percentage, kept within 1–6400. +fn link_zoom(a: &Args) -> Result> { + match a.get("zoom") { + None | Some(Value::Null) => Ok(None), + Some(Value::Number(p)) => match p.as_f64() { + Some(p) if p.is_finite() && p > 0.0 => Ok(Some(LinkZoom::Percent(p.clamp(1.0, 6400.0)))), + _ => Err(bad("zoom must be a positive percentage")), + }, + Some(Value::String(z)) => Ok(Some(match z.as_str() { + "fit_page" => LinkZoom::FitPage, + "fit_width" => LinkZoom::FitWidth, + "fit_height" => LinkZoom::FitHeight, + "fit_visible" => LinkZoom::FitVisible, + "actual" => LinkZoom::Percent(100.0), + "inherit" => LinkZoom::Inherit, + o => return Err(bad(format!("unknown zoom {o:?}"))), + })), + Some(_) => Err(bad("zoom is a name or a percentage")), + } +} + +/// How `link_list` reports a link's zoom: the names `link_zoom` reads, a percentage as a number. +fn zoom_value(zoom: LinkZoom) -> Value { + match zoom { + LinkZoom::FitPage => json!("fit_page"), + LinkZoom::FitWidth => json!("fit_width"), + LinkZoom::FitHeight => json!("fit_height"), + LinkZoom::FitVisible => json!("fit_visible"), + LinkZoom::Inherit => json!("inherit"), + LinkZoom::Percent(p) => json!(p), + } +} + fn dest_view_value(view: DestView) -> Value { match view { DestView::Top => json!({ "kind": "top" }), diff --git a/crates/automation/src/measure.rs b/crates/automation/src/measure.rs index ac3d2d95f1..932f85ad9a 100644 --- a/crates/automation/src/measure.rs +++ b/crates/automation/src/measure.rs @@ -147,7 +147,7 @@ impl Automation { Ok(json!({"point":point,"kind":s.kind,"distance":measure::distance(at,point)})) }) .transpose()?; - Ok(json!({"snap":snap,"truncated":geometry.truncated,"intersection_limited":intersection_limited,"segments":geometry.segments.len()})) + Ok(json!({"snap":snap,"truncated":geometry.truncated,"intersection_limited":intersection_limited,"segments":geometry.segment_count()})) } pub(crate) fn measurement_export(&self, a: &Args) -> Result { let all = self.doc(a)?.measurements().map_err(failed)?; diff --git a/crates/automation/src/model_render.rs b/crates/automation/src/model_render.rs index dbfabc6708..b69c5a6371 100644 --- a/crates/automation/src/model_render.rs +++ b/crates/automation/src/model_render.rs @@ -2,7 +2,9 @@ use pdfcraft_engine::model3d::Model3dKind; use pdfcraft_prc::PrcFile; -use pdfcraft_prc_render::{Camera, RenderOptions, assemble, render}; +use pdfcraft_prc_render::nav::{Axis, NamedView, Orbit, Upright, reach}; +use pdfcraft_prc_render::{Bounds, Camera, RenderOptions, WorldMesh, assemble, render}; +use serde_json::Value; use super::Automation; use super::dispatch::Content; @@ -22,7 +24,6 @@ impl Automation { let page = self.page(a)?; let width = side(a, "width")?; let height = side(a, "height")?; - let direction = direction(a)?; let perspective = a.opt_bool("perspective")?.unwrap_or(true); let doc = self.doc(a)?; let found = doc.model_3d_annotations(page).map_err(failed)?; @@ -42,7 +43,14 @@ impl Automation { let prc = PrcFile::parse(&model.data).map_err(|e| failed(format!("the 3D model could not be read: {e}")))?; let meshes = assemble(&prc).map_err(failed)?; let aspect = f64::from(width) / f64::from(height); - let camera = Camera::fit_meshes(&meshes, direction, up_for(direction), perspective, aspect).map_err(failed)?; + let camera = match a.get("orbit") { + Some(_) if a.get("direction").is_some() => return Err(Args::wrong("orbit", "left out when direction is given")), + Some(moves) => orbit_camera(moves, &meshes, perspective, aspect)?, + None => { + let direction = direction(a)?; + Camera::fit_meshes(&meshes, direction, up_for(direction), perspective, aspect).map_err(failed)? + } + }; let image = render(&meshes, &camera, &RenderOptions { width, height, ..RenderOptions::default() }).map_err(failed)?; let data = encode_png(image.width, image.height, &premultiplied(image.rgba))?; Ok(vec![Content::Png { data, width: image.width, height: image.height }]) @@ -71,6 +79,65 @@ fn direction(a: &Args) -> Result<[f64; 3]> { Ok(out) } +/// The keys `orbit` takes. +const ORBIT_KEYS: [&str; 7] = ["view", "up", "front", "turn", "zoom", "zoom_at", "pan"]; + +/// The camera `orbit` names: a named view about an up axis, then a turn, a zoom and a pan, as a +/// viewer's orbit control would apply them. +fn orbit_camera(moves: &Value, meshes: &[WorldMesh], perspective: bool, aspect: f64) -> Result { + let wrong = |what: &str| Args::wrong("orbit", what); + let moves = moves.as_object().ok_or_else(|| wrong("an object"))?; + if let Some(key) = moves.keys().find(|k| !ORBIT_KEYS.contains(&k.as_str())) { + return Err(wrong(&format!("an object without {key}; its keys are {}", ORBIT_KEYS.join(", ")))); + } + let token = |key: &str| moves.get(key).map(|v| v.as_str().ok_or_else(|| wrong(&format!("{key} as a string")))); + let axis = |key: &str| token(key).map(|t| Axis::from_token(t?).ok_or_else(|| wrong(&format!("{key} as one of +x, +y, +z, -x, -y, -z")))); + let mut upright = Upright::default(); + if let Some(up) = axis("up") { + upright = upright.with_up(up?); + } + if let Some(front) = axis("front") { + let front = front?; + if !upright.up().perpendicular(front) { + return Err(wrong("front perpendicular to up")); + } + upright = upright.with_front(front); + } + let view = match token("view") { + None => NamedView::Isometric, + Some(t) => NamedView::from_token(t?).ok_or_else(|| wrong("view as one of iso, front, right, top, back"))?, + }; + let mut orbit = Orbit::named(view, perspective, upright); + let bounds = Bounds::of(meshes).ok_or_else(|| failed("the model has no vertex"))?; + let reach = reach(meshes, &bounds); + if let Some(v) = moves.get("turn") { + let [yaw, pitch] = pair(v).ok_or_else(|| wrong("turn as [yaw, pitch] in degrees"))?; + orbit.turn(yaw.to_radians(), pitch.to_radians()); + } + if let Some(v) = moves.get("zoom") { + let factor = v.as_f64().filter(|f| f.is_finite() && *f > 0.0).ok_or_else(|| wrong("zoom as a positive number"))?; + let at = match moves.get("zoom_at") { + None => [0.0, 0.0], + Some(v) => pair(v).ok_or_else(|| wrong("zoom_at as [right, up] in picture heights"))?, + }; + orbit.zoom_at(&bounds, reach, aspect, factor, at); + } else if moves.contains_key("zoom_at") { + return Err(wrong("an object with zoom when it has zoom_at")); + } + if let Some(v) = moves.get("pan") { + orbit.pan_by(pair(v).ok_or_else(|| wrong("pan as [right, up] in model radii"))?); + } + orbit.camera(&bounds, reach, aspect).map_err(failed) +} + +/// Two finite numbers. +fn pair(v: &Value) -> Option<[f64; 2]> { + match v.as_array()?.as_slice() { + [a, b] => Some([a.as_f64().filter(|f| f.is_finite())?, b.as_f64().filter(|f| f.is_finite())?]), + _ => None, + } +} + /// Z is up, as CAD models are drawn, unless the view looks along it. fn up_for([x, y, z]: [f64; 3]) -> [f64; 3] { if x.hypot(y) <= z.abs() * 1e-6 { [0.0, 1.0, 0.0] } else { [0.0, 0.0, 1.0] } diff --git a/crates/automation/src/pages.rs b/crates/automation/src/pages.rs index 6db58ab475..7bab26e6c1 100644 --- a/crates/automation/src/pages.rs +++ b/crates/automation/src/pages.rs @@ -235,6 +235,10 @@ impl Automation { let (mut html_left, mut spent, mut cut) = (html_budget, false, false); for (&p, t) in pages.iter().zip(texts) { let mut page = json!({ "page": p + 1, "text": text(&t) }); + // Content cut at the renderer's limits (#623): the text is incomplete, and said so. + if t.truncated { + page["truncated"] = json!(true); + } if html { // Copy with formatting: the same glyphs as an HTML fragment, within one budget for // the whole call; a page past it gets none and the call says so. @@ -270,6 +274,8 @@ impl Automation { /// `limit` is reached, so a bounded search doesn't extract (and cache) the whole document /// (#971). One match past the limit supplies `next`. Reached extraction failures are reported /// as bounded warnings; `complete` requires both no continuation and no failed pages. + /// `stop_at_limit` drops that lookahead: the search ends at the `limit`th match, so a lone match + /// never sends it to the end of the document, and `next` is only where to resume (#5328). pub(crate) fn text_find(&mut self, a: &Args) -> Result { self.text_find_with(a, &mut extract_parallel) } @@ -289,11 +295,15 @@ impl Automation { Some(p) => return Err(ToolError::InvalidArgs(format!("start_page {p} is out of range: the document has {n} pages"))), }; let skip = usize::try_from(a.opt_int("skip")?.unwrap_or(0).max(0)).unwrap_or(usize::MAX); + let stop_at_limit = a.opt_bool("stop_at_limit")?.unwrap_or(false); let mut matches = Vec::new(); let mut next = None; + let mut stopped = false; let mut warnings = Vec::new(); let mut warning_count = 0usize; - let (mut first, mut batch) = (start, FIND_FIRST_BATCH); + // A search that stops at its limit starts with one page, so a match on it reads nothing + // more; the batches still double for a match further in. + let (mut first, mut batch) = (start, if stop_at_limit { 1 } else { FIND_FIRST_BATCH }); 'search: while first < n { let end = first.saturating_add(batch).min(n); let pages: Vec = (first..end).collect(); @@ -315,13 +325,19 @@ impl Automation { let skip_here = if p == start { skip } else { 0 }; // One extra match proves that the caller needs a continuation. Include skipped // matches on the starting page when bounding the temporary per-page result. - let needed = limit.saturating_sub(matches.len()).saturating_add(skip_here).saturating_add(1); + let needed = limit.saturating_sub(matches.len()).saturating_add(skip_here).saturating_add(usize::from(!stop_at_limit)); for (k, r) in text.find_opts_limited(query, false, false, needed).into_iter().enumerate().skip(skip_here) { if matches.len() == limit { next = Some(json!({ "start_page": p + 1, "skip": k })); break 'search; } matches.push(json!({ "page": p + 1, "text": text.text_of(r.clone()), "rects": text.line_rects(r) })); + if stop_at_limit && matches.len() == limit { + // Resume just past this match; whether another exists isn't looked at. + next = Some(json!({ "start_page": p + 1, "skip": k.saturating_add(1) })); + stopped = true; + break 'search; + } } } first = end; @@ -329,8 +345,13 @@ impl Automation { } let mut out = json!({ "query": query, "count": matches.len(), "complete": next.is_none() && warning_count == 0, "matches": matches, "warnings_truncated": warning_count > warnings.len(), "warning_count": warning_count, "warnings": warnings }); - if let (Some(next), Some(o)) = (next, out.as_object_mut()) { - o.insert("next".into(), next); + if let Some(o) = out.as_object_mut() { + if let Some(next) = next { + o.insert("next".into(), next); + } + if stop_at_limit { + o.insert("stopped_at_limit".into(), stopped.into()); + } } Ok(out) } @@ -393,6 +414,8 @@ fn render_warning_text(warning: pdfcraft_render::RenderWarning) -> &'static str match warning { RenderWarning::ContentTruncated => "content past the page's safety budget was skipped", RenderWarning::ColourFallback => "a colour that could not be computed was painted in a fallback colour", + RenderWarning::MalformedIndexedImage => "an image with a malformed Indexed colour table may be drawn wrongly or not at all", + RenderWarning::Type3WidthsMissing => "a Type 3 font has no /Widths; its glyphs were spaced by the widths their glyph procedures give", } } diff --git a/crates/automation/src/paths.rs b/crates/automation/src/paths.rs index e13d0a1452..425cb3792e 100644 --- a/crates/automation/src/paths.rs +++ b/crates/automation/src/paths.rs @@ -1,5 +1,5 @@ use super::error::{Result, failed}; -use super::{Automation, Path, PathBuf, StagingName, file_access, foreign_share, staging_suffixes}; +use super::{Automation, Path, PathBuf, StagingName, file_access, staging_suffixes}; impl Automation { // ---- paths ------------------------------------------------------------------------------- @@ -39,7 +39,13 @@ impl Automation { let Some(root) = &self.root else { return Ok(p.to_path_buf()) }; let outside = || failed(format!("{path} is outside the allowed directory {}", root.display())); let joined = lexical(&root.join(p)).ok_or_else(outside)?; - if foreign_share(&joined, root) { + // Followed one name at a time first: nothing outside the root or its own parent folders is + // looked up, not even to see whether it exists. What can't be followed inside the root is + // said below. Made absolute the way Windows reads it first: it drops a folder name's + // trailing dot (`in.` is `in`), but `follow` looks names up below `\\?\` folders, where it + // doesn't. + let as_read = std::path::absolute(&joined).map_err(|_| outside())?; + if matches!(file_access::follow(path, &as_read, root), file_access::Toward::Outside) { return Err(outside()); } let mut existing = joined.as_path(); diff --git a/crates/automation/src/printing.rs b/crates/automation/src/printing.rs index 9a1c1a6b7c..dd3f949db2 100644 --- a/crates/automation/src/printing.rs +++ b/crates/automation/src/printing.rs @@ -1,7 +1,7 @@ //! Printing tools: list printers, and print (or save the print-ready PDF) with Acrobat's Print //! dialog options. -use pdfcraft_engine::print::{self, Binding, BookletSubset, Content, Layout, Orientation, PAPERS, PageOrder, SizeMode, Subset, spool}; +use pdfcraft_engine::print::{self, Binding, BookletSubset, Layout, Orientation, PAPERS, PageOrder, SizeMode, Subset, spool}; use serde_json::{Value, json}; use crate::{Args, Automation, Result, ToolError, failed, write_atomic}; @@ -60,8 +60,10 @@ fn paper_named(name: Option<&str>) -> Result<(usize, (f64, f64))> { /// The spooler job for `doc_print` to `printer`. It feeds the paper the PDF was laid out on /// (`PAPERS[paper]`, as `media` after the driver's `options`) rather than the queue's default -/// paper, which a job naming none got (A4 under Letter sheets on the queue in #3752). -fn spool_job(a: &Args, printer: &str, title: String, duplex: Option, paper: usize) -> Result { +/// paper, which a job naming none got (A4 under Letter sheets on the queue in #3752), and tells +/// CUPS which way the laid-out `sheets` run, as the Print dialog does (#2713): an orientation the +/// caller chose in `options` stands. +fn spool_job(a: &Args, printer: &str, title: String, duplex: Option, paper: usize, sheets: &[print::Sheet]) -> Result { let options = match a.get("options") { None => Vec::new(), Some(Value::Object(m)) => m @@ -81,11 +83,54 @@ fn spool_job(a: &Args, printer: &str, title: String, duplex: Option = options.into_iter().chain(print::media_option(paper)).collect(); + print::add_cups_orientation(&mut options, sheets); + options + }, }) } impl Automation { + /// Print production ▸ Convert colours: every colour on the pages in gray, rgb or cmyk. + pub(crate) fn convert_colours(&mut self, a: &Args) -> Result { + let n = self.doc(a)?.info.pages.len(); + let pages = match a.opt_ints("pages")? { + Some(_) => self.pages(a, "pages")?, + None => (0..n).collect(), + }; + let target = match a.opt_str("to")?.unwrap_or("gray") { + "gray" | "grayscale" => pdfcraft_engine::ColourTarget::Gray, + "rgb" => pdfcraft_engine::ColourTarget::Rgb, + "cmyk" => pdfcraft_engine::ColourTarget::Cmyk, + other => return Err(ToolError::InvalidArgs(format!("unknown target {other:?} (gray, rgb, cmyk)"))), + }; + let settings = pdfcraft_engine::ConvertColours { + target, + preserve_black: a.opt_bool("preserve_black")?.unwrap_or(true), + images: a.opt_bool("images")?.unwrap_or(true), + annotations: a.opt_bool("annotations")?.unwrap_or(false), + }; + let id = self.doc(a)?.id; + let found = self.session.count_colours(id, &pages, &settings); + if found.colours == 0 && found.images == 0 { + return Ok(json!({ "colours": 0, "images": 0, "images_left": found.images_left, "pages": 0, "loses_conformance": [] })); + } + // PDF/X has no declaration of its own here: it always carries an output intent. + let declared = self.doc(a)?.standards(); + let pdfa = declared.pdfa.map(|(part, level)| format!("PDF/A-{part}{}", level.to_lowercase())); + let loses: Vec = pdfa.into_iter().chain(declared.output_intents.iter().map(|i| format!("output intent {i}"))).collect(); + let r = self.apply(a, pdfcraft_engine::Edit::ConvertColours { pages, settings })?; + Ok(json!({ + "colours": found.colours, + "images": found.images, + "images_left": found.images_left, + "pages": found.pages, + "loses_conformance": loses, + "undo": r["undo"], + })) + } + /// `printers`: the system's printers, the default marked, and each one's two-sided support /// where the spooler reports it. pub(crate) fn printers(&self) -> Result { @@ -129,11 +174,20 @@ impl Automation { let pages = print::select_pages(count, a.opt_str("pages")?, &labels, subset, a.opt_bool("reverse")?.unwrap_or(false)) .map_err(|e| bad(e.to_string()))?; let layout = match a.opt_str("layout")?.unwrap_or("fit") { - "fit" => Layout::Size(SizeMode::Fit), - "actual" => Layout::Size(SizeMode::Actual), - "shrink" => Layout::Size(SizeMode::Shrink), "custom" => Layout::Size(SizeMode::Custom(a.opt_num("scale")?.ok_or_else(|| bad("custom needs scale (percent)"))?)), "multiple" => { + // Custom pages per sheet: `cols` across by `rows` down, both or neither. + let grid = match (a.opt_int("cols")?, a.opt_int("rows")?) { + (None, None) => None, + (Some(cols), Some(rows)) => { + if a.opt_int("per_sheet")?.is_some() { + return Err(bad("give per_sheet, or cols and rows, not both")); + } + let grid = usize::try_from(cols).ok().zip(usize::try_from(rows).ok()).and_then(|(cols, rows)| Layout::grid(cols, rows)); + Some(grid.ok_or_else(|| bad(format!("cols and rows must each be 1 to {}", print::MAX_GRID)))?) + } + _ => return Err(bad("a custom grid needs both cols and rows")), + }; let per = a.opt_int("per_sheet")?.unwrap_or(2).clamp(1, 256) as usize; let order = match a.opt_str("order")?.unwrap_or("horizontal") { "horizontal" => PageOrder::Horizontal, @@ -143,14 +197,15 @@ impl Automation { "cut-stack" => PageOrder::CutStack, o => return Err(bad(format!("unknown order {o:?}"))), }; - match Layout::multiple(per) { - Layout::Multiple { cols, rows, per_sheet, .. } => Layout::Multiple { + match grid.unwrap_or_else(|| Layout::multiple(per)) { + Layout::Multiple { cols, rows, per_sheet, custom, .. } => Layout::Multiple { cols, rows, per_sheet, order, border: a.opt_bool("border")?.unwrap_or(false), auto_rotate: a.opt_bool("auto_rotate")?.unwrap_or(true), + custom, }, l => l, } @@ -169,9 +224,12 @@ impl Automation { overlap: a.opt_num("overlap")?.unwrap_or(18.0), cut_marks: a.opt_bool("cut_marks")?.unwrap_or(true), }, - l => return Err(bad(format!("unknown layout {l:?}"))), + l => Layout::Size(print::names::parse_size_mode(l, 100.0).ok_or_else(|| bad(format!("unknown layout {l:?}")))?), }; let duplex = job_duplex(a.opt_str("duplex")?.unwrap_or("off"), &layout)?; + if (a.opt_int("cols")?.is_some() || a.opt_int("rows")?.is_some()) && !matches!(layout, Layout::Multiple { .. }) { + return Err(bad("cols and rows need layout multiple")); + } if a.opt_str("order")? == Some("cut-stack") && !matches!(layout, Layout::Multiple { .. }) { return Err(bad("cut-stack order needs layout multiple")); } @@ -181,13 +239,8 @@ impl Automation { "landscape" => Orientation::Landscape, o => return Err(bad(format!("unknown orientation {o:?}"))), }; - let content = match a.opt_str("comments_forms")?.unwrap_or("document-and-markups") { - "document" => Content::Document, - "document-and-markups" => Content::DocumentAndMarkups, - "document-and-stamps" => Content::DocumentAndStamps, - "form-fields-only" => Content::FormFieldsOnly, - c => return Err(bad(format!("unknown comments_forms {c:?}"))), - }; + let content = a.opt_str("comments_forms")?.unwrap_or("document-and-markups"); + let content = print::names::parse_content(content).ok_or_else(|| bad(format!("unknown comments_forms {content:?}")))?; let (paper_index, paper) = paper_named(a.opt_str("paper")?)?; // Print as image (Acrobat's Advanced Print Setup): pictures at `dpi`, grey when // `grayscale` is also asked for. @@ -202,7 +255,8 @@ impl Automation { }; let settings = print::Settings { pages, paper, orientation, layout, content, raster, crops: Vec::new() }; let sizes: Vec<(f64, f64)> = self.doc(a)?.info.pages.iter().map(|p| (p.width as f64, p.height as f64)).collect(); - let sheets = print::layout(&sizes, &settings).map_err(|e| bad(e.to_string()))?.len(); + let laid_out = print::layout(&sizes, &settings).map_err(|e| bad(e.to_string()))?; + let sheets = laid_out.len(); let bytes = self.session.print_pdf(id, &settings).map_err(failed)?; let mut out = json!({ "sheets": sheets, "pages": settings.pages.len(), "bytes": bytes.len() }); match (a.opt_str("path")?, a.opt_str("printer")?) { @@ -212,7 +266,7 @@ impl Automation { out["path"] = json!(path.to_string_lossy()); } (None, Some(printer)) => { - let job = spool_job(a, printer, self.doc(a)?.name.clone(), duplex, paper_index)?; + let job = spool_job(a, printer, self.doc(a)?.name.clone(), duplex, paper_index, &laid_out)?; out["job"] = json!(spool::submit(&bytes, &job).map_err(|e| failed(e.to_string()))?); } _ => return Err(bad("pass either path (save the print-ready PDF) or printer (a name from printers, or \"default\")")), @@ -251,13 +305,31 @@ mod tests { assert!(duplex("sideways", false).is_err()); } + /// #2713, #3739: `doc_print` to a printer tells CUPS when every sheet runs landscape, after + /// the paper, so a landscape sheet is turned onto the portrait paper rather than printed + /// unrotated and cut off. Portrait and mixed jobs say nothing, and an orientation passed in + /// `options` stands and isn't sent twice. + #[test] + fn printing_landscape_sheets_to_a_printer_asks_cups_to_turn_them() { + let sheets = |sizes: &[(f64, f64)]| sizes.iter().map(|&size| print::Sheet { size, ..Default::default() }).collect::>(); + let job = |args: Value, s: &[print::Sheet]| spool_job(&Args(&args), "Office", "a.pdf".into(), None, 0, s).unwrap().options; + let media = ("media".to_string(), "Letter".to_string()); + let landscape = sheets(&[(792.0, 612.0), (792.0, 612.0)]); + let turned = if cfg!(windows) { vec![media.clone()] } else { vec![media.clone(), ("orientation-requested".to_string(), "4".to_string())] }; + assert_eq!(job(json!({}), &landscape), turned); + assert_eq!(job(json!({}), &sheets(&[(612.0, 792.0)])), vec![media.clone()], "portrait sheets"); + assert_eq!(job(json!({}), &sheets(&[(792.0, 612.0), (612.0, 792.0)])), vec![media.clone()], "a job that mixes them"); + let chosen = ("orientation-requested".to_string(), "5".to_string()); + assert_eq!(job(json!({ "options": { "orientation-requested": "5" } }), &landscape), [chosen, media], "the caller's own choice, once"); + } + /// #3752: `doc_print` to a printer sent no paper, so the queue's default paper stood in for /// the one the sheets were laid out on (A4 under Letter sheets on the report's queue). #[test] fn printing_to_a_printer_feeds_the_laid_out_paper() { let job = |args: Value, paper_name: Option<&str>| { let (index, _) = paper_named(paper_name).unwrap(); - spool_job(&Args(&args), "Office", "a.pdf".into(), None, index).unwrap() + spool_job(&Args(&args), "Office", "a.pdf".into(), None, index, &[]).unwrap() }; let media = |m: &str| ("media".to_string(), m.to_string()); // US Letter when no paper is asked for, after the driver's own options. @@ -273,7 +345,7 @@ mod tests { assert_eq!(job(json!({}), Some("letter")).options, [media("Letter")]); assert_eq!(job(json!({}), Some("ARCH D")).options, [media("na_arch-d_24x36in")]); assert!(matches!(paper_named(Some("Quarto")), Err(ToolError::InvalidArgs(m)) if m.contains("US Letter"))); - assert_eq!(spool_job(&Args(&json!({})), "default", String::new(), None, 0).ok().map(|j| j.printer), Some(None)); - assert!(spool_job(&Args(&json!({ "options": { "InputSlot": 3 } })), "Office", String::new(), None, 0).is_err()); + assert_eq!(spool_job(&Args(&json!({})), "default", String::new(), None, 0, &[]).ok().map(|j| j.printer), Some(None)); + assert!(spool_job(&Args(&json!({ "options": { "InputSlot": 3 } })), "Office", String::new(), None, 0, &[]).is_err()); } } diff --git a/crates/automation/src/redact.rs b/crates/automation/src/redact.rs index 6ec66ce680..1ed8fcbde0 100644 --- a/crates/automation/src/redact.rs +++ b/crates/automation/src/redact.rs @@ -2,8 +2,8 @@ //! (removing what they cover for good) or clear them. use pdfcraft_engine::{ - Edit, Hidden, NewAnnotation, REDACT_MAX_WORDS, REDACT_VALIDATORS, REDACTION_CODE_SETS, RedactPattern, RedactRegex, RedactValidator, - RedactionCodeSet, Shape, Style, find_pattern, rect_quad, redact_word_list, + Edit, HIDDEN, Hidden, NewAnnotation, REDACT_MAX_WORDS, REDACT_VALIDATORS, REDACTION_CODE_SETS, RedactPattern, RedactRegex, RedactValidator, + RedactionCodeSet, Shape, Style, find_pattern, rect_quad, redact_word_list, redacted_file_name, }; use serde_json::{Value, json}; @@ -200,6 +200,26 @@ impl Automation { let after = self.doc(a)?.redaction_marks(); out["applied"] = json!(before - after); out["marks_pending"] = json!(after); + // Where the removed text is still held that applying leaves as it is. + let left = self.doc(a)?.redaction_elsewhere().unwrap_or_default(); + out["still_holding"] = json!({ + "bookmarks": left.bookmarks, + "attachments": left.attachments, + "comments": left.comments, + "xmp": left.xmp, + "total": left.total(), + "unsearched": left.unsearched, + }); + // What became of a PDF/A or PDF/UA claim in the document XMP the scrub removed. + let claim = |c: pdfcraft_engine::RedactionClaim| match c { + pdfcraft_engine::RedactionClaim::None => "none", + pdfcraft_engine::RedactionClaim::Kept => "kept", + pdfcraft_engine::RedactionClaim::Lost => "lost", + }; + let id = self.doc(a)?.redaction_identification().unwrap_or_default(); + out["identification"] = json!({ "pdfa": claim(id.pdfa), "pdfua": claim(id.pdfua) }); + // The name to save the redacted copy under, as Save does in the app. + out["save_as"] = json!(redacted_file_name(&self.doc(a)?.name)); Ok(out) } @@ -235,6 +255,9 @@ impl Automation { Ok(json!({ "findings": items, "count": items.len() })) } + /// `doc_remove_hidden`: remove the listed categories of hidden information, or all of them + /// (Sanitize Document) when none are listed, and report how many items the listed + /// categories lost as `removed`. pub(crate) fn doc_remove_hidden(&mut self, a: &Args) -> Result { let which = match a.get("categories") { None => None, @@ -253,8 +276,11 @@ impl Automation { // One scan, for this check and for what was removed. The removal refuses when it counts // every listed category as 0 (an AcroForm with no fields counts 1), with an error about // redaction marks; this reads the same scan, so it refuses exactly then. A document that - // can't be edited has no scan: `apply` says why. - let info = self.doc(a)?.hidden_info(); + // can't be edited has no scan: `apply` says why. Only the listed categories are counted, + // here and after: the others would read every page's content and every object for + // nothing (#5873). + let listed = which.clone().unwrap_or_else(|| HIDDEN.to_vec()); + let info = self.doc(a)?.hidden_info_of(&listed); let edit = match which { None => Edit::Sanitize, Some(which) => { @@ -266,7 +292,7 @@ impl Automation { }; let before: usize = info.iter().map(|c| c.1).sum(); let mut out = self.apply(a, edit)?; - let after: usize = self.doc(a)?.hidden_info().iter().map(|c| c.1).sum(); + let after: usize = self.doc(a)?.hidden_info_of(&listed).iter().map(|c| c.1).sum(); out["removed"] = json!(before.saturating_sub(after)); Ok(out) } diff --git a/crates/automation/src/signing.rs b/crates/automation/src/signing.rs index df0fd643fb..f2b583c070 100644 --- a/crates/automation/src/signing.rs +++ b/crates/automation/src/signing.rs @@ -2,6 +2,7 @@ //! document, and manage trusted certificates. Rectangles are points from the top-left of the //! displayed page. +use pdfcraft_engine::js::SignEvent; use pdfcraft_engine::sign::{self, Certificate, DigitalId, Modification, Name, PrivateKey}; use pdfcraft_engine::{SignOptions, SignatureInfo, SignatureStatus}; use serde_json::{Value, json}; @@ -175,6 +176,8 @@ impl Automation { Ok(json!({ "path": path.to_string_lossy(), "certificate": cert_json(&id.certificate) })) } + /// The digital ID `id` names: a .p12/.pfx file opened with `password`, or an OS key store + /// identity (`keychain:` on macOS, `windows:` on Windows). fn open_id(&self, a: &Args) -> Result { // A macOS Keychain identity: "keychain:" or "keychain:". if let Some(r) = a.str("id")?.strip_prefix("keychain:") { @@ -197,6 +200,8 @@ impl Automation { }) } + /// `sign_document`: sign the document with a digital ID into `out`, visibly (in a field or a + /// rectangle) or not, running a dynamic form's signing scripts around it. pub(crate) fn sign_document(&mut self, a: &Args) -> Result { let id = self.open_id(a)?; let doc = self.doc(a)?; @@ -246,30 +251,94 @@ impl Automation { Some(v) => labels_arg(v)?, }; let out = self.resolve(a.str("out")?, true)?; - let opts = SignOptions { - field, - page, - rect, - reason: a.opt_str("reason")?.map(str::to_string), - location: a.opt_str("location")?.map(str::to_string), - contact: a.opt_str("contact")?.map(str::to_string), - certify, - appearance, - appearance_labels, - graphic, - ..SignOptions::default() + // A dynamic form's signing scripts run only when asked: what preSign changes is signed, + // and the caller hasn't seen it. What they changed is listed. + let run_scripts = a.opt_bool("run_form_scripts")?.unwrap_or(false); + // Every argument is read before the preSign scripts change anything, so a bad one is + // refused with the document as it was. + let reason = a.opt_str("reason")?.map(str::to_string); + let location = a.opt_str("location")?.map(str::to_string); + let contact = a.opt_str("contact")?.map(str::to_string); + let signing = field.clone(); + let scripted = signing.as_deref().is_some_and(|f| { + self.session.sign_event_scripted(doc_id, f, SignEvent::Before) || self.session.sign_event_scripted(doc_id, f, SignEvent::After) + }); + // Each field's value and whether it is read-only, to tell what the preSign scripts changed. + let values = |me: &Automation| -> std::collections::BTreeMap, bool)> { + me.session.get(doc_id).map(|d| d.form.iter().map(|f| (f.name.clone(), (f.value.clone(), f.read_only()))).collect()).unwrap_or_default() }; + let was = values(self); + let before = match &signing { + Some(f) if run_scripts => self.session.run_sign_event(doc_id, f, SignEvent::Before).map_err(failed)?, + _ => false, + }; + let changed: Vec = if before { + values(self) + .into_iter() + .filter(|(name, now)| was.get(name) != Some(now)) + .map(|(name, (value, read_only))| { + let (old, old_ro) = was.get(&name).cloned().unwrap_or_default(); + json!({ "field": name, "before": old, "after": value, "read_only_before": old_ro, "read_only": read_only }) + }) + .collect() + } else { + Vec::new() + }; + let opts = + SignOptions { field, page, rect, reason, location, contact, certify, appearance, appearance_labels, graphic, ..SignOptions::default() }; let signed = self.session.sign(doc_id, &id, opts).map_err(|e| match e { // A graphic the signer refuses is the caller's to fix. pdfcraft_engine::EditError::Sign(m) if m.starts_with("the signature graphic") => bad(m), e => failed(e), - })?; - write_atomic(&out, &signed)?; + }); + let signed = match signed { + Ok(s) => s, + Err(e) => { + // Not signed: what the preSign scripts did is taken back. + if before { + let _ = self.session.undo(doc_id); + let _ = self.session.take_js_output(doc_id); + } + return Err(e); + } + }; + if let Err(e) = write_atomic(&out, &signed) { + // Not saved, so not signed: what the preSign scripts did is taken back too. + if before { + let _ = self.session.undo(doc_id); + let _ = self.session.take_js_output(doc_id); + } + return Err(e); + } let path = out.to_string_lossy().into_owned(); self.session.mark_signed(doc_id, signed.clone(), Some(path.clone())).map_err(failed)?; + // Then its postSign scripts; what they change is a change after signing, not saved yet. + // The file is signed and saved by now: a postSign step that fails is reported, not an error. + let (after, after_error) = match &signing { + Some(f) if run_scripts => match self.session.run_sign_event(doc_id, f, SignEvent::After) { + Ok(ran) => (ran, None), + Err(e) => (true, Some(format!("the postSign step couldn't finish: {e}"))), + }, + _ => (false, None), + }; + let mut scripts = self.session.take_js_output(doc_id); + scripts.errors.extend(after_error); let doc = self.doc(a)?; let newest = doc.signatures.iter().filter(|s| s.signed).max_by_key(|s| s.revision).cloned(); - Ok(json!({ "path": path, "bytes": signed.len(), "signature": newest.map(|s| self.sig_json(a, &s)).transpose()? })) + let mut result = json!({ "path": path, "bytes": signed.len(), "signature": newest.map(|s| self.sig_json(a, &s)).transpose()? }); + if before || after { + result["form_scripts"] = json!({ + "before_signing": before, + "after_signing": after, + "changed": changed, + "alerts": scripts.alerts.iter().map(crate::a11y::form_tools::alert_json).collect::>(), + "errors": scripts.errors, + "unsaved_changes": self.doc(a)?.dirty, + }); + } else if scripted { + result["form_scripts"] = json!({ "skipped": true, "why": "the field has signing scripts; pass run_form_scripts: true to run them" }); + } + Ok(result) } /// `{"image": path}` (PNG or JPEG), `{"text": name}` (typed in the script font) or @@ -310,6 +379,9 @@ impl Automation { pub(crate) fn sign_trust(&mut self, a: &Args) -> Result { let mut certs: Vec = if a.opt_bool("clear")?.unwrap_or(false) { Vec::new() } else { self.session.trusted_certificates().to_vec() }; + // Which certificates are already in, by their bytes: a set, so a bundle of many distinct + // certificates isn't compared against every one before it. + let mut known: std::collections::HashSet> = certs.iter().map(|c| c.raw.clone()).collect(); let paths: Vec = match a.get("paths") { None => Vec::new(), Some(v) => v @@ -330,7 +402,7 @@ impl Automation { sign::x509::load_certificates(&bytes).map_err(failed)? }; for c in found { - if !certs.iter().any(|t| t.raw == c.raw) { + if known.insert(c.raw.clone()) { certs.push(c); } } diff --git a/crates/automation/src/tests.rs b/crates/automation/src/tests.rs index ad0aa039dc..408cb7cac0 100644 --- a/crates/automation/src/tests.rs +++ b/crates/automation/src/tests.rs @@ -146,6 +146,84 @@ fn search_bounds_warning_volume_and_does_not_reach_distant_failures() { assert!(warnings.iter().all(|w| w["error"].as_str().unwrap().len() <= FIND_MAX_WARNING_BYTES && w["error_truncated"] == true)); } +#[test] +fn stop_at_limit_ends_at_a_lone_match_without_reading_the_distant_tail() { + let mut a = Automation::new(); + let id = search_document(&mut a, 200); + // #5328: "Page 1" is only on page 1 (every other readable page reads "Page 2"), and the last + // page fails, so only a search that runs on to the end hears about it. + let requested = std::cell::RefCell::new(Vec::new()); + let mut extract = |bytes: &Arc>, password: Option>, pages: &[usize]| { + requested.borrow_mut().extend_from_slice(pages); + let filler = extract_parallel(bytes, password.clone(), &[1]).remove(0).1; + let first = extract_parallel(bytes, password, &[0]).remove(0).1; + pages + .iter() + .map(|&p| { + ( + p, + match p { + 0 => first.clone(), + 199 => Err("synthetic page 200 failure".into()), + _ => filler.clone(), + }, + ) + }) + .collect::>() + }; + let looked_ahead = a.text_find_with(&Args(&json!({ "doc": id.0, "query": "Page 1", "limit": 1 })), &mut extract).unwrap(); + assert_eq!((looked_ahead["count"].clone(), looked_ahead["warning_count"].clone()), (json!(1), json!(1)), "{looked_ahead}"); + assert!(looked_ahead.get("next").is_none() && looked_ahead.get("stopped_at_limit").is_none()); + + a.texts.remove(&id); + requested.borrow_mut().clear(); + let args = json!({ "doc": id.0, "query": "Page 1", "limit": 1, "stop_at_limit": true }); + let stopped = a.text_find_with(&Args(&args), &mut extract).unwrap(); + assert_eq!(stopped["matches"][0]["page"], 1); + assert_eq!((stopped["count"].clone(), stopped["warning_count"].clone()), (json!(1), json!(0)), "{stopped}"); + assert_eq!((stopped["complete"].clone(), stopped["stopped_at_limit"].clone()), (json!(false), json!(true))); + assert_eq!(stopped["next"], json!({ "start_page": 1, "skip": 1 })); + assert_eq!(*requested.borrow(), [0], "a match on the first page reads only that page"); + + // Resuming finds nothing more and reaches the failing page, which it now reports. + let mut resume = args.clone(); + resume["start_page"] = stopped["next"]["start_page"].clone(); + resume["skip"] = stopped["next"]["skip"].clone(); + let rest = a.text_find_with(&Args(&resume), &mut extract).unwrap(); + assert_eq!((rest["count"].clone(), rest["stopped_at_limit"].clone(), rest["complete"].clone()), (json!(0), json!(false), json!(false))); + assert_eq!(rest["warnings"][0]["page"], 200); + assert!(rest.get("next").is_none()); +} + +#[test] +fn stop_at_limit_continuations_return_every_match_once_and_end_complete() { + let mut a = Automation::new(); + let id = search_document(&mut a, 120); + let all = json(&mut a, "text_find", json!({ "doc": id.0, "query": "1", "limit": 10_000 })); + assert_eq!(all["complete"], true); + // "1" appears twice on pages like 11 and 111, so resuming has to skip within a page. + for limit in [1, 2, 3] { + let mut args = json!({ "doc": id.0, "query": "1", "limit": limit, "stop_at_limit": true }); + let mut pieced = Vec::new(); + for _ in 0..=all["count"].as_u64().unwrap() { + let part = json(&mut a, "text_find", args.clone()); + assert_eq!(part["warning_count"], 0); + pieced.extend(part["matches"].as_array().unwrap().iter().cloned()); + if part["stopped_at_limit"] == false { + assert_eq!(part["complete"], true); + assert!(part.get("next").is_none()); + break; + } + assert_eq!((part["count"].clone(), part["complete"].clone()), (json!(limit), json!(false))); + args["start_page"] = part["next"]["start_page"].clone(); + args["skip"] = part["next"]["skip"].clone(); + } + assert_eq!(pieced, *all["matches"].as_array().unwrap(), "limit {limit}: no match lost or repeated"); + } + let none = json(&mut a, "text_find", json!({ "doc": id.0, "query": "absent", "limit": 1, "stop_at_limit": true })); + assert_eq!((none["count"].clone(), none["complete"].clone(), none["stopped_at_limit"].clone()), (json!(0), json!(true), json!(false))); +} + #[test] fn a_bounded_text_find_reads_only_the_pages_it_needs_and_says_it_stopped() { let dir = std::env::temp_dir().join(format!("pdfcraft-automation-find-{}", std::process::id())); @@ -192,6 +270,30 @@ fn a_bounded_text_find_reads_only_the_pages_it_needs_and_says_it_stopped() { let _ = std::fs::remove_dir_all(&dir); } +#[test] +fn a_save_without_a_path_stays_inside_the_root() { + let base = std::env::temp_dir().join(format!("pdfcraft-automation-save-root-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&base); + let root = base.join("root"); + let outside = base.join("outside"); + std::fs::create_dir_all(&root).unwrap(); + std::fs::create_dir_all(&outside).unwrap(); + let mut a = Automation::new().with_root(&root).unwrap(); + let bytes = Arc::new(numbered_pages(1)); + // Remembered path inside the root: still saves. + let inside = root.canonicalize().unwrap().join("in.pdf"); + let id = a.session.open("in.pdf", Some(inside.to_string_lossy().into_owned()), bytes.clone(), None).unwrap(); + json(&mut a, "doc_save", json!({ "doc": id.0 })); + assert!(inside.exists()); + // Remembered path outside the root: refused, and nothing is written there. + let escaped = outside.join("out.pdf"); + let id = a.session.open("out.pdf", Some(escaped.to_string_lossy().into_owned()), bytes, None).unwrap(); + let error = a.call("doc_save", &json!({ "doc": id.0 })).unwrap_err(); + assert!(error.to_string().contains("outside the allowed directory"), "{error}"); + assert!(!escaped.exists()); + let _ = std::fs::remove_dir_all(&base); +} + #[test] fn dot_dot_resolves_by_name_and_never_climbs_above_the_start() { let base = std::env::temp_dir(); @@ -219,6 +321,7 @@ fn names_from_documents_become_one_plain_file_name() { #[cfg(windows)] #[test] fn other_shares_and_device_paths_are_refused_by_name() { + use crate::file_access::foreign_share; let local = Path::new(r"\\?\C:\work\root"); for p in [ r"\\host\share\x.pdf", @@ -242,6 +345,28 @@ fn other_shares_and_device_paths_are_refused_by_name() { assert!(foreign_share(Path::new(r"\\attacker\docs\x.pdf"), shared)); } +/// Windows drops a trailing dot from a folder name when it opens a path (`root.` is `root`), so +/// with a root, a path that names the root or a folder above it that way still reaches the file +/// inside. A dotted name outside stays outside. +#[cfg(windows)] +#[test] +fn a_trailing_dot_on_the_root_or_above_it_names_the_folder_windows_opens() { + let base = std::env::temp_dir().join(format!("pdfcraft-automation-dot-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&base); + std::fs::create_dir_all(base.join("root")).unwrap(); + std::fs::create_dir_all(base.join("outside")).unwrap(); + std::fs::write(base.join("root").join("a.pdf"), b"x").unwrap(); + std::fs::write(base.join("outside").join("b.pdf"), b"x").unwrap(); + let a = Automation::new().with_root(base.join("root")).unwrap(); + let shown = base.to_string_lossy().into_owned(); + for p in [format!(r"{shown}\root.\a.pdf"), format!(r"{shown}.\root\a.pdf")] { + let resolved = a.resolve(&p, false); + assert!(resolved.is_ok(), "{p}: {resolved:?}"); + } + assert!(a.resolve(&format!(r"{shown}\outside.\b.pdf"), false).is_err()); + let _ = std::fs::remove_dir_all(&base); +} + /// A fresh, empty folder for one staging test. fn staging_dir(test: &str) -> PathBuf { let dir = std::env::temp_dir().join(format!("pdfcraft-staging-{}-{test}", std::process::id())); diff --git a/crates/automation/src/tools.rs b/crates/automation/src/tools.rs index feface85e4..e1320972b8 100644 --- a/crates/automation/src/tools.rs +++ b/crates/automation/src/tools.rs @@ -140,8 +140,8 @@ pub fn tools() -> Vec { .with(schema(json!({ "files": {"type":"array","minItems":1,"maxItems":1000,"description":"PDFs in Bates sequence order.","items":{"type":"object","properties":{"input":{"type":"string","minLength":1,"description":"Input PDF path."},"output":{"type":"string","minLength":1,"description":"New output PDF path; must not exist."},"password":{"type":"string","description":"Open or permissions password, if needed."}},"required":["input","output"],"additionalProperties":false}}, "start_number": {"type":"integer","minimum":1,"description":"First number in the sequence (default 1)."}, "digits": {"type":"integer","minimum":1,"maximum":12,"description":"Zero-padded number width (default 6)."}, - "prefix": {"type":"string","maxLength":128,"description":"Text before the number: at most 128 UTF-8 bytes, without # or >>."}, "suffix": {"type":"string","maxLength":128,"description":"Text after the number: at most 128 UTF-8 bytes, without # or >>."}, - "dry_run": {"type":"boolean","description":"Return each file's inclusive number interval without writing (default false)."} + "prefix": {"type":"string","maxLength":128,"description":"Bates prefix, up to 128 characters; cannot contain # or >>."}, "suffix": {"type":"string","maxLength":128,"description":"Bates suffix, up to 128 characters; cannot contain # or >>."}, + "dry_run": {"type":"boolean","description":"Return each file's inclusive number interval without writing (default false). Output collisions are checked conservatively: Windows compares case-insensitively, while Unix preserves case, so case-sensitive Windows and case-insensitive Unix volumes can differ from normal runs."} }), &["files"])), t("doc_batch", "Process files and folders", "Run convert, ocr and/or optimize over files and folders, one file at a time. recursive defaults to false. Preserve paths relative to each selected folder inside out_dir; duplicate inputs run once. Skip unsupported files, links and the output subtree. Existing outputs receive numbered names and are never replaced. Continue after per-file errors unless stop_on_error is true. Return completed/failed/skipped counts and per-file results. Discovery is bounded to 10000 entries and 64 folder levels. Convert must be first; each step can appear once. OCR uses installed models. Optimize uses Reduce File Size settings, optionally overriding target ppi (threshold 1.5 times ppi; black-and-white images go to the larger of ppi and 300) and JPEG quality; a larger candidate keeps the original bytes. Sources and open documents are unchanged.") .with(schema(json!({ @@ -154,17 +154,40 @@ pub fn tools() -> Vec { "dpi": {"type":"number","minimum":72,"maximum":600}, "language": {"type":"string"}, "skip_text_pages": {"type":"boolean"} }), &["paths","out_dir","steps"])), + t("doc_fix_hairlines", "Fix hairlines", "Print production: strokes narrower than narrower_than points on the page (after every transformation; zero-width lines always) are given width points, in page content and form XObjects; Type 3 glyphs and tiling patterns only when asked, changed in place wherever they are used. Both widths at most 10. Returns fixed (strokes), pages, and elsewhere (glyphs and patterns changed that something outside pages also uses); nothing to fix gives 0 and no edit.") + .cmd("prepress.hairlines") + .with(schema( + json!({ + "doc": doc(), "pages": pages("to fix (default all)"), + "narrower_than": { "type": "number", "minimum": 0, "description": "Points on the page (default 0.216)." }, + "width": { "type": "number", "exclusiveMinimum": 0, "description": "Points on the page (default 0.72)." }, + "type3": { "type": "boolean" }, "patterns": { "type": "boolean" }, + }), + &["doc"], + )), t("attachment_add", "Attach a file", "Attach the file at path to the document (document level, the Attachments panel; not a paper-clip comment on a page). name defaults to the file's name; the attachment's key, unique in the document, is returned. Undoable.") .cmd("edit.attach") .with(schema(json!({"doc": doc(), "path": {"type": "string"}, "name": {"type": "string"}, "description": {"type": "string"}}), &["doc", "path"])), t("attachment_delete", "Delete an attachment", "Remove the document-level attachment with this name (as doc_info lists it). Undoable.") .destructive() .with(schema(json!({"doc": doc(), "name": {"type": "string"}}), &["doc", "name"])), + t("doc_convert_colours", "Convert colours", "Print production: every colour operator in the pages' content and the artwork it draws rewritten in the target space (to: gray, rgb or cmyk) by the standard device formulas (device spaces, ICC by component count, Lab, Indexed, Separation and DeviceN tints; patterns, shadings and inline images stay). Not a prepress separation: no ICC profile and no output intent is used. Spot colours are converted, not kept: exact through a Type 2 tint, otherwise a gray of their darkness; /None stays. preserve_black (default true) keeps pure black as 100% K / gray 0; images (default true) re-encodes 8-bit gray and RGB pictures (JPEG stays JPEG, compressed again at quality 80; CMYK written lossless; soft masks kept), and leaves indexed, CMYK, masked and other pictures as they are; annotations also converts the pages' comment and field appearances. Returns how many colours and pictures changed on how many pages, images_left (pictures not in the target that stayed), and loses_conformance (the PDF/A level and output intents the file declared: it may no longer meet them, and still declares them; standards_remove_id takes that off). Undoable.") + .cmd("prepress.convert_colors") + .with(schema( + json!({ + "doc": doc(), "pages": pages("to convert (default all)"), + "to": { "type": "string", "enum": ["gray", "rgb", "cmyk"] }, + "preserve_black": { "type": "boolean" }, "images": { "type": "boolean" }, "annotations": { "type": "boolean" }, + }), + &["doc"], + )), + t("attachment_edit", "Edit an attachment description", "Change the description of a document attachment by its exact key from doc_info, or by a unique displayed name. An empty description clears it. The file and other attachment data are preserved. Undoable.") + .with(schema(json!({"doc": doc(), "key": {"type": "string"}, "name": {"type": "string"}, "description": {"type": "string"}}), &["doc", "description"])), t("doc_open", "Open a PDF", "Open a PDF file and return its document id, page count and whether it can be edited.") .cmd("file.open") .with(schema(json!({ "path": { "type": "string" }, "password": { "type": "string", "description": "User or owner password for encrypted files." } }), &["path"])), t("doc_list", "List open documents", "List the open documents with their ids, page counts and unsaved state.").ro().with(schema(json!({}), &[])), - t("doc_info", "Inspect a document", "Metadata, page sizes and labels, bookmarks, annotations, form fields, links, layers, page transitions and display times, the destination it opens at, attachments, fonts, security, repair notes and the objects that could not be read.") + t("doc_info", "Inspect a document", "Metadata, page sizes and labels, bookmarks, annotations, form fields, links, layers and the Layers panel's tree (layer_order), page transitions and display times, the destination it opens at, attachments (and, for a PDF portfolio, the one it presents first), fonts, security, repair notes and the objects that could not be read.") .ro() .cmd("file.properties") .with(schema(json!({ "doc": doc() }), &["doc"])), @@ -187,7 +210,7 @@ pub fn tools() -> Vec { }), &["doc"], )), - t("doc_set_info", "Set document metadata", "Set a document information entry such as Title, Author, Subject or Keywords, or a custom property under any other name (no spaces, delimiters, / or #; at most 127 bytes; doc_info lists them under custom); an empty value removes it. When the file has XMP metadata, the matching XMP property (dc:title, dc:creator, dc:description, pdf:Keywords, xmp:CreatorTool, pdf:Producer) is changed too, so viewers that read XMP first show the new value. Undoable.") + t("doc_set_info", "Set document metadata", "Set a document information entry such as Title, Author, Subject or Keywords, or a custom property under any other name (no spaces, delimiters, / or #; at most 127 bytes; doc_info lists them under custom); an empty value removes it. When the file has XMP metadata, the matching XMP property (dc:title, dc:creator, dc:description, pdf:Keywords, xmp:CreatorTool, pdf:Producer) is changed too, so viewers that read XMP first show the new value, and a custom property is kept there as pdfx: when its name can be an XML name (in a PDF/A file only one already there is changed). Undoable.") .with(schema(json!({ "doc": doc(), "key": { "type": "string" }, "value": { "type": "string" } }), &["doc", "key", "value"])), t("page_render", "Render a page", "Render one page to a PNG image (default 96 dpi, at most 600). When the picture is incomplete or approximate (content cut at the page's safety budget, a colour painted in a fallback colour), a JSON part follows the image: { page, warnings: [sentence, ...] }.") .ro() @@ -195,28 +218,47 @@ pub fn tools() -> Vec { "hide_text_block": { "type": "integer", "minimum": 1, "description": "Render an inline editor background without this paragraph (number from text_paragraphs, resolved as text_edit does: refused once an edit merged that paragraph into a neighbour). Does not modify the document." }, "hide_added_text": { "type": "integer", "minimum": 1, "description": "Render a retyping background without this added text item (added index from object_list). Mutually exclusive with hide_text_block. Does not modify the document." } }), &["doc", "page"])), - t("model_render", "Render a 3D model", "Draw a page's 3D model (a 3D annotation's PRC stream) to a PNG, framed to fill the image, with headlight shading on a white background. model is its 1-based position among the page's 3D models (default 1). direction is the view direction [x, y, z] from the eye towards the model (default [-1, 1, -1]); Z is up unless the view looks along Z. perspective defaults to true. U3D models can't be drawn yet and give an error. Does not modify the document.") + t("model_render", "Render a 3D model", "Draw a page's 3D model (a 3D annotation's PRC stream) to a PNG, framed to fill the image, with headlight shading on a white background. model is its 1-based position among the page's 3D models (default 1). direction is the view direction [x, y, z] from the eye towards the model (default [-1, 1, -1]); Z is up unless the view looks along Z. orbit, in place of direction, frames the model as a viewer's orbit control does (the same size from every side) and applies, in order: view (iso, front, right, top or back; default iso) about up (+x, +y, +z, -x, -y or -z; default +z) with the Front view looking along front (an axis perpendicular to up; default +y, or -z when y is up), then turn [yaw, pitch] in degrees (pitch looks down and stops just short of the up axis), then zoom (a factor, the result held within 0.05 to 50) about the picture point zoom_at [right, up] in picture heights from the centre (default the centre), then pan [right, up] in model radii (it moves the target, so the model moves the other way). perspective defaults to true. U3D models can't be drawn yet and give an error. Does not modify the document.") .ro() .with(schema(json!({ "doc": doc(), "page": { "type": "integer", "minimum": 1 }, "model": { "type": "integer", "minimum": 1 }, "width": { "type": "integer", "minimum": 1, "maximum": 4096 }, "height": { "type": "integer", "minimum": 1, "maximum": 4096 }, - "direction": { "type": "array", "items": { "type": "number" }, "minItems": 3, "maxItems": 3 }, - "perspective": { "type": "boolean" } + "direction": { "type": "array", "items": { "type": "number" }, "minItems": 3, "maxItems": 3, "description": "The view direction from the eye towards the model (default [-1, 1, -1]). Mutually exclusive with orbit." }, + "perspective": { "type": "boolean" }, + "orbit": { "type": "object", "description": "A viewer's orbit moves, applied in the order of these keys; all optional. Mutually exclusive with direction.", "properties": { + "view": { "type": "string", "enum": ["iso", "front", "right", "top", "back"], "description": "The view to start from (default iso)." }, + "up": { "type": "string", "enum": ["+x", "+y", "+z", "-x", "-y", "-z"], "description": "The model axis that appears upward (default +z)." }, + "front": { "type": "string", "enum": ["+x", "+y", "+z", "-x", "-y", "-z"], "description": "The axis the Front view looks along, perpendicular to up (default +y, or -z when y is up)." }, + "turn": { "type": "array", "items": { "type": "number" }, "minItems": 2, "maxItems": 2, "description": "[yaw, pitch] in degrees, pitch looking down (default [0, 0])." }, + "zoom": { "type": "number", "exclusiveMinimum": 0, "description": "A factor on the fitted view (default 1); the result is held within 0.05 to 50." }, + "zoom_at": { "type": "array", "items": { "type": "number" }, "minItems": 2, "maxItems": 2, "description": "The picture point the zoom keeps still, [right, up] in picture heights from the centre (default [0, 0]). Requires zoom." }, + "pan": { "type": "array", "items": { "type": "number" }, "minItems": 2, "maxItems": 2, "description": "Moves the target [right, up] in model radii, so the model moves the other way (default [0, 0]); each held within 1000." } + }, "additionalProperties": false } }), &["doc", "page"])), t("object_list", "List editable objects", "List existing paragraphs, Image/Form artwork and added content once, with source kind/index references, displayed rectangles and document generation. Indexes change when content changes; use generation in object_move to reject stale references.") .ro().with(schema(json!({"doc":doc(),"page":{"type":"integer","minimum":1}}), &["doc","page"])), - t("object_move", "Move objects together", "Atomically translate 1–1000 mixed references from object_list by [dx, dy] in displayed points (right/down). Preserves glyph codes, fonts, image data and relative spacing; success is one undo step. Text used as a clipping path and vertical text are refused. A bad/stale reference refuses the entire move.") + t("object_move", "Move objects together", "Atomically translate 1–1000 mixed references from object_list (or with all: true every object on the page) by [dx, dy] in displayed points (right/down). Preserves glyph codes, fonts, image data and relative spacing; success is one undo step. Text used as a clipping path and vertical text are refused. A bad/stale reference refuses the entire move.") .with(schema(json!({"doc":doc(),"page":{"type":"integer","minimum":1}, "objects":{"type":"array","minItems":1,"maxItems":1000,"uniqueItems":true,"items":{ "type":"object","properties":{"kind":{"type":"string","enum":["added","text","image"]},"index":{"type":"integer","minimum":1}},"required":["kind","index"],"additionalProperties":false}}, + "all":{"type":"boolean","description":"Every object on the page, instead of objects."}, "offset":{"type":"array","items":{"type":"number"},"minItems":2,"maxItems":2}, - "generation":{"type":"integer","minimum":0,"description":"Use generation returned by object_list to reject a stale selection."}}), &["doc","page","objects","offset"])), + "generation":{"type":"integer","minimum":0,"description":"Use generation returned by object_list to reject a stale selection."}}), &["doc","page","offset"])), + t("object_copy", "Copy or cut objects together", "Edit ▸ Copy (cut: Cut) of 1–1000 mixed references from object_list, or with all: true every object on the page, as ⌘A then ⌘C select and copy them in Edit a PDF. The copy keeps their content operators and every resource they use (fonts, images, ExtGStates, colour spaces, patterns, shadings) and is kept for object_paste into any open document, also after this one changes or closes. cut then deletes them as one undo step. Refused when the document doesn't allow copying (cut: or changes), or when text used as a clipping path is drawn before them. Returns the count copied and their text.") + .with(schema(json!({"doc":doc(),"page":{"type":"integer","minimum":1}, + "objects":{"type":"array","minItems":1,"maxItems":1000,"uniqueItems":true,"items":{ + "type":"object","properties":{"kind":{"type":"string","enum":["added","text","image"]},"index":{"type":"integer","minimum":1}},"required":["kind","index"],"additionalProperties":false}}, + "all":{"type":"boolean","description":"Every object on the page, instead of objects."}, + "cut":{"type":"boolean","description":"Delete them after copying (one undo step)."}, + "generation":{"type":"integer","minimum":0,"description":"Use generation returned by object_list to reject a stale selection."}}), &["doc","page"])), + t("object_paste", "Paste copied objects", "Edit ▸ Paste of what object_copy copied, onto page of doc as page content that draws the same: where it was, or on the page it came from 18 points down and to the right, and 18 more for each further paste onto the same page; centred when that would leave it off the page. Resources join the page's under names that don't clash; into another document the fonts, images and other objects they use are copied (each once, sharing what is there; refused past 20,000 objects or 256 MB). Pasted content is untagged and in no layer. One undo step.") + .with(schema(json!({"doc":doc(),"page":{"type":"integer","minimum":1}}), &["doc","page"])), t("text_word_count", "Word count", "Count Unicode words and characters on selected pages (default all), with totals and per-page counts. Table/code exclusions use PDF structure tags, not visual guesses; unclassified text is retained. Image exclusion removes Figure-tagged and invisible/OCR text. Does not run OCR. Copy permission is required. Fails rather than returning a partial total when a page cannot be read.") .ro() .cmd("file.word_count") .with(schema(json!({ "doc": doc(), "pages": pages("to count (default: all)"), "exclude_tables": { "type": "boolean" }, "exclude_code": { "type": "boolean" }, "exclude_images": { "type": "boolean" } }), &["doc"])), - t("text_extract", "Extract text", "Extract the text of some or all pages, in reading order. With rect, only the text inside that rectangle on each page, as Column select takes it: one row per visual line, side-by-side pieces (table cells) separated by a tab. With format \"html\", each page also carries html, the same text as Copy with formatting puts on the clipboard (paragraphs, bold, italic, size, colour and font family), and styles_incomplete when some styles had to be left out. With rect, the html is that text in line order, not split into the column's rows and cells. The HTML of one call is capped at 32 MiB in total; pages past the cap carry empty html with styles_incomplete. html_truncated is set when that cap is reached or when a page's own copy was cut at its span or size cap.") + t("text_extract", "Extract text", "Extract the text of some or all pages, in reading order. With rect, only the text inside that rectangle on each page, as Column select takes it: one row per visual line, side-by-side pieces (table cells) separated by a tab. With format \"html\", each page also carries html, the same text as Copy with formatting puts on the clipboard (paragraphs, bold, italic, size, colour and font family), and styles_incomplete when some styles had to be left out. With rect, the html is that text in line order, not split into the column's rows and cells. A page whose content was cut at the renderer's limits carries truncated: true; its text is incomplete. The HTML of one call is capped at 32 MiB in total; pages past the cap carry empty html with styles_incomplete. html_truncated is set when that cap is reached or when a page's own copy was cut at its span or size cap.") .ro() .cmd("edit.column_select") .with(schema( @@ -227,12 +269,13 @@ pub fn tools() -> Vec { }), &["doc"], )), - t("text_find", "Find text", "Find a phrase (case-insensitive, whitespace-normalised) and return matches in page order with page and line rectangles in points (origin top-left). Reads growing batches, looking one match past the limit for a continuation. Failed pages do not discard readable matches: warnings contains up to 100 {page, error, error_truncated} entries (errors capped at 1024 UTF-8 bytes), warning_count counts failed pages, and warnings_truncated reports a capped list. complete is false after a page failure or when more matches exist; next is present only for a match-limit continuation.") + t("text_find", "Find text", "Find a phrase (case-insensitive, whitespace-normalised) and return matches in page order with page and line rectangles in points (origin top-left). Reads growing batches, looking one match past the limit for a continuation. Failed pages do not discard readable matches: warnings contains up to 100 {page, error, error_truncated} entries (errors capped at 1024 UTF-8 bytes), warning_count counts failed pages, and warnings_truncated reports a capped list. complete is false after a page failure or when more matches exist; next is present only for a match-limit continuation. With stop_at_limit the search ends at the limit-th match instead of looking for one more, so a lone early match doesn't read the rest of the document.") .ro() .cmd("edit.find") .with(schema(json!({ "doc": doc(), "query": { "type": "string", "minLength": 1 }, "limit": { "type": "integer", "minimum": 1, "description": "Maximum matches (default 500). When another match exists, next holds start_page and skip to continue. complete is false for either additional matches or failed pages; an exhausted search with warnings has no next." }, "start_page": { "type": "integer", "minimum": 1, "description": "Page to start searching from (default 1)." }, - "skip": { "type": "integer", "minimum": 0, "description": "Matches to pass over on start_page before returning any (default 0)." } }), &["doc", "query"])), + "skip": { "type": "integer", "minimum": 0, "description": "Matches to pass over on start_page before returning any (default 0)." }, + "stop_at_limit": { "type": "boolean", "description": "Stop at the limit-th match without looking for another (default false). The answer then has stopped_at_limit; when true, complete is false and next is only where to resume, so following it may find nothing more." } }), &["doc", "query"])), t("page_rotate", "Rotate pages", "Rotate pages by a multiple of 90 degrees (positive is clockwise): the listed pages (default all), filtered like Acrobat's Rotate Pages by subset (all, even, odd page numbers) and orientation (all, landscape, portrait). Undoable.") .cmd("page.rotate") .with(schema( @@ -477,15 +520,29 @@ pub fn tools() -> Vec { }), &["doc", "from", "to"], )), - t("form_fields", "List form fields", "Every interactive form field: name, type (text, checkbox, radio, combo, list, button, signature), value, options, page, rect (top-left-origin points) and rotation (0, 90, 180 or 270 degrees counterclockwise), read-only and required flags. A calculated field includes calculate: {op, fields} or {notation} as form_set_props takes them, or {script} for a custom Calculate script (set with form_set_script). Push buttons include caption (first widget's normal label) and captions [{widget, caption}] for all widgets (1-based).") + t("form_fields", "List form fields", "Every interactive form field: name, type (text, checkbox, radio, combo, list, button, signature), value, options, page, rect (top-left-origin points) and rotation (0, 90, 180 or 270 degrees counterclockwise), read-only and required flags. A calculated field includes calculate: {op, fields} or {notation} as form_set_props takes them, or {script} for a custom Calculate script (set with form_set_script). Push buttons include caption (first widget's normal label) and captions [{widget, caption}] for all widgets (1-based). Radio groups include on_widgets, the 1-based numbers of the buttons that are on (buttons sharing an option tell apart only there). A check box whose widgets have different on states also lists them as options, and which one is checked (null when none). Combo and list boxes include selected: the 1-based positions in options of the selected ones, which tells apart options that share an export value.") .ro() .with(schema(json!({ "doc": doc() }), &["doc"])), t( "form_fill", "Fill in form fields", - "Set several fields at once (one undo step). values maps field names (from form_fields) to: a string for text fields, combo boxes and radio groups (an option), true/false for check boxes, an array of strings for multi-select lists. Appearances are regenerated so every viewer shows the values. A value whose characters the field's font can't draw (Chinese in a Helvetica field) is still saved; `warnings` in the result names the field and character: PdfCraft draws `?` there and asks other viewers to redraw the field from its value.", + "Set several fields at once (one undo step). values maps field names (from form_fields) to: a string for text fields, combo boxes and radio groups (an option), true/false for check boxes (or one of a check box's options, to check that widget and clear the others), an array of strings for multi-select lists; {\"widget\": n} turns on button n (1-based) of a check box or radio group and never turns it off (to clear, use null for a radio group that allows it, false for a check box); of radio buttons sharing an option only that one is on unless the group has RadiosInUnison. A radio option is an export value as form_fields lists it in options; where one button's on-state name equals another button's export value, the export value wins, and an on-state name that no export value matches still selects its button. A radio option may also be a JSON number: a whole number names the option spelled as that integer (1, 1.0 and 1e0 all name \"1\"), any other number only an option spelled exactly as JSON prints it (1.5). A value that names no option is refused with the options listed. Appearances are regenerated so every viewer shows the values. Thai the field's font can't hold is drawn from PdfCraft's built-in Thai face (a build without one refuses it). Any other value whose characters the field's font can't draw (Chinese in a Helvetica field) is still saved; `warnings` in the result names the field and character: PdfCraft draws `?` there and asks other viewers to redraw the field from its value.", ) .with(schema(json!({ "doc": doc(), "values": { "type": "object", "description": "Field name → value." } }), &["doc", "values"])), + t( + "form_focus", + "Move the focus into or out of a field", + "A field gains (event enter) or loses (event exit) focus, as when someone clicks or tabs into or out of it, and the dynamic XFA form's enter or exit scripts for it run (other forms have none, and nothing happens). widget picks the radio button or copy of the field (1-based, default 1; see form_fields). What the scripts change is one undoable edit, and none when they change nothing. Changing a field's value (form_fill) runs its exit scripts already, so the next exit of that field doesn't run them again. Returns changed, and the scripts' alerts, console output, requests and errors.", + ) + .with(schema( + json!({ + "doc": doc(), + "field": { "type": "string", "description": "Field name (from form_fields)." }, + "event": { "type": "string", "enum": ["enter", "exit"] }, + "widget": { "type": "integer", "minimum": 1, "description": "Widget of the field (1-based, default 1): a radio button of the group, or a copy of the field." }, + }), + &["doc", "field", "event"], + )), t("form_reset", "Clear form", "Reset fields to their default values: all of them, or only those listed. Undoable.") .destructive() .with(schema(json!({ "doc": doc(), "fields": { "type": "array", "items": { "type": "string" } } }), &["doc"])), @@ -532,7 +589,7 @@ pub fn tools() -> Vec { }), &["doc", "page", "rect", "rows"], )), - t("stamp_custom", "Add a custom stamp", "Stamp a picture: a PDF page (file_page, default 1, kept as vector content in a form XObject) or an image file. Either centred at at [x, y] (points from the top-left of the displayed page) at its natural size, at most 200 pt, or fitted inside rect [x0, y0, x1, y1] (the same coordinates; each side 1–14400 pt), scaled to keep its shape and centred in the box. Like at, rect is not clipped to the page. Pass at or rect, not both. name labels it (default: the file name). Undoable.") + t("stamp_custom", "Add a custom stamp", "Stamp a picture: a PDF page (file_page, default 1, kept as vector content in a form XObject) or an image file. Either centred at at [x, y] (points from the top-left of the displayed page) at its natural size, at most 200 pt, or fitted inside rect [x0, y0, x1, y1] (the same coordinates; each side 1–14400 pt), scaled to keep its shape and centred in the box. Like at, rect is not clipped to the page. Pass at or rect, not both. name labels it (default: the file name). line adds a text line under the picture with {name} (the author), {date} and {time} filled in now, as a dynamic stamp. Undoable.") .cmd("comment.stamp") .with(schema( json!({ @@ -544,6 +601,7 @@ pub fn tools() -> Vec { "file_page": { "type": "integer", "minimum": 1 }, "name": { "type": "string" }, "author": { "type": "string" }, + "line": { "type": "string", "maxLength": 200 }, }), &["doc", "page", "path"], )), @@ -566,7 +624,7 @@ pub fn tools() -> Vec { t( "form_set_props", "Field properties", - "Change field properties: give field for one field, or fields for a list of unique field names (1–1000). A bulk change is atomic and one Undo step; an invalid or locked field leaves every field unchanged. Only the given properties change, and partial appearance changes preserve each field's other colours and font. name (rename: a dotted name such as MASTER.first is the full name and moves the field under those parent fields; a plain name renames it within its parent), rect (position), rotation and caption require a single field. A rename also renames the field in the calculations that read it (calculate field lists and notation, and getField calls naming it in a string literal; names built at run time are not rewritten), in the same Undo step, and formulas_updated lists the fields whose calculation changed. caption changes one push-button normal label (widget: 1-based index, default 1); an empty string clears it. Other widgets and icon-only layouts are preserved. General and Options: tooltip, read_only, required, multiline, max_length (0 = no limit), options (combo/list items), font_size (0 = auto), format, validate, calculate (Acrobat's native AF rules). Options: align (left, center, right), default (Reset form's value; check boxes/radio buttons use their on state, \"\" for off), flags { scroll, rich_text, password, file_select, spell_check, comb (needs max_length), sort, editable, multi_select, commit_immediately }, check_style (check boxes/radio buttons: check, circle, cross, diamond, square, star). rotation is 0, 90, 180 or 270 degrees counterclockwise and swaps width/height when the axis changes. Undoable.", + "Change field properties: give field for one field, or fields for a list of unique field names (1–1000). A bulk change is atomic and one Undo step; an invalid or locked field leaves every field unchanged. Only the given properties change, and partial appearance changes preserve each field's other colours and font. name (rename: a dotted name such as MASTER.first is the full name and moves the field under those parent fields; a plain name renames it within its parent), rect (position), rotation and caption require a single field. A rename also renames the field in the calculations that read it (calculate field lists and notation, and getField calls naming it in a string literal; names built at run time are not rewritten), in the same Undo step, and formulas_updated lists the fields whose calculation changed. caption changes one push-button normal label (widget: 1-based index, default 1); an empty string clears it. Other widgets and icon-only layouts are preserved. General and Options: tooltip, read_only, required, multiline, max_length (0 = no limit), options (combo/list items), font_size (0 = auto), format, validate, calculate (Acrobat's native AF rules). Options: align (left, center, right), default (Reset form's value; check boxes/radio buttons use their on state, \"\" for off), flags { scroll, rich_text, password, file_select, spell_check, comb (needs max_length), sort, editable, multi_select, commit_immediately }, check_style (check boxes/radio buttons: check, circle, cross, diamond, square, star). rotation is 0, 90, 180 or 270 degrees counterclockwise; the field keeps its rectangle and its content turns inside it. Undoable.", ) .with(schema( json!({ @@ -590,7 +648,7 @@ pub fn tools() -> Vec { "flags": { "type": "object", "additionalProperties": { "type": "boolean" } }, "font_size": { "type": "number", "minimum": 0 }, "rect": { "type": "array", "items": { "type": "number" }, "minItems": 4, "maxItems": 4, "description": "Move/resize: points from the top-left of the displayed page." }, - "rotation": { "type": "integer", "enum": [0, 90, 180, 270], "description": "Widget rotation in degrees, counterclockwise. Changing between horizontal and vertical swaps the rectangle around its center." }, + "rotation": { "type": "integer", "enum": [0, 90, 180, 270], "description": "Widget rotation in degrees, counterclockwise. The field keeps its rectangle; its content turns inside it." }, "appearance": { "type": "object", "description": "Appearance tab: border, fill, text_color (#RRGGBB, a name or \"none\"), width (0–12 points; 1 thin, 2 medium, 3 thick), style (solid, dashed, beveled, inset, underline), font (helvetica, times, courier)." }, "format": { "description": "Format tab: {\"type\": \"number\", \"decimals\": 2, \"currency\": \"$\", \"separator\": 0-4, \"negative\": 0-3}, {\"type\": \"percent\"}, {\"type\": \"date\"|\"time\", \"pattern\": \"mm/dd/yyyy\"}, {\"type\": \"zip\"|\"zip4\"|\"phone\"|\"ssn\"}, {\"type\": \"mask\", \"mask\": \"AA-9999\"}, or \"none\"." }, "validate": { "description": "Validate tab: {\"min\": 0, \"max\": 100} (either may be left out, or null) or \"none\"." }, @@ -647,7 +705,7 @@ pub fn tools() -> Vec { t( "redact_apply", "Apply redactions", - "Apply the redaction marks (all, or those on pages): text, images, vectors, comments and form fields under them are removed for good and boxes are drawn in their place. A verification pass fails the operation if anything readable remains. Undoable until saved; the saved file no longer contains the content.", + "Apply the redaction marks (all, or those on pages): text, images, vectors, comments and form fields under them are removed for good and boxes are drawn in their place. A verification pass fails the operation if anything readable remains. Document metadata (Info entries, the document's and pages' XMP) that repeats the removed text is removed too; when the document XMP goes, identification says what became of its PDF/A and PDF/UA claims (pdfa, pdfua: none, kept when written back in a packet holding only the identification, or lost). still_holding counts the places applying leaves as they are that still hold the removed text: bookmarks (titles), attachments (names, file names, descriptions), comments (text, rich text, subject or author of those that remain) and xmp (XMP on images, fonts, forms, annotations and other objects), with their total; doc_remove_hidden removes them. unsearched: true means the removed text couldn't be searched for, so every such place holding any text is counted. Undoable until saved; the saved file no longer contains the content. The result's save_as is the document's name with _Redacted before the extension (contract.pdf gives contract_Redacted.pdf; a name that already ends in _Redacted is unchanged); pass it to doc_save as path to save under that name.", ) .destructive() .with(schema(json!({ "doc": doc(), "pages": pages("whose marks to apply (default: all)") }), &["doc"])), @@ -688,7 +746,7 @@ pub fn tools() -> Vec { t( "doc_print", "Print", - "Print with Acrobat's Print dialog options, or save the print-ready PDF. pages: a range such as \"1-3, 6, 9-\" (page labels allowed; default all); subset odd/even (by page number, as Acrobat counts them: pages 2, 7-10 with even prints 2, 8 and 10, not every second page of the range); reverse. layout: fit (default), actual, shrink, custom (scale %), multiple (per_sheet 2/4/6/9/16, order, border, auto_rotate; cut-stack order arranges single-sided sheets for cutting into piles and stacking left to right, top to bottom, keeping sheet order within each pile; duplex must be off), booklet (booklet_subset both/front/back, binding left/right; a booklet sets two-sided itself, as Acrobat does: both flips on the short edge, front or back prints one-sided), poster (scale %, overlap pt, cut_marks). orientation auto/portrait/landscape; comments_forms document / document-and-markups (default) / document-and-stamps / form-fields-only; paper Letter/Legal/Tabloid, ARCH A/B/C/D/E1/E, ANSI C/D/E, A0/A1/A2/A3/A4/A5/A6, or US #10 Envelope (short names or full preset labels); use layout actual for 100% in the print-ready PDF; printer drivers may still substitute paper or scale. Then path (save) or printer (a name or \"default\") with copies, collate, duplex off/long-edge/short-edge, grayscale, and options (driver options from printer_options, CUPS).", + "Print with Acrobat's Print dialog options, or save the print-ready PDF. pages: a range such as \"1-3, 6, 9-\" (page labels allowed, \"(n)\" for the nth page whatever its label, unless a page is labelled exactly \"(n)\", since labels are looked up first; default all); subset odd/even (by page number, as Acrobat counts them: pages 2, 7-10 with even prints 2, 8 and 10, not every second page of the range); reverse. layout: fit (default), actual, shrink, custom (scale %), multiple (per_sheet 2/4/6/9/16, which turns the sheet to suit the grid, or a custom grid of cols across by rows down, each 1-99, on the sheet as oriented (auto prints portrait); order, border, auto_rotate; cut-stack order arranges single-sided sheets for cutting into piles and stacking left to right, top to bottom, keeping sheet order within each pile; duplex must be off), booklet (booklet_subset both/front/back, binding left/right; a booklet sets two-sided itself, as Acrobat does: both flips on the short edge, front or back prints one-sided), poster (scale %, overlap pt, cut_marks). orientation auto/portrait/landscape; comments_forms document / document-and-markups (default) / document-and-stamps / form-fields-only; paper Letter/Legal/Tabloid, ARCH A/B/C/D/E1/E, ANSI C/D/E, A0/A1/A2/A3/A4/A5/A6, or US #10 Envelope (short names or full preset labels); use layout actual for 100% in the print-ready PDF; printer drivers may still substitute paper or scale. Then path (save) or printer (a name or \"default\") with copies, collate, duplex off/long-edge/short-edge, grayscale, and options (driver options from printer_options, CUPS).", ) .with(schema( json!({ @@ -699,6 +757,8 @@ pub fn tools() -> Vec { "layout": { "type": "string", "enum": ["fit", "actual", "shrink", "custom", "multiple", "booklet", "poster"] }, "scale": { "type": "number", "exclusiveMinimum": 0 }, "per_sheet": { "type": "integer", "minimum": 1, "maximum": 256 }, + "cols": { "type": "integer", "minimum": 1, "maximum": 99, "description": "Custom pages per sheet: columns across the sheet; give rows too, and no per_sheet." }, + "rows": { "type": "integer", "minimum": 1, "maximum": 99, "description": "Custom pages per sheet: rows down the sheet; give cols too, and no per_sheet." }, "order": { "type": "string", "enum": ["horizontal", "horizontal-reversed", "vertical", "vertical-reversed", "cut-stack"] }, "border": { "type": "boolean" }, "auto_rotate": { "type": "boolean" }, @@ -791,30 +851,32 @@ pub fn tools() -> Vec { t("content_delete", "Delete added content", "Delete an added item (page, index from content_list). Undoable.") .destructive() .with(schema(json!({ "doc": doc(), "page": { "type": "integer", "minimum": 1 }, "index": { "type": "integer", "minimum": 1 } }), &["doc", "page", "index"])), - t("link_list", "List links", "Every link: page, 1-based index (for link_edit/link_delete), rect (points from the top-left of the page), and where it goes (url or to_page).") + t("link_list", "List links", "Every link: page, 1-based index (for link_edit/link_delete), rect (points from the top-left of the page), and where it goes (url, or to_page with its zoom). A relative url also has resolved_url, the address resolved against the document's base URI (catalog /URI /Base); a link active only in its /QuadPoints has active_areas, one rect per quadrilateral. With text_links: true, also the web and email addresses written as plain page text with no link over them, which the viewer opens on a click: page, rects (one per line), url, and blocked (why it won't open) when the link check refuses it. Only http, https and mailto addresses are found. At most 100 pages are read per call, from text_links_from (default 1); text_links_next_page then gives the page to continue from.") .ro() - .with(schema(json!({ "doc": doc() }), &["doc"])), + .with(schema(json!({ "doc": doc(), "text_links": { "type": "boolean" }, "text_links_from": { "type": "integer", "minimum": 1 } }), &["doc"])), t("link_open", "Open a linked PDF", "Follow a local GoToR link in PdfCraft: open the referenced PDF as a new document, respecting --root. page and index use link_list numbering. Named destinations are reported but not applied. Absolute file specifications require allow_absolute: true after reviewing the target; relative links need no opt-in.") .with(schema( json!({ "doc": doc(), "page": { "type": "integer", "minimum": 1 }, "index": { "type": "integer", "minimum": 1 }, "password": { "type": "string", "description": "Optional password for the linked PDF." }, "allow_absolute": { "type": "boolean", "description": "Explicitly allow an absolute local GoToR target (default false). Existing validation and --root confinement still apply." } }), &["doc", "page", "index"], )), - t("link_add", "Add a link", "Add a link over rect on page that opens url or goes to to_page. Appearance: visible (rectangle), color, width 1-3, highlight none/invert/outline/inset. Undoable.") + t("link_add", "Add a link", "Add a link over rect on page that opens url or goes to to_page, shown at zoom: fit_page (default), fit_width, fit_height, fit_visible, actual, inherit (keep the reader's zoom) or a percentage. Appearance: visible (rectangle), color, width 1-3, highlight none/invert/outline/inset. Undoable.") .cmd("edit.link") .with(schema( json!({ "doc": doc(), "page": { "type": "integer", "minimum": 1 }, "rect": { "type": "array", "items": { "type": "number" }, "minItems": 4, "maxItems": 4 }, "url": { "type": "string" }, "to_page": { "type": "integer", "minimum": 1 }, + "zoom": { "description": "fit_page, fit_width, fit_height, fit_visible, actual, inherit, or a percentage" }, "visible": { "type": "boolean" }, "color": { "type": "string" }, "width": { "type": "number" }, "highlight": { "type": "string", "enum": ["none", "invert", "outline", "inset"] }, }), &["doc", "page", "rect"], )), - t("link_edit", "Edit a link", "Change a link's rect, destination (url or to_page) or appearance. Undoable.") + t("link_edit", "Edit a link", "Change a link's rect, destination (url, or to_page and zoom as in link_add; a new to_page keeps the link's zoom, a zoom alone keeps its page) or appearance. Undoable.") .with(schema( json!({ "doc": doc(), "page": { "type": "integer", "minimum": 1 }, "index": { "type": "integer", "minimum": 1 }, "rect": { "type": "array", "items": { "type": "number" }, "minItems": 4, "maxItems": 4 }, "url": { "type": "string" }, "to_page": { "type": "integer", "minimum": 1 }, + "zoom": { "description": "fit_page, fit_width, fit_height, fit_visible, actual, inherit, or a percentage" }, "visible": { "type": "boolean" }, "color": { "type": "string" }, "width": { "type": "number" }, "highlight": { "type": "string", "enum": ["none", "invert", "outline", "inset"] }, }), &["doc", "page", "index"], @@ -822,7 +884,7 @@ pub fn tools() -> Vec { t("link_delete", "Delete a link", "Delete one link (page, index from link_list). Undoable.") .destructive() .with(schema(json!({ "doc": doc(), "page": { "type": "integer", "minimum": 1 }, "index": { "type": "integer", "minimum": 1 } }), &["doc", "page", "index"])), - t("links_from_urls", "Create links from URLs", "Find web addresses (http://, https://, www.) in the text of every page and make them clickable links. Returns the URLs. Undoable.") + t("links_from_urls", "Create links from URLs", "Find web addresses (http://, https://, www.) in the text of every page and make them clickable links, leaving out text on a layer the viewer hides (layer_set). Returns the URLs. Undoable.") .cmd("edit.links_from_urls") .with(schema(json!({ "doc": doc() }), &["doc"])), t("links_remove", "Remove all links", "Remove every link in the document (or on the given pages). Undoable.") @@ -845,7 +907,7 @@ pub fn tools() -> Vec { .ro().cmd("measure.snap").with(schema(json!({"doc":doc(),"page":{"type":"integer","minimum":1},"at":point(),"tolerance":{"type":"number","minimum":0,"maximum":10000},"endpoints":{"type":"boolean"},"midpoints":{"type":"boolean"},"intersections":{"type":"boolean"},"paths":{"type":"boolean"}}), &["doc","page","at"])), t("measure_export", "Export measurements as CSV", "Atomically write saved measurement values, labels, authors and scale ratios as spreadsheet-safe CSV. Returns how many unsupported measurements were left out.") .cmd("measure.export").with(schema(json!({"doc":doc(),"out":path_arg()}), &["doc","out"])), - t("comment_list", "List comments", "Every comment (annotation other than links, form widgets and pop-ups) with its page, index, id, type, author, text, date, rectangle, colour, review status, checkmark and replies (each with its own status and checkmark).") + t("comment_list", "List comments", "Every comment (annotation other than links, form widgets and pop-ups) with its page, index, id, type, author, text, date, rectangle, colour, review status, checkmark and replies (each with its own status and checkmark); the points that shape a line, polygon or callout, and a callout's text box. Annotations grouped with a comment (/RT /Group) are listed once, as that comment with its attributes, and named in its `group`. comment_delete deletes just the member named, as Acrobat does (Replace Text's strikeout and caret go together).") .ro() .with(schema(json!({ "doc": doc(), "page": { "type": "integer", "minimum": 1, "description": "Only this page." } }), &["doc"])), t( @@ -854,7 +916,7 @@ pub fn tools() -> Vec { "Add a comment as Acrobat's commenting tools do. Geometry is in points with the origin at the top-left of the displayed page, y down (as in page_render images at 72 dpi and text_find rects). \ note: `at` [x, y] (icon top-left). stamp: `at` [x, y] (its centre) and `stamp`: approved, completed, confidential, draft, final, for comment, for public release, information only, not approved, not for public release, preliminary results, void, accepted, initial here, rejected, sign here, witness; dynamic: true for the dynamic approved/confidential/received/reviewed/revised stamps with a By … at … line. highlight/underline/strikeout/squiggly: `find` (text on the page to mark; every match with all: true) or `quads`. \ rectangle/oval/textbox: `rect` [x0, y0, x1, y1]. line/arrow: `from`, `to`. Optional `endings` (two names: start, end) for a line, arrow or polyline, or one name for a callout: None, Square, Circle, Diamond, OpenArrow, ClosedArrow, Butt, ROpenArrow, RClosedArrow, Slash. An arrow without endings is None then OpenArrow. ink: `strokes` [[[x, y], …], …]. \ - polygon/cloud/polyline (connected lines): `points` [[x, y], …]. callout: `rect` (its text box), `to` (the point the arrow touches), optional `knee`. caret (inserted text): `at`, the insertion point on the baseline. replace (Replace Text): `find` or `quads` like highlight, `contents` the replacement. attachment: `path` (the file), `at`, optional `icon` (PushPin, Paperclip, Graph, Tag). Undoable. Text a text box, callout or stamp draws in the standard fonts that they can't draw (Chinese, Greek) is saved as typed and drawn as `?`; `warnings` in the result says so.", + polygon/cloud/polyline (connected lines): `points` [[x, y], …]. callout: `rect` (its text box), `to` (the point the arrow touches), optional `knee`. caret (inserted text): `at`, the insertion point on the baseline. replace (Replace Text): `find` or `quads` like highlight, `contents` the replacement. attachment: `path` (the file), `at`, optional `icon` (PushPin, Paperclip, Graph, Tag). Undoable. Thai in a text box or callout is drawn from PdfCraft's built-in Thai face (a build without one refuses it). Other text a text box, callout or stamp draws in the standard fonts that they can't draw (Chinese, Greek) is saved as typed and drawn as `?`; `warnings` in the result says so.", ) .with(schema( json!({ @@ -923,7 +985,7 @@ pub fn tools() -> Vec { .ro() .cmd("sign.digital") .with(schema(json!({}), &[])), - t("sign_document", "Sign a document", "Sign with a digital ID (a .p12/.pfx path, or on macOS a Keychain identity: \"keychain:\" from sign_keychain_ids, or on Windows a store identity: \"windows:\" from sign_windows_ids) and save the signed file to `out` (signing always saves, as in Acrobat; the document then shows the signed file). Sign an existing empty signature field (`field`), or a new one on `page` at `rect` (omit rect for an invisible signature). certify: no_changes, form_fill or comments makes a certification signature. graphic shows a picture where the signer's name would be large (Acrobat: Configure Signature Appearance): {\"image\": path} (PNG or JPEG), {\"text\": name} (typed in the Fill & Sign script font) or {\"strokes\": [[[x, y], …], …]} (y up, the drawing pad 1 wide); it needs a visible signature. appearance chooses what a visible signature shows (default: name, date and labels; reason, location and distinguished_name are off unless turned on). labels gives the words of its labels, e.g. in another language (default English: \"Digitally signed by\", \"DN: \", \"Reason: \", \"Location: \", \"Date: \"; include the separator); each must be in Western European characters (Windows-1252), the only ones the signature's font draws, or the call is refused naming the label and character. PAdES B-B, SHA-256 (SHA-384 for P-384 keys).") + t("sign_document", "Sign a document", "Sign with a digital ID (a .p12/.pfx path, or on macOS a Keychain identity: \"keychain:\" from sign_keychain_ids, or on Windows a store identity: \"windows:\" from sign_windows_ids) and save the signed file to `out` (signing always saves, as in Acrobat; the document then shows the signed file). Sign an existing empty signature field (`field`), or a new one on `page` at `rect` (omit rect for an invisible signature). certify: no_changes, form_fill or comments makes a certification signature. graphic shows a picture where the signer's name would be large (Acrobat: Configure Signature Appearance): {\"image\": path} (PNG or JPEG), {\"text\": name} (typed in the Fill & Sign script font) or {\"strokes\": [[[x, y], …], …]} (y up, the drawing pad 1 wide); it needs a visible signature. appearance chooses what a visible signature shows (default: name, date and labels; reason, location and distinguished_name are off unless turned on). labels gives the words of its labels, e.g. in another language (default English: \"Digitally signed by\", \"DN: \", \"Reason: \", \"Location: \", \"Date: \"; include the separator); each must be in Western European characters (Windows-1252), the only ones the signature's font draws, or the call is refused naming the label and character. PAdES B-B, SHA-256 (SHA-384 for P-384 keys). On a dynamic (XFA) form, run_form_scripts: true runs the field's own signing scripts: preSign just before signing, so what it changes is signed too (form_scripts.changed lists each field it changed, with the values before and after), and postSign after, left unsaved. Without it they don't run, and form_scripts.skipped says the field has them.") .cmd("sign.digital") .with(schema( json!({ @@ -974,6 +1036,7 @@ pub fn tools() -> Vec { "additionalProperties": false, }, "out": { "type": "string", "description": "Where to save the signed document." }, + "run_form_scripts": { "type": "boolean", "description": "Run a dynamic form's preSign and postSign scripts for the field (default false)." }, }), &["doc", "id", "out"], )), @@ -1016,7 +1079,7 @@ pub fn tools() -> Vec { json!({ "doc": doc(), "sort": { "type": "string", "enum": ["page", "author", "date", "type"] }, "out": save_out, "open": open }), &["doc"], )), - t("comment_edit", "Edit a comment", "Change a comment's text, colour, opacity (the whole comment's, or its fill's and text's own), line width, font size (text box/typewriter/callout), fill (rectangle/oval/polygon/text box/callout; \"none\" removes it), border colour (`border_color`, text box/typewriter/callout; \"text\" gives it the text's colour again), rectangle (rectangle/oval/text box/stamp), position (`move` [dx, dy] in points) or shape (`points`: a line's two ends, or the corners of connected lines, a polygon or a cloud). Stamps keep their original appearance when resized. One undo step.").with(schema( + t("comment_edit", "Edit a comment", "Change a comment's text, colour, opacity (the whole comment's, or its fill's and text's own), line width, font size (text box/typewriter/callout), fill (rectangle/oval/polygon/text box/callout; \"none\" removes it), border colour (`border_color`, text box/typewriter/callout; \"text\" gives it the text's colour again), rectangle (rectangle/oval/text box/stamp, or a callout's text box: its arrow point stays and its leader line re-attaches), position (`move` [dx, dy] in points) or shape (`points`: a line's two ends, the corners of connected lines, a polygon or a cloud, or a callout's arrow point and knee, as comment_list gives them). Stamps keep their original appearance when resized. One undo step.").with(schema( comment_ref(json!({ "contents": { "type": "string" }, "color": color(), @@ -1030,13 +1093,18 @@ pub fn tools() -> Vec { "font_size": { "type": "number", "minimum": 1, "maximum": 400, "description": "Font size in points of a text box, typewriter text or callout; the box grows to hold the text." }, "rect": { "type": "array", "items": { "type": "number" }, "minItems": 4, "maxItems": 4 }, "move": point(), - "points": { "type": "array", "items": point(), "minItems": 2, "description": "A line's two ends, or every corner of connected lines, a polygon or a cloud, in page coordinates." }, + "points": { "type": "array", "items": point(), "minItems": 1, "description": "A line's two ends, every corner of connected lines, a polygon or a cloud, or a callout's arrow point and (optionally) knee, in page coordinates. A callout given its arrow point alone keeps the knee it has." }, })), &["doc"], )), t("comment_image_preview", "Preview image signature layers", "Return a PNG of the page without the selected image signature/initials (layer=background, default), or its embedded image with alpha (layer=image). Includes the displayed rectangle, document rotation, image_rotation (how far clockwise the image is shown turned) and annotation opacity. Cache these layers for live placement/resizing; commit once with comment_edit. Does not change the document or undo history.") .ro().with(schema(comment_ref(json!({ "layer": { "type": "string", "enum": ["background", "image"], "default": "background" }, "dpi": { "type": "number", "minimum": 1, "maximum": 600, "default": 96 } })), &["doc"])), t("comment_delete", "Delete a comment", "Delete a comment with its pop-up and replies, or one reply (its id, or page and index, from comment_list) with its pop-up and the replies to it; the comment and its other replies stay. Undoable.").destructive().with(schema(comment_ref(json!({})), &["doc"])), + t("comment_popup", "Open, close or move a comment's pop-up", "Open or close a comment's pop-up window (open), or move it (rect, in page view coordinates as comment_list reports). A comment without a pop-up gets one beside it. Text boxes, Fill & Sign marks and replies have none of their own. Undoable.") + .with(schema(comment_ref(json!({ + "open": { "type": "boolean" }, + "rect": { "type": "array", "items": { "type": "number" }, "minItems": 4, "maxItems": 4 }, + })), &["doc"])), t( "comments_copy", "Copy comments", @@ -1125,7 +1193,7 @@ pub fn tools() -> Vec { t( "doc_header_footer", "Add header and footer", - "Add a header and/or footer to pages. Text boxes: header_left/center/right, footer_left/center/right. Tokens: <<1>> page number, <> page count, <<1 of n>>, <>, <<1/n>>, dates <> <> <> <> (and more), Bates <>. shrink: true scales the page content down evenly into the room the header and footer leave free, so they don't cover it. font: Helvetica (default), Times or Courier. replace: true swaps out existing ones (Update). Undoable.", + "Add a header and/or footer to pages. Text boxes: header_left/center/right, footer_left/center/right. Tokens: <<1>> page number, <> page count, <<1 of n>>, <>, <<1/n>>, dates <> <> <> <> (and more), Bates <>. shrink: true scales the page content down evenly into the room the header and footer leave free, so they don't cover it. font: Helvetica (default), Times or Courier; Thai is drawn from PdfCraft's built-in Thai face beside it (a build without one refuses it). replace: true swaps out existing ones (Update). Undoable.", ) .cmd("edit.header_footer") .with(schema( @@ -1144,7 +1212,7 @@ pub fn tools() -> Vec { }), &["doc"], )), - t("doc_watermark", "Add watermark", "Add a watermark to pages: text, or a picture from `file` (an image, or page `file_page` of a PDF) at `scale` of the page (default 0.5); rotated (degrees counter-clockwise, default 45), semi-transparent (opacity 0–1, default 0.5), text fitted to the page unless font_size is given, on top unless behind: true; moved from the page centre by `offset_x` (right) and `offset_y` (up) points (default 0). replace: true swaps out existing ones. Undoable.") + t("doc_watermark", "Add watermark", "Add a watermark to pages: text, or a picture from `file` (an image, or page `file_page` of a PDF) at `scale` of the page (default 0.5); rotated (degrees counter-clockwise, default 45), semi-transparent (opacity 0–1, default 0.5), text fitted to the page unless font_size is given (Thai drawn from PdfCraft's built-in Thai face; a build without one refuses it), on top unless behind: true; moved from the page centre by `offset_x` (right) and `offset_y` (up) points (default 0). replace: true swaps out existing ones. Undoable.") .cmd("edit.watermark") .with(schema( json!({ @@ -1171,13 +1239,27 @@ pub fn tools() -> Vec { json!({ "doc": doc(), "pages": pages("to fill (default: all)"), "color": { "type": "string" }, "file": { "type": "string" }, "file_page": { "type": "integer", "minimum": 1 }, "scale": { "type": "number", "exclusiveMinimum": 0, "maximum": 1 }, "opacity": { "type": "number", "minimum": 0, "maximum": 1 }, "replace": { "type": "boolean" }, "replace_backdrop": { "type": "boolean", "description": "Replace a white backdrop the page paints itself, which would hide the background. Default true." } }), &["doc"], )), + t("doc_printer_marks", "Add printer marks", "Print production: trim marks, bleed marks (dashed, at the bleed box), registration targets, colour bars and a page-information line (label, page n of N, date) drawn outside the trim box as a removable page mark, in registration colour (Separation All, so on every plate) except the process-ink colour bars; the page is widened by make_room points a side when the marks would fall outside it (the trim box stays). replace (default true) replaces marks added before. Undoable; doc_remove_marks kind printer_marks takes them off.") + .cmd("prepress.marks") + .with(schema( + json!({ + "doc": doc(), "pages": pages("the pages to mark (default all)"), + "trim": { "type": "boolean" }, "bleed": { "type": "boolean" }, "registration": { "type": "boolean" }, "color_bars": { "type": "boolean" }, "page_info": { "type": "boolean" }, + "line_weight": { "type": "number", "minimum": 0.1, "maximum": 4, "description": "Points (Acrobat offers 0.25, 0.5 and 1)." }, + "make_room": { "type": "number", "maximum": 288, "description": "Points to widen the page by on each side when needed (default 36); negative leaves the page as it is." }, + "label": { "type": "string", "description": "The page-information line's label (default: the file's name)." }, + "date": { "type": "string", "description": "The page-information line's date (default: today)." }, + "replace": { "type": "boolean" }, + }), + &["doc"], + )), t("doc_backdrops", "List page backdrops", "Pages that paint their own backdrop, which hides a background added behind the content: `fill` (an opaque white fill doc_background can replace) or `image` (a page-sized picture, e.g. a scan, that still hides it). Pages are 1-based; pages without one are left out.") .ro() .cmd("edit.background") .with(schema(json!({ "doc": doc(), "pages": pages("to check (default: all)") }), &["doc"])), - t("doc_remove_marks", "Remove header & footer, watermark or background", "Remove every header and footer, watermark or background PdfCraft (or a compatible tool) added. Undoable.") + t("doc_remove_marks", "Remove header & footer, watermark, background or printer marks", "Remove every header and footer, watermark, background or printer marks PdfCraft (or a compatible tool) added. Undoable.") .destructive() - .with(schema(json!({ "doc": doc(), "kind": { "type": "string", "enum": ["header_footer", "watermark", "background"] } }), &["doc", "kind"])), + .with(schema(json!({ "doc": doc(), "kind": { "type": "string", "enum": ["header_footer", "watermark", "background", "printer_marks"] } }), &["doc", "kind"])), t("doc_export_images", "Export pages as images", "Write pages as PNG, JPEG or TIFF files (`_page_.png|jpg|tif`) into a folder, at a resolution (default 150 dpi). JPEG and TIFF are flattened onto white paper. A page too large for the renderer at that resolution is drawn at the most it allows, listed in `lower_dpi` ({page, dpi}). Includes unsaved edits.") .cmd("export.image") .with(schema( @@ -1297,6 +1379,11 @@ pub fn tools() -> Vec { "doc": doc(), "object": { "type": "integer", "minimum": 1 }, "generation": { "type": "integer", "minimum": 0, "maximum": 65535 }, }), &["doc", "object", "generation"])), + t("accessibility_layout_table_to_blocks", "Change layout table to blocks", "Turn a table used only for layout (label and value pairs, a letterhead grid) into plain blocks in one step: the Table tag, its THead/TBody/TFoot and TR tags become Div, its TH, TD and Caption tags P (Div when they hold tags of their own). Content, reading order and the tags inside cells stay; a table nested in a cell stays a table; Table attributes (Summary, Scope, spans) are removed, other attributes kept. The tag's type must be Table after the role map; use object and generation from accessibility_tags. Undoable. Returns the tags as accessibility_tags does.") + .with(schema(json!({ + "doc": doc(), "object": { "type": "integer", "minimum": 1 }, + "generation": { "type": "integer", "minimum": 0, "maximum": 65535 }, + }), &["doc", "object", "generation"])), t("accessibility_move_tag", "Move structure tag", "Move a reachable indirect structure tag before or after a direct sibling in their parent's /K array, changing tagged reading order without changing page appearance. Use object and generation values from accessibility_tags. Rejects distinct parents, duplicate references and positions that are already current. Undoable.") .with(schema(json!({ "doc": doc(), "object": { "type": "integer", "minimum": 1 }, @@ -1308,7 +1395,7 @@ pub fn tools() -> Vec { t("form_merge_data", "Merge data files into spreadsheet", "Collect the field values of form data files (FDF, XFDF) or filled-in PDF forms into one CSV file at path: a column per field name, a row per file. Returns the row and column counts.") .cmd("form.merge_data") .with(schema(json!({ "paths": { "type": "array", "items": { "type": "string" }, "minItems": 1 }, "path": { "type": "string" } }), &["paths", "path"])), - t("js_run", "Run JavaScript", "Run Acrobat JavaScript in the document, as the JavaScript console does (or as push button `field`'s Mouse Up script when field is given; on a laid-out XFA form, `field` runs that button's XFA click script instead, JavaScript or FormCalc, which can add and remove rows and show or hide subforms). The form object model is available: this/getField, event, app, util, console, display, color, and the document-level scripts (except the PDF container scripts when XFA layout is active; they remain available through js_document_scripts). Field changes and resetForm are applied as one undoable step; returns the script's alerts (each with its message, title, icon and buttons, in order; every question is still answered OK), console output, requests (print, page, url, submit), error, and the errors of document-level scripts that failed (the rest still ran).") + t("js_run", "Run JavaScript", "Run Acrobat JavaScript in the document, as the JavaScript console does (or as push button `field`'s Mouse Up script when field is given; on a laid-out XFA form, `field` runs that button's XFA click script instead, JavaScript or FormCalc, which can add and remove rows and show or hide subforms). The form object model is available: this/getField, event, app, util, console, display, color, and the document-level scripts (except the PDF container scripts when XFA layout is active; they remain available through js_document_scripts). Field changes and resetForm are applied as one undoable step; returns the alerts (each with its message, title, icon and buttons, in order; every question is still answered OK), console output and requests (print, page, url, submit) of the script and of the field scripts its changes ran, the script's error, the errors of document-level scripts that failed (the rest still ran), and in warnings the errors of applying the changes, such as values of an XFA form that could not be written to its data.") .cmd("tools.js_console") .with(schema( json!({ "doc": doc(), "script": { "type": "string" }, "field": { "type": "string", "description": "Run as this button's Mouse Up event." } }), @@ -1337,10 +1424,10 @@ pub fn tools() -> Vec { t("print_readiness", "Print readiness", "Verification only. Reports effective image resolution where each image is drawn (not against the page size), page boxes and bleed past the trim, output intents, fonts that are not embedded, and the PDF/A-2b structural result. Does not modify the document, embed fonts, invent a press profile, or check PDF/X.") .ro() .with(schema(json!({ "doc": doc() }), &["doc"])), - t("pdfa_convert", "Save as PDF/A", "Standards ▸ Save as PDF/A: fix what can be fixed for PDF/A-2b or 3b (XMP identification and metadata, an sRGB output intent, forbidden actions and JavaScript, annotation print flags, image interpolation, encryption). Returns what was fixed and what remains (e.g. fonts that aren't embedded). Undoable; save the document to keep it.") + t("pdfa_convert", "Save as PDF/A", "Standards ▸ Save as PDF/A: fix what can be fixed for PDF/A-2b or 3b (XMP identification and metadata, an sRGB output intent, forbidden actions and JavaScript, annotation print flags, image interpolation, encryption, the file header, the file identifier). Returns what was fixed and what remains (e.g. fonts that aren't embedded). Undoable; save the document to keep it.") .cmd("standards.pdfa") .with(schema(json!({ "doc": doc(), "level": { "type": "string", "enum": ["2b", "3b"] } }), &["doc"])), - t("standards_remove_id", "Remove PDF/A or PDF/X identification", "Remove what identifies the document as PDF/A (its pdfaid: XMP properties) or PDF/X (GTS_PDFXVersion and GTS_PDFXConformance, in the document information and as pdfxid: XMP properties), e.g. after edits it no longer meets. Output intents and other metadata stay. Fails when the document declares nothing of the kind. Undoable.") + t("standards_remove_id", "Remove PDF/A or PDF/X identification", "Remove what identifies the document as PDF/A (its pdfaid: XMP properties) or PDF/X (GTS_PDFXVersion and GTS_PDFXConformance, in the document information and as pdfxid: or pdfx: XMP properties), e.g. after edits it no longer meets. Output intents and other metadata stay. Fails when the document declares nothing of the kind. Undoable.") .with(schema(json!({ "doc": doc(), "standard": { "type": "string", "enum": ["pdfa", "pdfx"] } }), &["doc", "standard"])), t("action_list", "List actions", "Action Wizard: the built-in actions (name, description, steps) and every step an action can use (id, label, whether it takes a text argument).") .ro() @@ -1479,7 +1566,7 @@ pub fn tools() -> Vec { t( "fill_sign_add", "Fill & Sign: type text or place a mark", - "Fill in a form that has no fields, as Acrobat's Fill & Sign does: type text (`text`, default 10 pt; `font_size` 4–300 pt; optional `color` as #RRGGBB for text/date, default black; `border` 0–12 pt around it, default none, in its colour or `border_color`), place a check, cross, dot or line, today's date (in Preferences ▸ Date format, see fill_sign_date_format, or `format`; `font_size` also applies), or a signature or initials (either `text` drawn in a script font, or `path` to a local PNG/JPEG image up to 4 MiB and 32 megapixels, scaled down to 4 megapixels, preserving transparency; with `scan: true`, `path` is instead a photo of a signature on paper, within the same limits: the signature is found, and the paper, shadows and table around it are removed, with `sensitivity` 0–100, default 50, picking up fainter ink when higher). `at` [x, y] is in points from the top-left of the page: text's top-left, mark's centre, signature's left edge centred vertically. Image signatures fit within 150 pt wide and 32 pt tall (24 pt for initials). Creates movable, undoable annotations. Typed text the standard fonts can't draw is saved and drawn as `?` (`warnings` in the result says so); a typed signature neither the script font nor its fallback can draw is refused (pass `path` instead).", + "Fill in a form that has no fields, as Acrobat's Fill & Sign does: type text (`text`, default 10 pt; `font_size` 4–300 pt; optional `color` as #RRGGBB for text/date, default black; `border` 0–12 pt around it, default none, in its colour or `border_color`), place a check, cross, dot or line, today's date (in Preferences ▸ Date format, see fill_sign_date_format, or `format`; `font_size` also applies), or a signature or initials (either `text` drawn in a script font, or `path` to a local PNG/JPEG image up to 4 MiB and 32 megapixels, scaled down to 4 megapixels, preserving transparency; with `scan: true`, `path` is instead a photo of a signature on paper, within the same limits: the signature is found, and the paper, shadows and table around it are removed, with `sensitivity` 0–100, default 50, picking up fainter ink when higher). `at` [x, y] is in points from the top-left of the page: text's top-left, mark's centre, signature's left edge centred vertically. Image signatures fit within 150 pt wide and 32 pt tall (24 pt for initials). Creates movable, undoable annotations. Typed Thai is drawn from PdfCraft's built-in Thai face (a build without one refuses it); other typed text the standard fonts can't draw is saved and drawn as `?` (`warnings` in the result says so); a typed signature neither the script font nor its fallback can draw is refused (pass `path` instead).", ) .cmd("sign.fill.text") .with(schema( @@ -1508,7 +1595,7 @@ pub fn tools() -> Vec { t( "doc_create", "Create a PDF", - "Create a new, unsaved document and return it like doc_open: `blank` (pages, width, height in points; default 1 US Letter page), `images` (paths of PNG, JPEG, TIFF (every page), GIF or BMP files, one page each at the image's resolution) or `text` (a .txt path, or `text` directly; Western European (WinAnsi) characters only, others are refused). Save it with doc_save and a path.", + "Create a new, unsaved document and return it like doc_open: `blank` (pages, width, height in points; default 1 US Letter page), `images` (paths of PNG, JPEG, TIFF (every page), GIF or BMP files, one page each at the image's resolution) or `text` (a .txt path, or `text` directly; set in Helvetica, or in the bundled Inter face when it has Latin, Greek or Cyrillic letters outside Western European (WinAnsi); other characters are refused). Save it with doc_save and a path.", ) .with(schema( json!({ @@ -1638,7 +1725,7 @@ pub fn tools() -> Vec { .with(schema(json!({ "doc": doc() }), &["doc"])), t("doc_open_revision", "Open a revision", "Open saved revision `revision` (1 = the oldest) of a document as a new, unsaved document, to see the file as it was then.") .with(schema(json!({ "doc": doc(), "revision": { "type": "integer", "minimum": 1 } }), &["doc", "revision"])), - t("doc_optimize", "Optimize PDF", "Write an optimized copy to `path` (Acrobat's PDF Optimizer). preset: standard (default; Reduce File Size's choices) or mobile (images to 96 ppi above 144, JPEG high quality); the options below change it. compatibility: retain (default; keeps the document's version, or 1.5 when object streams are written), 1.4, 1.5, 1.6, 1.7 or 2.0. merge_fragments (default true) first puts a page picture drawn as many stacked image strips (as some PDF printers write it) back together into one image, so it can be resampled (merged_images, fragments_merged). color / gray: { downsample, ppi, above_ppi, compression: jpeg|zip|retain, quality 1–100 } (defaults: downsample to 150 ppi above 225, JPEG 60). mono: { downsample, ppi, above_ppi, compression: ccitt_g4|jbig2|retain } for black-and-white images and image masks (default: averaged down to 300 ppi above 450 ppi, counted in images_resampled; stored losslessly as CCITT Group 4 or JBIG2 (or 1-bit Flate, once resampled or packed from more bits) when that is smaller, up to 16 JBIG2 images sharing one symbol dictionary (/JBIG2Globals) where that is smaller still (image masks and DeviceGray images named directly, of at most 64 Mi pixels, which redaction can store again without it), Group 3 fax images, JBIG2 images, black-and-white RGB and indexed images using only palette entries 0 and 1 among them, counted in images_recompressed; retain resamples only, a CCITT image staying CCITT, a JBIG2 image JBIG2 and any other becoming Flate). Images past what one run may decode stay as they are, counted in images_refused; streams it only stores with Flate past that stay LZW, uncounted. Images are measured where pages draw them and replaced only when smaller. discard_*: thumbnails (default true), alternate_images (true), tags, print_settings; flate_unencoded (true); object_streams (default true; always off for 1.4); lzw_to_flate (true) decodes LZW streams and stores them with Flate instead, leaving any it cannot decode unchanged (lzw_streams counts the streams converted); remove_invalid_links and remove_unreferenced_dests (true); images and forms the pages list but never draw are always dropped (unused_xobjects). discard: Remove Hidden Information categories (metadata, attachments, comments, form-fields, hidden-text, hidden-layers, bookmarks, links-actions-scripts, private-data). Signed documents are refused. target_bytes (optional): if the copy is larger, retry with up to 5 progressively stronger image settings (downsample to at most 110, 96, 72, 54, 36 ppi, applied to colour and grey images; images set to JPEG also get JPEG quality at most 45, 35, 28, 20, 12, while zip and retain images keep their encoding and only get the lower ppi) and stop at the first copy that fits; reports target_reached, attempts and target_settings, and writes the smallest copy with target_reached false when none fits. Without it nothing changes. The open document is unchanged.") + t("doc_optimize", "Optimize PDF", "Write an optimized copy to `path` (Acrobat's PDF Optimizer). preset: standard (default; Reduce File Size's choices) or mobile (images to 96 ppi above 144, JPEG high quality); the options below change it. compatibility: retain (default; keeps the document's version, or 1.5 when object streams are written), 1.4, 1.5, 1.6, 1.7 or 2.0. merge_fragments (default true) first puts a page picture drawn as many stacked image strips (as some PDF printers write it) back together into one image, so it can be resampled (merged_images, fragments_merged). color / gray: { downsample, ppi, above_ppi, compression: jpeg|zip|retain, quality 1–100 } (defaults: downsample to 150 ppi above 225, JPEG 60). mono: { downsample, ppi, above_ppi, compression: ccitt_g4|jbig2|retain } for black-and-white images and image masks (default: averaged down to 300 ppi above 450 ppi, counted in images_resampled; stored losslessly as CCITT Group 4 or JBIG2 (or 1-bit Flate, once resampled or packed from more bits) when that is smaller, up to 16 JBIG2 images sharing one symbol dictionary (/JBIG2Globals) where that is smaller still (image masks and DeviceGray images named directly, of at most 64 Mi pixels, which redaction can store again without it), Group 3 fax images, JBIG2 images, black-and-white RGB and indexed images using only palette entries 0 and 1 among them, counted in images_recompressed; retain resamples only, a CCITT image staying CCITT, a JBIG2 image JBIG2 and any other becoming Flate). Images past what one run may decode stay as they are, counted in images_refused; streams it only stores with Flate past that stay LZW, uncounted. Images are measured where pages draw them and replaced only when smaller. discard_*: thumbnails (default true), alternate_images (true), tags, search_index (removes the catalog's /PieceInfo entries whose application name contains \"index\" in any letter case; search_indexes), print_settings (entries removed: print_settings), form_actions (submit, import and reset actions removed: form_actions; the fields and links stay, and the actions chained after one still run); flate_unencoded (true); object_streams (default true; always off for 1.4); lzw_to_flate (true) decodes LZW streams and stores them with Flate instead, leaving any it cannot decode unchanged (lzw_streams counts the streams converted); remove_invalid_links and remove_unreferenced_dests (true); discard_external_refs takes GoToR, GoToE, Launch and ImportData actions, and Thread actions with /F, out of links and bookmarks, removes a link left with nothing to do, and keeps links within the document and web links (external_links, external_bookmarks); images and forms the pages list but never draw are always dropped (unused_xobjects). discard: Remove Hidden Information categories (metadata, attachments, comments, form-fields, hidden-text, hidden-layers, bookmarks, links-actions-scripts, private-data). Signed documents are refused. target_bytes (optional): if the copy is larger, retry with up to 5 progressively stronger image settings (downsample to at most 110, 96, 72, 54, 36 ppi, applied to colour and grey images; images set to JPEG also get JPEG quality at most 45, 35, 28, 20, 12, while zip and retain images keep their encoding and only get the lower ppi) and stop at the first copy that fits; reports target_reached, attempts and target_settings, and writes the smallest copy with target_reached false when none fits. Without it nothing changes. The open document is unchanged.") .cmd("optimize.advanced") .with(schema( json!({ @@ -1654,12 +1741,15 @@ pub fn tools() -> Vec { "discard_thumbnails": { "type": "boolean" }, "discard_alternate_images": { "type": "boolean" }, "discard_tags": { "type": "boolean" }, + "discard_search_index": { "type": "boolean" }, "discard_print_settings": { "type": "boolean" }, + "discard_form_actions": { "type": "boolean" }, "flate_unencoded": { "type": "boolean" }, "object_streams": { "type": "boolean" }, "lzw_to_flate": { "type": "boolean" }, "remove_invalid_links": { "type": "boolean" }, "remove_unreferenced_dests": { "type": "boolean" }, + "discard_external_refs": { "type": "boolean" }, "discard": { "type": "array", "items": { "type": "string" } }, }), &["doc", "path"], diff --git a/crates/automation/tests/it/automation.rs b/crates/automation/tests/it/automation.rs index 4d0620addf..bc9d2e9ee5 100644 --- a/crates/automation/tests/it/automation.rs +++ b/crates/automation/tests/it/automation.rs @@ -2107,6 +2107,249 @@ fn destinations_are_listed_with_their_position() { assert_eq!((links["links"][2]["url"].clone(), links["links"][2].get("destination")), (json!("https://example.com"), None)); } +/// Web, `www.` and email addresses written as plain page text, which the viewer opens on a click +/// (#4207), are listed by link_list with text_links: true; one under a link annotation is not. +#[test] +fn addresses_written_as_page_text_are_listed_with_text_links() { + let dir = workdir("text-links"); + let content = [ + (700, "Visit https://example.com/plain-text-url."), + (650, "or www.example.com/no-scheme"), + (600, "Write to someone@example.com, please"), + (550, "https://example.org/covered"), + (500, "Version 1.2.3 and file:///etc/passwd"), + ] + .iter() + .map(|(y, s)| format!("BT /F1 14 Tf 72 {y} Td ({s}) Tj ET\n")) + .collect::(); + let objs: Vec = vec![ + "<< /Type /Catalog /Pages 2 0 R >>".into(), + "<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>".into(), + "<< /Type /Page /Parent 2 0 R /Resources << /Font << /F1 5 0 R >> >> /Contents 4 0 R /Annots [6 0 R] >>".into(), + format!("<< /Length {} >>\nstream\n{content}endstream", content.len()), + "<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>".into(), + "<< /Type /Annot /Subtype /Link /Rect [60 545 400 565] /Border [0 0 0] /A << /S /URI /URI (https://example.org/annotation) >> >>".into(), + ]; + let mut pdf = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offsets.push(pdf.len()); + pdf.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes()); + } + let xref = pdf.len(); + pdf.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offsets { + pdf.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + pdf.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes()); + std::fs::write(dir.join("text.pdf"), pdf).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "text.pdf" }))["doc"].as_u64().unwrap(); + assert_eq!(ok(&mut a, "link_list", json!({ "doc": doc })).get("text_links"), None, "only on request"); + let listed = ok(&mut a, "link_list", json!({ "doc": doc, "text_links": true })); + let found = listed["text_links"].as_array().unwrap(); + let urls: Vec<&str> = found.iter().map(|l| l["url"].as_str().unwrap()).collect(); + assert_eq!(urls, ["https://example.com/plain-text-url", "https://www.example.com/no-scheme", "mailto:someone@example.com"], "{listed}"); + assert!(found.iter().all(|l| l["page"] == 1 && l.get("blocked").is_none()), "{listed}"); + // Rectangles from the top-left of the page: the first line's baseline is 792 − 700 = 92 down. + let r = found[0]["rects"][0].as_array().unwrap().iter().map(|v| v.as_f64().unwrap()).collect::>(); + assert!(r[0] > 100.0 && r[0] < 140.0 && r[1] < 92.0 && r[3] > 80.0, "{r:?}"); +} + +/// One PDF of `pages` pages that all draw `content` in Helvetica. +fn same_text_pages(pages: usize, content: &str) -> Vec { + let kids: String = (0..pages).map(|k| format!("{} 0 R ", k + 5)).collect(); + let mut objs: Vec = vec![ + "<< /Type /Catalog /Pages 2 0 R >>".into(), + format!("<< /Type /Pages /Kids [{kids}] /Count {pages} /MediaBox [0 0 612 792] >>"), + format!("<< /Length {} >>\nstream\n{content}endstream", content.len()), + "<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>".into(), + ]; + objs.extend((0..pages).map(|_| "<< /Type /Page /Parent 2 0 R /Resources << /Font << /F1 4 0 R >> >> /Contents 3 0 R >>".to_string())); + let mut pdf = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offsets.push(pdf.len()); + pdf.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes()); + } + let xref = pdf.len(); + pdf.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offsets { + pdf.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + pdf.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes()); + pdf +} + +/// Review on #4472: link_list with text_links reads a share of a large document's pages per +/// call and says where to go on, so a 10,000-page document is never read all at once; a call +/// also stops after the page that brings it to 2,000 addresses. +#[test] +fn text_links_of_a_large_document_are_read_a_share_at_a_time() { + let dir = workdir("text-links-large"); + std::fs::write(dir.join("long.pdf"), same_text_pages(10_000, "BT /F1 14 Tf 72 700 Td (See https://example.com/page now) Tj ET\n")).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "long.pdf" }))["doc"].as_u64().unwrap(); + let first = ok(&mut a, "link_list", json!({ "doc": doc, "text_links": true })); + let found = first["text_links"].as_array().unwrap(); + assert_eq!(found.len(), 100, "the first 100 pages"); + assert_eq!((found[0]["page"].clone(), found[99]["page"].clone()), (json!(1), json!(100))); + assert_eq!(first["text_links_next_page"], json!(101)); + let last = ok(&mut a, "link_list", json!({ "doc": doc, "text_links": true, "text_links_from": 9_951 })); + let found = last["text_links"].as_array().unwrap(); + assert_eq!((found.len(), found[0]["page"].clone()), (50, json!(9_951))); + assert_eq!(last.get("text_links_next_page"), None, "nothing after the last page"); + for from in [0, -1, 10_001, i64::MAX] { + err(&mut a, "link_list", json!({ "doc": doc, "text_links": true, "text_links_from": from })); + } + + // 52 lines of 10 addresses: 520 on a page, kept to 512; the fourth page brings a call to + // 2,048, so it stops there. + let line = "x@example.org ".repeat(10); + let content: String = (0..52).map(|k| format!("BT /F1 6 Tf 20 {} Td ({line}) Tj ET\n", 760 - k * 7)).collect(); + std::fs::write(dir.join("dense.pdf"), same_text_pages(6, &content)).unwrap(); + let doc = ok(&mut a, "doc_open", json!({ "path": "dense.pdf" }))["doc"].as_u64().unwrap(); + let dense = ok(&mut a, "link_list", json!({ "doc": doc, "text_links": true })); + assert_eq!(dense["text_links"].as_array().unwrap().len(), 4 * 512); + assert_eq!(dense["text_links_next_page"], json!(5)); + let rest = ok(&mut a, "link_list", json!({ "doc": doc, "text_links": true, "text_links_from": 5 })); + assert_eq!((rest["text_links"].as_array().unwrap().len(), rest.get("text_links_next_page")), (2 * 512, None)); +} + +/// An address on a layer the viewer hides is not a text link, and Create links from URLs makes +/// no link there: link_list and links_from_urls read the page text with the layers as layer_set +/// left them (ISO 32000-2 §8.11.3.1: hidden optional content "shall not be drawn"). +#[test] +fn text_links_leave_out_addresses_on_hidden_layers() { + let dir = workdir("text-links-layers"); + let content = + "BT /F1 14 Tf 72 700 Td (https://example.com/shown) Tj ET\n/OC /L1 BDC BT /F1 14 Tf 72 650 Td (https://example.com/layered) Tj ET EMC\n"; + let objs: Vec = vec![ + "<< /Type /Catalog /Pages 2 0 R /OCProperties << /OCGs [6 0 R] /D << /Order [6 0 R] >> >> >>".into(), + "<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>".into(), + "<< /Type /Page /Parent 2 0 R /Resources << /Font << /F1 5 0 R >> /Properties << /L1 6 0 R >> >> /Contents 4 0 R >>".into(), + format!("<< /Length {} >>\nstream\n{content}endstream", content.len()), + "<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>".into(), + "<< /Type /OCG /Name (Addresses) >>".into(), + ]; + let mut pdf = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offsets.push(pdf.len()); + pdf.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes()); + } + let xref = pdf.len(); + pdf.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offsets { + pdf.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + pdf.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes()); + std::fs::write(dir.join("layers.pdf"), pdf).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "layers.pdf" }))["doc"].as_u64().unwrap(); + let urls = |a: &mut Automation| -> Vec { + let listed = ok(a, "link_list", json!({ "doc": doc, "text_links": true })); + listed["text_links"].as_array().unwrap().iter().map(|l| l["url"].as_str().unwrap().to_owned()).collect() + }; + assert_eq!(urls(&mut a), ["https://example.com/shown", "https://example.com/layered"]); + ok(&mut a, "layer_set", json!({ "doc": doc, "layer": "Addresses", "visible": false })); + assert_eq!(urls(&mut a), ["https://example.com/shown"]); + let created = ok(&mut a, "links_from_urls", json!({ "doc": doc })); + assert_eq!(created["created"], json!(["https://example.com/shown"]), "no link over the hidden address: {created}"); + ok(&mut a, "edit_undo", json!({ "doc": doc })); + ok(&mut a, "layer_set", json!({ "doc": doc, "layer": "Addresses", "visible": true })); + assert_eq!(urls(&mut a), ["https://example.com/shown", "https://example.com/layered"]); + let created = ok(&mut a, "links_from_urls", json!({ "doc": doc })); + assert_eq!(created["created"], json!(["https://example.com/shown", "https://example.com/layered"]), "{created}"); +} + +/// Table 211), and a link with `/QuadPoints` is active only there (§12.5.6.5, Table 176): both as +/// link_list and doc_info report them. +#[test] +fn links_resolve_relative_addresses_and_report_their_quad_points() { + let dir = workdir("link-base-quads"); + let objs: Vec<&str> = vec![ + "<< /Type /Catalog /Pages 2 0 R /URI << /Base (https://example.com/base/) >> >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>", + "<< /Type /Page /Parent 2 0 R /Annots [4 0 R 5 0 R 6 0 R] >>", + "<< /Type /Annot /Subtype /Link /Rect [72 700 300 716] /Border [0 0 0] /A << /S /URI /URI (page.html) >> >>", + "<< /Type /Annot /Subtype /Link /Rect [72 670 300 686] /QuadPoints [72 670 120 670 120 686 72 686] /Border [0 0 0] /A << /S /URI /URI (https://example.org/q) >> >>", + "<< /Type /Annot /Subtype /Link /Rect [72 640 300 656] /Border [0 0 0] /A << /S /URI /URI (javascript:alert\\(1\\)) >> >>", + ]; + let mut pdf = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offsets.push(pdf.len()); + pdf.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes()); + } + let xref = pdf.len(); + pdf.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offsets { + pdf.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + pdf.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes()); + std::fs::write(dir.join("links.pdf"), pdf).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "links.pdf" }))["doc"].as_u64().unwrap(); + let links = ok(&mut a, "link_list", json!({ "doc": doc })); + let [relative, quads, script] = [&links["links"][0], &links["links"][1], &links["links"][2]]; + assert_eq!( + (relative["url"].clone(), relative["resolved_url"].clone()), + (json!("page.html"), json!("https://example.com/base/page.html")), + "{links}" + ); + assert_eq!(relative.get("active_areas"), None, "{links}"); + // 792 − 686 = 106 points from the top: only the first word, not the whole rect. + assert_eq!(quads["active_areas"], json!([[72.0, 106.0, 120.0, 122.0]]), "{links}"); + assert_eq!(quads.get("resolved_url"), None, "an absolute address is not resolved"); + // A base never turns a script into a web address. + assert_eq!((script["url"].clone(), script.get("resolved_url")), (json!("javascript:alert(1)"), None), "{links}"); + let info = ok(&mut a, "doc_info", json!({ "doc": doc })); + assert_eq!(info["links"][0]["target"]["uri"], "https://example.com/base/page.html", "{info}"); +} + +/// Once inspection's text budget is spent, a relative address stays raw, and the viewer opens it +/// as written: `link_list` reports that address, not a second resolution against `/URI /Base`. +#[test] +fn link_list_reports_the_address_the_viewer_opens_past_the_text_budget() { + let dir = workdir("link-base-budget"); + // Each link spends what the base adds to its address; well past 16 MiB of it in all. + let base = format!("https://example.com/{}/", "b".repeat(8000)); + let count = (16 << 20) / base.len() + 20; + let annots = "<< /Type /Annot /Subtype /Link /Rect [0 0 10 10] /Border [0 0 0] /A 4 0 R >> ".repeat(count); + let catalog = format!("<< /Type /Catalog /Pages 2 0 R /URI << /Base ({base}) >> >>"); + let page = format!("<< /Type /Page /Parent 2 0 R /Annots [{annots}] >>"); + let objs = [catalog.as_str(), "<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>", page.as_str(), "<< /S /URI /URI (p.html) >>"]; + let mut pdf = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offsets.push(pdf.len()); + pdf.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes()); + } + let xref = pdf.len(); + pdf.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offsets { + pdf.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + pdf.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes()); + std::fs::write(dir.join("links.pdf"), pdf).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "links.pdf" }))["doc"].as_u64().unwrap(); + let listed = ok(&mut a, "link_list", json!({ "doc": doc })); + let links = listed["links"].as_array().unwrap(); + assert_eq!(links.len(), count); + let info = ok(&mut a, "doc_info", json!({ "doc": doc })); + let opened = info["links"].as_array().unwrap(); + assert_eq!(opened.len(), count); + let resolved = format!("{base}p.html"); + assert_eq!(links[0]["resolved_url"], json!(resolved), "the first links resolve"); + assert_eq!(opened[0]["target"]["uri"], json!(resolved)); + // Past the budget the viewer keeps (and opens) the raw address; the listing says so. + let last = count - 1; + assert_eq!(opened[last]["target"]["uri"], json!("p.html")); + assert_eq!((links[last]["url"].clone(), links[last].get("resolved_url")), (json!("p.html"), None)); +} + #[test] fn bookmarks_from_structure_through_tools() { let dir = workdir("bookmarks-structure"); @@ -2155,6 +2398,98 @@ fn numbering_pages_through_tools() { assert!(matches!(a.call("page_number", &json!({ "doc": doc, "from": 1, "to": 1, "style": "klingon" })), Err(ToolError::InvalidArgs(_)))); } +/// ISO 32000-2 §12.5.6.3: a state is "for the specified user" (`/T`), so there is one status per +/// reviewer (their latest), and a checkmark ("marked by the user", Table 174) belongs only to the +/// reviewer who set it. `comment_list` reports each reviewer's status and who has a checkmark on; +/// the replies written are hidden (a state reply records a state, not a note) and a reviewer's +/// later change replies to their earlier one. +#[test] +fn review_statuses_are_per_reviewer_through_tools() { + let dir = workdir("statuses"); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "a.pdf" }))["doc"].as_u64().unwrap(); + let id = + ok(&mut a, "comment_add", json!({ "doc": doc, "page": 1, "type": "note", "at": [20, 20], "contents": "Check", "author": "Ada" }))["comment"] + ["id"] + .as_str() + .unwrap() + .to_string(); + for (status, author) in [("accepted", "Bob"), ("completed", "Carol"), ("rejected", "Bob")] { + ok(&mut a, "comment_set_status", json!({ "doc": doc, "id": id, "status": status, "author": author })); + } + ok(&mut a, "comment_mark", json!({ "doc": doc, "id": id, "author": "Bob" })); + ok(&mut a, "comment_mark", json!({ "doc": doc, "id": id, "author": "Carol" })); + ok(&mut a, "comment_mark", json!({ "doc": doc, "id": id, "author": "Carol", "marked": false })); + ok(&mut a, "doc_save", json!({ "doc": doc, "path": "statuses.pdf" })); + let doc = ok(&mut a, "doc_open", json!({ "path": "statuses.pdf" }))["doc"].as_u64().unwrap(); + let list = ok(&mut a, "comment_list", json!({ "doc": doc })); + assert_eq!(list["count"], 1, "status replies aren't comments: {list}"); + let c = &list["comments"][0]; + assert_eq!(c["statuses"], json!([{ "author": "Bob", "status": "Rejected" }, { "author": "Carol", "status": "Completed" }])); + assert_eq!(c["status"], "Rejected", "the latest set by anyone"); + assert_eq!((c["marked"].clone(), c["marked_by"].clone()), (json!(true), json!(["Bob"]))); + assert_eq!(c["replies"], json!([])); + + let bytes = std::fs::read(dir.join("statuses.pdf")).unwrap(); + let saved = pdfcraft_cos::Document::open(std::sync::Arc::new(bytes)).unwrap(); + let replies: Vec = saved + .object_numbers() + .into_iter() + .filter_map(|n| saved.get(pdfcraft_cos::ObjRef { num: n, generation: 0 }).as_dict().cloned()) + .filter(|d| d.contains(b"StateModel")) + .collect(); + assert_eq!(replies.len(), 6); + assert!(replies.iter().all(|d| d.int(b"F") == Some(30)), "every state reply is hidden: {replies:?}"); +} + +/// ISO 32000-2 §12.5.6.3: a state reply's `/IRT` "shall refer to the original annotation", which +/// for a status set on a reply is that reply. The reviewer's next change replies to their previous +/// state reply, and the chain, found by reference, stays the reply's: it is reported on the reply +/// and never on its comment. +#[test] +fn statuses_set_on_a_reply_chain_on_that_reply_through_tools() { + let dir = workdir("reply-statuses"); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "a.pdf" }))["doc"].as_u64().unwrap(); + let id = + ok(&mut a, "comment_add", json!({ "doc": doc, "page": 1, "type": "note", "at": [20, 20], "contents": "Check", "author": "Ada" }))["comment"] + ["id"] + .as_str() + .unwrap() + .to_string(); + ok(&mut a, "comment_reply", json!({ "doc": doc, "id": id, "text": "ANSWER", "author": "Bob" })); + let listed = ok(&mut a, "comment_list", json!({ "doc": doc })); + let reply_id = listed["comments"][0]["replies"][0]["id"].as_str().unwrap().to_string(); + for (status, author) in [("accepted", "Carol"), ("completed", "Dan"), ("rejected", "Carol")] { + ok(&mut a, "comment_set_status", json!({ "doc": doc, "id": reply_id, "status": status, "author": author })); + } + ok(&mut a, "doc_save", json!({ "doc": doc, "path": "reply-statuses.pdf" })); + let doc = ok(&mut a, "doc_open", json!({ "path": "reply-statuses.pdf" }))["doc"].as_u64().unwrap(); + let list = ok(&mut a, "comment_list", json!({ "doc": doc })); + let c = &list["comments"][0]; + assert_eq!((&c["status"], &c["statuses"]), (&Value::Null, &json!([])), "not the comment's: {c}"); + assert_eq!(c["replies"].as_array().map(Vec::len), Some(1), "{c}"); + let reply = &c["replies"][0]; + assert_eq!(reply["statuses"], json!([{ "author": "Carol", "status": "Rejected" }, { "author": "Dan", "status": "Completed" }]), "{reply}"); + + // Carol's second status replies to her first, which replies to the reply. + let bytes = std::fs::read(dir.join("reply-statuses.pdf")).unwrap(); + let saved = pdfcraft_cos::Document::open(std::sync::Arc::new(bytes)).unwrap(); + let dicts: Vec<(pdfcraft_cos::ObjRef, pdfcraft_cos::Dict)> = saved + .object_numbers() + .into_iter() + .map(|n| pdfcraft_cos::ObjRef { num: n, generation: 0 }) + .filter_map(|r| Some((r, saved.get(r).as_dict()?.clone()))) + .collect(); + let find = |text: &str| -> pdfcraft_cos::ObjRef { + dicts.iter().find(|(_, d)| d.get(b"Contents").and_then(|o| o.as_string()).is_some_and(|s| s.to_text().contains(text))).unwrap().0 + }; + let irt = |r: pdfcraft_cos::ObjRef| dicts.iter().find(|(x, _)| *x == r).and_then(|(_, d)| d.get(b"IRT").and_then(|o| o.as_ref())); + let (answer, first, second) = (find("ANSWER"), find("Accepted set by Carol"), find("Rejected set by Carol")); + assert_eq!(irt(first), Some(answer)); + assert_eq!(irt(second), Some(first)); +} + /// Typed text and text boxes take an oval or a circle for their border through the tools; /// comment_list reports it (a rectangle is not reported). Only typed text, dates and text boxes /// have one, and only the three shapes. @@ -2509,6 +2844,193 @@ trailer << /Root 1 0 R >> assert_eq!((&reply["marked"], &reply["status"]), (&json!(true), &json!("Accepted")), "{reply}"); } +// Contributor-original PDF: a Square, a Circle grouped with it (`/RT /Group`) that keeps its own +// author, text, colour, dates and subject, an ordinary reply, a reply to the Circle, a review +// status reply and a reply with a `/RT` name that is neither `R` nor `Group`. +const GROUP_FIXTURE: &[u8] = b"%PDF-1.7 +1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj +2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj +3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 300 200] /Annots [5 0 R 6 0 R 7 0 R 8 0 R 9 0 R 10 0 R] >> endobj +5 0 obj << /Type /Annot /Subtype /Square /Rect [20 20 100 80] /NM (sq-primary) /T (Alice) /Contents (Primary) /C [1 0 0] >> endobj +6 0 obj << /Type /Annot /Subtype /Circle /Rect [180 120 260 180] /NM (circ-sub) /T (Bob) /Contents (Subordinate) /C [0 0 1] /IRT 5 0 R /RT /Group /M (D:20260101000000) /CreationDate (D:20260101000000) /Subj (Oval) >> endobj +7 0 obj << /Type /Annot /Subtype /Text /Rect [120 20 140 40] /NM (reply) /T (Carol) /Contents (Reply) /IRT 5 0 R >> endobj +8 0 obj << /Type /Annot /Subtype /Text /Rect [120 50 140 70] /NM (sub-reply) /T (Dan) /Contents (Reply to the subordinate) /IRT 6 0 R >> endobj +9 0 obj << /Type /Annot /Subtype /Text /Rect [120 80 140 100] /NM (status) /T (Eve) /IRT 5 0 R /StateModel /Review /State /Accepted >> endobj +10 0 obj << /Type /Annot /Subtype /Text /Rect [150 20 170 40] /NM (odd) /T (Fay) /Contents (Odd reply type) /IRT 5 0 R /RT /Bogus >> endobj +trailer << /Root 1 0 R >> +%%EOF"; + +/// ISO 32000-2 §12.5.6.2: an annotation with `/RT /Group` is a subordinate of the group whose +/// primary its `/IRT` names. It lists as one comment, with the primary's attributes; the +/// subordinate's own are ignored, and a reply to it joins the primary's thread. +#[test] +fn a_group_lists_as_its_primary() { + let dir = workdir("rt-group"); + std::fs::write(dir.join("group.pdf"), GROUP_FIXTURE).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "group.pdf" }))["doc"].as_u64().unwrap(); + let check = |list: &Value, when: &str| { + assert_eq!(list["count"], 1, "{when}: one comment, the group's primary: {list}"); + let c = &list["comments"][0]; + assert_eq!((c["id"].as_str(), c["author"].as_str(), c["contents"].as_str()), (Some("sq-primary"), Some("Alice"), Some("Primary")), "{when}"); + assert_eq!(c["color"], "#FF0000", "{when}: the primary's colour"); + let group: Vec<&str> = c["group"].as_array().unwrap().iter().filter_map(|m| m["id"].as_str()).collect(); + assert_eq!(group, ["circ-sub"], "{when}: the subordinate is a member of the group"); + assert_eq!(c["group"][0]["type"], "Circle", "{when}"); + let mut replies: Vec<&str> = c["replies"].as_array().unwrap().iter().filter_map(|r| r["id"].as_str()).collect(); + replies.sort_unstable(); + assert_eq!(replies, ["odd", "reply", "sub-reply"], "{when}: an unknown /RT reads as a reply, a reply to the subordinate joins the thread"); + assert_eq!(c["status"], "Accepted", "{when}: a status reply is still a reply"); + let text = list.to_string(); + for leak in ["Bob", "Subordinate", "#0000FF"] { + assert!(!text.contains(leak), "{when}: the subordinate's own {leak:?} is ignored: {list}"); + } + }; + // As opened (the inspector), and after an edit (the engine's own list). + check(&ok(&mut a, "comment_list", json!({ "doc": doc })), "opened"); + ok(&mut a, "comment_add", json!({ "doc": doc, "page": 1, "type": "note", "at": [280, 20], "contents": "Other" })); + let after = ok(&mut a, "comment_list", json!({ "doc": doc })); + let primary = after["comments"].as_array().unwrap().iter().find(|c| c["id"] == "sq-primary").cloned().unwrap(); + check(&json!({ "count": 1, "comments": [primary] }), "edited"); + assert_eq!(after["count"], 2, "{after}"); + + // Deleting a member deletes just it, as Acrobat does: the subordinate, or the primary, which + // leaves the subordinate a comment of its own. One undo step each. + ok(&mut a, "comment_delete", json!({ "doc": doc, "id": "circ-sub" })); + let left = ok(&mut a, "comment_list", json!({ "doc": doc })); + assert_eq!(left["count"], 2, "the primary and the new note are left: {left}"); + assert!(left.to_string().contains("sq-primary") && !left.to_string().contains("circ-sub"), "{left}"); + ok(&mut a, "edit_undo", json!({ "doc": doc })); + ok(&mut a, "comment_delete", json!({ "doc": doc, "id": "sq-primary" })); + let left = ok(&mut a, "comment_list", json!({ "doc": doc })); + let ids: Vec<&str> = left["comments"].as_array().unwrap().iter().filter_map(|c| c["id"].as_str()).collect(); + assert!(ids.contains(&"circ-sub") && !ids.contains(&"sq-primary") && ids.len() == 2, "the new note and the Circle, on its own: {left}"); + ok(&mut a, "edit_undo", json!({ "doc": doc })); + let back = ok(&mut a, "comment_list", json!({ "doc": doc })); + let primary = back["comments"].as_array().unwrap().iter().find(|c| c["id"] == "sq-primary").cloned().unwrap(); + check(&json!({ "count": 1, "comments": [primary] }), "undone"); +} + +// Contributor-original PDF: group subordinates whose `/IRT` chains loop (each other, and one +// itself), and a chain of two subordinates that leads to a primary. +const GROUP_LOOP_FIXTURE: &[u8] = b"%PDF-1.7 +1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj +2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj +3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 300 200] /Annots [5 0 R 6 0 R 7 0 R 8 0 R 9 0 R 10 0 R] >> endobj +5 0 obj << /Type /Annot /Subtype /Circle /Rect [10 10 30 30] /NM (a) /Contents (Loop A) /IRT 6 0 R /RT /Group >> endobj +6 0 obj << /Type /Annot /Subtype /Circle /Rect [40 10 60 30] /NM (b) /Contents (Loop B) /IRT 5 0 R /RT /Group >> endobj +7 0 obj << /Type /Annot /Subtype /Circle /Rect [70 10 90 30] /NM (self) /Contents (Itself) /IRT 7 0 R /RT /Group >> endobj +8 0 obj << /Type /Annot /Subtype /Square /Rect [10 100 60 150] /NM (top) /Contents (Top) >> endobj +9 0 obj << /Type /Annot /Subtype /Circle /Rect [70 100 90 120] /NM (mid) /Contents (Middle) /IRT 8 0 R /RT /Group >> endobj +10 0 obj << /Type /Annot /Subtype /Circle /Rect [100 100 120 120] /NM (low) /Contents (Lowest) /IRT 9 0 R /RT /Group >> endobj +trailer << /Root 1 0 R >> +%%EOF"; + +/// Looping `/IRT` chains between group subordinates neither hang nor panic, and list as comments +/// of their own, as looping replies do; a subordinate of a subordinate joins the group of the +/// primary the chain leads to. +#[test] +fn looping_groups_end_and_chained_groups_reach_their_primary() { + let dir = workdir("rt-group-loop"); + std::fs::write(dir.join("loop.pdf"), GROUP_LOOP_FIXTURE).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "loop.pdf" }))["doc"].as_u64().unwrap(); + let list = ok(&mut a, "comment_list", json!({ "doc": doc })); + let comments = list["comments"].as_array().unwrap(); + let mut ids: Vec<&str> = comments.iter().filter_map(|c| c["id"].as_str()).collect(); + ids.sort_unstable(); + assert_eq!(ids, ["a", "b", "self", "top"], "{list}"); + let top = comments.iter().find(|c| c["id"] == "top").unwrap(); + let group: Vec<&str> = top["group"].as_array().unwrap().iter().filter_map(|m| m["id"].as_str()).collect(); + assert_eq!(group, ["mid", "low"], "{list}"); + assert_eq!(top["replies"], json!([]), "{list}"); + // Deleting a looping member ends too and takes just it, as any group member goes alone; so + // does the lowest subordinate of the chain. + ok(&mut a, "comment_delete", json!({ "doc": doc, "id": "a" })); + ok(&mut a, "comment_delete", json!({ "doc": doc, "id": "self" })); + ok(&mut a, "comment_delete", json!({ "doc": doc, "id": "low" })); + let list = ok(&mut a, "comment_list", json!({ "doc": doc })); + let mut ids: Vec<&str> = list["comments"].as_array().unwrap().iter().filter_map(|c| c["id"].as_str()).collect(); + ids.sort_unstable(); + assert_eq!(ids, ["b", "top"], "{list}"); + let top = list["comments"].as_array().unwrap().iter().find(|c| c["id"] == "top").cloned().unwrap_or_default(); + let group: Vec<&str> = top["group"].as_array().into_iter().flatten().filter_map(|m| m["id"].as_str()).collect(); + assert_eq!(group, ["mid"], "{list}"); +} + +// Contributor-original PDF: a Replace Text group as earlier PdfCraft builds wrote it, the +// strikeout (primary) with empty `/Contents` and the replacement only on the caret (subordinate). +const OLD_REPLACE_FIXTURE: &[u8] = b"%PDF-1.7 +1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj +2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj +3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 300 200] /Annots [5 0 R 6 0 R] >> endobj +5 0 obj << /Type /Annot /Subtype /StrikeOut /Rect [20 100 120 120] /QuadPoints [20 120 120 120 20 100 120 100] /NM (strike) /T (Ada) /Contents () /IT /StrikeOutTextEdit >> endobj +6 0 obj << /Type /Annot /Subtype /Caret /Rect [115 95 125 105] /NM (caret) /T (Ada) /Contents (red) /IRT 5 0 R /RT /Group /Subj (Replace Text) >> endobj +trailer << /Root 1 0 R >> +%%EOF"; + +// Contributor-original PDF: a Replace Text group with the caret as the primary (holding the +// text, `/IT /Replace`) and the strikeout as its subordinate, each with its own author, colour +// and dates. +const REPLACE_FIXTURE: &[u8] = b"%PDF-1.7 +1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj +2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj +3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 300 200] /Annots [5 0 R 6 0 R] >> endobj +5 0 obj << /Type /Annot /Subtype /Caret /Rect [115 95 125 105] /NM (caret) /T (Ada) /C [0 0 1] /Contents (busy) /IT /Replace /Subj (Replace Text) /M (D:20260101000000) >> endobj +6 0 obj << /Type /Annot /Subtype /StrikeOut /Rect [20 100 120 120] /QuadPoints [20 120 120 120 20 100 120 100] /NM (strike) /T (Grace) /C [1 0 0] /IRT 5 0 R /RT /Group /IT /StrikeOutTextEdit /Subj (Cross-Out) /M (D:20260101000000) >> endobj +trailer << /Root 1 0 R >> +%%EOF"; + +/// A Replace Text comment lists once, with its replacement text: one made now (the caret is the +/// primary and holds it), one with that orientation from another writer, and one saved by an +/// earlier PdfCraft (the strikeout primary with no text, the caret holding it), which is read +/// leniently and not rewritten. +#[test] +fn replace_text_lists_its_replacement() { + let dir = workdir("rt-group-replace"); + std::fs::write(dir.join("old.pdf"), OLD_REPLACE_FIXTURE).unwrap(); + std::fs::write(dir.join("other.pdf"), REPLACE_FIXTURE).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "a.pdf" }))["doc"].as_u64().unwrap(); + ok(&mut a, "comment_add", json!({ "doc": doc, "page": 1, "type": "replace", "find": "Page 1", "contents": "red" })); + let list = ok(&mut a, "comment_list", json!({ "doc": doc })); + assert_eq!(list["count"], 1, "{list}"); + let c = &list["comments"][0]; + assert_eq!( + (c["type"].as_str(), c["contents"].as_str(), c["group"][0]["type"].as_str()), + (Some("Caret"), Some("red"), Some("StrikeOut")), + "{list}" + ); + ok(&mut a, "doc_save", json!({ "doc": doc, "path": "new.pdf" })); + let mut b = auto(&dir); + for (path, text, primary) in [("new.pdf", "red", "Caret"), ("other.pdf", "busy", "Caret"), ("old.pdf", "red", "StrikeOut")] { + let doc = ok(&mut b, "doc_open", json!({ "path": path }))["doc"].as_u64().unwrap(); + let list = ok(&mut b, "comment_list", json!({ "doc": doc })); + let c = &list["comments"][0]; + assert_eq!( + (list["count"].as_u64(), c["type"].as_str(), c["contents"].as_str()), + (Some(1), Some(primary), Some(text)), + "{path} as opened: {list}" + ); + assert!(!list.to_string().contains("Grace"), "{path}: the subordinate's author is ignored: {list}"); + // And from the engine's own list, after an edit. + ok(&mut b, "comment_add", json!({ "doc": doc, "page": 1, "type": "note", "at": [280, 20], "contents": "Other" })); + let list = ok(&mut b, "comment_list", json!({ "doc": doc })); + assert_eq!(list["count"], 2, "{path}: {list}"); + let c = list["comments"].as_array().unwrap().iter().find(|c| c["type"] == primary).cloned().unwrap(); + assert_eq!(c["contents"], text, "{path} after an edit: {list}"); + // Replace Text is one comment: deleting either half deletes both, as Acrobat does, in + // either orientation. + for id in [c["id"].clone(), c["group"][0]["id"].clone()] { + ok(&mut b, "comment_delete", json!({ "doc": doc, "id": id })); + let left = ok(&mut b, "comment_list", json!({ "doc": doc })); + assert_eq!(left["count"], 1, "{path}: deleting {id} leaves only the note: {left}"); + assert!(!left.to_string().contains("StrikeOut") && !left.to_string().contains("Caret"), "{path}: {left}"); + ok(&mut b, "edit_undo", json!({ "doc": doc })); + } + } +} + #[test] fn protecting_through_tools() { let dir = workdir("protect"); @@ -2818,6 +3340,298 @@ fn form_fill_warns_when_a_value_does_not_fit() { assert!(r.get("warnings").is_none(), "{r}"); } +/// `form_fill` takes a radio option as `form_fields` lists it: the export value. With `/Opt [(1) +/// (2) (3)]` and on states named by position (`/0`, `/1`, `/2`; ISO 32000-2 Table 230), `1` is the +/// first button. It used to be read as the state `/1` and turn the second one on. +#[test] +fn form_fill_picks_a_radio_option_by_its_export_value() { + let objs = [ + "<< /Type /Catalog /Pages 2 0 R /AcroForm 4 0 R >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 400 300] >>", + "<< /Type /Page /Parent 2 0 R /Annots [6 0 R 7 0 R 8 0 R] >>", + "<< /Fields [5 0 R] /DA (/Helv 0 Tf 0 g) >>", + "<< /FT /Btn /T (title) /Ff 49152 /Opt [(1) (2) (3)] /V /Off /Kids [6 0 R 7 0 R 8 0 R] >>", + "<< /Type /Annot /Subtype /Widget /Parent 5 0 R /AS /Off /Rect [10 250 22 262] /P 3 0 R /AP << /N << /0 9 0 R /Off 9 0 R >> >> >>", + "<< /Type /Annot /Subtype /Widget /Parent 5 0 R /AS /Off /Rect [40 250 52 262] /P 3 0 R /AP << /N << /1 9 0 R /Off 9 0 R >> >> >>", + "<< /Type /Annot /Subtype /Widget /Parent 5 0 R /AS /Off /Rect [70 250 82 262] /P 3 0 R /AP << /N << /2 9 0 R /Off 9 0 R >> >> >>", + "<< /Type /XObject /Subtype /Form /BBox [0 0 12 12] /Length 0 >>\nstream\n\nendstream", + ]; + let dir = workdir("radio-export"); + std::fs::write(dir.join("form.pdf"), pdf_of(&objs)).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "form.pdf" }))["doc"].as_u64().unwrap(); + assert_eq!(ok(&mut a, "form_fields", json!({ "doc": doc }))["fields"][0]["options"], json!(["1", "2", "3"])); + for pick in ["1", "2", "3"] { + ok(&mut a, "form_fill", json!({ "doc": doc, "values": { "title": pick } })); + assert_eq!(ok(&mut a, "form_fields", json!({ "doc": doc }))["fields"][0]["value"], pick); + // The button drawn on is the one picked. + let f = a.session().get(pdfcraft_engine::DocId(doc)).unwrap().form[0].clone(); + let on: Vec = f.widgets.iter().enumerate().filter(|(_, w)| w.state.as_deref().is_some_and(|s| s != "Off")).map(|(i, _)| i).collect(); + assert_eq!(on, [pick.parse::().unwrap() - 1], "picked {pick}"); + } + // A number is the same option: `1` is the first button, not the state `/1`. + for pick in [1, 2, 3] { + ok(&mut a, "form_fill", json!({ "doc": doc, "values": { "title": pick } })); + assert_eq!(ok(&mut a, "form_fields", json!({ "doc": doc }))["fields"][0]["value"], pick.to_string(), "picked {pick}"); + } + // An on-state name the options do not list still selects its button. + ok(&mut a, "form_fill", json!({ "doc": doc, "values": { "title": "0" } })); + assert_eq!(ok(&mut a, "form_fields", json!({ "doc": doc }))["fields"][0]["value"], "1"); +} + +/// The ambiguous pair: the second button's on state is named `1` and the first button exports +/// `1`. The export value wins, however the number is written in JSON (`1`, `1.0`, `1e0`, `0.1e1`); +/// `-0` is `0`, the first button's state. A string is matched as written, so `"1.0"` names no +/// option, and neither does a number that is not whole (`1.5`) or past 2^53 (`1e300`): each is +/// refused with the options listed and the selection kept. +#[test] +fn form_fill_reads_a_radio_number_by_its_integer_spelling() { + let objs = [ + "<< /Type /Catalog /Pages 2 0 R /AcroForm 4 0 R >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 400 300] >>", + "<< /Type /Page /Parent 2 0 R /Annots [6 0 R 7 0 R 8 0 R] >>", + "<< /Fields [5 0 R] /DA (/Helv 0 Tf 0 g) >>", + "<< /FT /Btn /T (title) /Ff 49152 /Opt [(1) (2) (3)] /V /Off /Kids [6 0 R 7 0 R 8 0 R] >>", + "<< /Type /Annot /Subtype /Widget /Parent 5 0 R /AS /Off /Rect [10 250 22 262] /P 3 0 R /AP << /N << /0 9 0 R /Off 9 0 R >> >> >>", + "<< /Type /Annot /Subtype /Widget /Parent 5 0 R /AS /Off /Rect [40 250 52 262] /P 3 0 R /AP << /N << /1 9 0 R /Off 9 0 R >> >> >>", + "<< /Type /Annot /Subtype /Widget /Parent 5 0 R /AS /Off /Rect [70 250 82 262] /P 3 0 R /AP << /N << /2 9 0 R /Off 9 0 R >> >> >>", + "<< /Type /XObject /Subtype /Form /BBox [0 0 12 12] /Length 0 >>\nstream\n\nendstream", + ]; + let dir = workdir("radio-number-spelling"); + std::fs::write(dir.join("form.pdf"), pdf_of(&objs)).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "form.pdf" }))["doc"].as_u64().unwrap(); + // The values as JSON text, so each number keeps the spelling a client sends. + let fill = |lit: &str| -> Value { serde_json::from_str(&format!(r#"{{"doc": {doc}, "values": {{"title": {lit}}}}}"#)).unwrap() }; + let on = |a: &Automation| -> Vec { + let f = a.session().get(pdfcraft_engine::DocId(doc)).unwrap().form[0].clone(); + f.widgets.iter().enumerate().filter(|(_, w)| w.state.as_deref().is_some_and(|s| s != "Off")).map(|(i, _)| i).collect() + }; + for (lit, button) in [("2", 1), ("1", 0), ("3.0", 2), ("1.0", 0), ("2e0", 1), ("1e0", 0), ("0.1e1", 0), ("-0", 0), ("\"1\"", 0), ("2.000", 1)] { + // From another button, so a fill that does nothing would show. + ok(&mut a, "form_fill", fill(if button == 2 { "1" } else { "3" })); + ok(&mut a, "form_fill", fill(lit)); + assert_eq!(on(&a), [button], "{lit} selects button {}", button + 1); + } + ok(&mut a, "form_fill", fill("2")); + for lit in ["\"1.0\"", "1.5", "-1", "1e300", "18446744073709551615", "9007199254740993.0"] { + let e = err(&mut a, "form_fill", fill(lit)).to_string(); + assert!(e.contains("has no option") && e.contains("options: 1, 2, 3"), "{lit}: {e}"); + assert_eq!(on(&a), [1], "{lit} keeps the selection"); + } +} + +/// Export values written as decimals (`/Opt [(1.0) (2.0)]`, states `/0 /1`): a number is compared +/// with them as a number first, so `1`, `1.0` and `1e0` all pick the first button and `2.0` the +/// second, where `1.0` had become `"1"`, matched no export value and picked the button whose state +/// is `/1`. A string is still matched as written, and a number no export value equals is refused. +#[test] +fn form_fill_matches_a_radio_number_to_decimal_export_values() { + let objs = [ + "<< /Type /Catalog /Pages 2 0 R /AcroForm 4 0 R >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 400 300] >>", + "<< /Type /Page /Parent 2 0 R /Annots [6 0 R 7 0 R] >>", + "<< /Fields [5 0 R] /DA (/Helv 0 Tf 0 g) >>", + "<< /FT /Btn /T (rate) /Ff 49152 /Opt [(1.0) (2.0)] /V /Off /Kids [6 0 R 7 0 R] >>", + "<< /Type /Annot /Subtype /Widget /Parent 5 0 R /AS /Off /Rect [10 250 22 262] /P 3 0 R /AP << /N << /0 8 0 R /Off 8 0 R >> >> >>", + "<< /Type /Annot /Subtype /Widget /Parent 5 0 R /AS /Off /Rect [40 250 52 262] /P 3 0 R /AP << /N << /1 8 0 R /Off 8 0 R >> >> >>", + "<< /Type /XObject /Subtype /Form /BBox [0 0 12 12] /Length 0 >> +stream + +endstream", + ]; + let dir = workdir("radio-decimal-exports"); + std::fs::write(dir.join("form.pdf"), pdf_of(&objs)).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "form.pdf" }))["doc"].as_u64().unwrap(); + let fill = |lit: &str| -> Value { serde_json::from_str(&format!(r#"{{"doc": {doc}, "values": {{"rate": {lit}}}}}"#)).unwrap() }; + let on = |a: &Automation| -> Vec { + let f = a.session().get(pdfcraft_engine::DocId(doc)).unwrap().form[0].clone(); + f.widgets.iter().enumerate().filter(|(_, w)| w.state.as_deref().is_some_and(|s| s != "Off")).map(|(i, _)| i).collect() + }; + for (lit, button) in [("1.0", 0), ("2.0", 1), ("1", 0), ("2", 1), ("1e0", 0), ("\"1.0\"", 0), ("\"2.0\"", 1)] { + // From the other button, so a fill that does nothing would show. + ok(&mut a, "form_fill", fill(if button == 0 { "2.0" } else { "1.0" })); + ok(&mut a, "form_fill", fill(lit)); + assert_eq!(on(&a), [button], "{lit} selects button {}", button + 1); + } + let e = err(&mut a, "form_fill", fill("3")).to_string(); + assert!(e.contains("has no option"), "{e}"); +} + +/// Past 2^53 a float no longer holds every integer, so `9007199254740992` and `9007199254740993` +/// are the same `f64`. A number there is matched only by its exact text, never compared as a +/// number with the export values: with `/Opt [(9007199254740992) (2)]` the number +/// `9007199254740993` names no option, nor does `9007199254740992` with +/// `/Opt [(9007199254740993) (2)]`. Each is refused and the selection kept, while the number +/// spelled as its export value still picks that button. +#[test] +fn form_fill_matches_a_radio_number_past_2_53_only_by_its_text() { + for (export, other) in [("9007199254740992", "9007199254740993"), ("9007199254740993", "9007199254740992")] { + let field = format!("<< /FT /Btn /T (big) /Ff 49152 /Opt [({export}) (2)] /V /Off /Kids [6 0 R 7 0 R] >>"); + let objs = [ + "<< /Type /Catalog /Pages 2 0 R /AcroForm 4 0 R >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 400 300] >>", + "<< /Type /Page /Parent 2 0 R /Annots [6 0 R 7 0 R] >>", + "<< /Fields [5 0 R] /DA (/Helv 0 Tf 0 g) >>", + field.as_str(), + "<< /Type /Annot /Subtype /Widget /Parent 5 0 R /AS /Off /Rect [10 250 22 262] /P 3 0 R /AP << /N << /0 8 0 R /Off 8 0 R >> >> >>", + "<< /Type /Annot /Subtype /Widget /Parent 5 0 R /AS /Off /Rect [40 250 52 262] /P 3 0 R /AP << /N << /1 8 0 R /Off 8 0 R >> >> >>", + "<< /Type /XObject /Subtype /Form /BBox [0 0 12 12] /Length 0 >> +stream + +endstream", + ]; + let dir = workdir(&format!("radio-past-2-53-{export}")); + std::fs::write(dir.join("form.pdf"), pdf_of(&objs)).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "form.pdf" }))["doc"].as_u64().unwrap(); + let fill = |lit: &str| -> Value { serde_json::from_str(&format!(r#"{{"doc": {doc}, "values": {{"big": {lit}}}}}"#)).unwrap() }; + let on = |a: &Automation| -> Vec { + let f = a.session().get(pdfcraft_engine::DocId(doc)).unwrap().form[0].clone(); + f.widgets.iter().enumerate().filter(|(_, w)| w.state.as_deref().is_some_and(|s| s != "Off")).map(|(i, _)| i).collect() + }; + ok(&mut a, "form_fill", fill(export)); + assert_eq!(on(&a), [0], "{export} selects the button exported as {export}"); + ok(&mut a, "form_fill", fill("2")); + let e = err(&mut a, "form_fill", fill(other)).to_string(); + assert!(e.contains("has no option"), "{other} with /Opt [({export}) (2)]: {e}"); + assert_eq!(on(&a), [1], "{other} keeps the selection"); + } +} + +/// `form_fill` with {"widget": n} turns on that radio button alone when others share its on +/// state (ISO 32000-2 Table 229, RadiosInUnison clear), and `form_fields` says which is on. +#[test] +fn form_fill_picks_one_of_radio_buttons_sharing_an_on_state() { + let objs = [ + "<< /Type /Catalog /Pages 2 0 R /AcroForm 4 0 R >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 400 300] >>", + "<< /Type /Page /Parent 2 0 R /Annots [6 0 R 7 0 R] >>", + "<< /Fields [5 0 R] /DA (/Helv 0 Tf 0 g) >>", + "<< /FT /Btn /T (pick) /Ff 49152 /V /Off /Kids [6 0 R 7 0 R] >>", + "<< /Type /Annot /Subtype /Widget /Parent 5 0 R /AS /Off /Rect [10 250 26 266] /P 3 0 R /AP << /N << /Yes 8 0 R /Off 8 0 R >> >> >>", + "<< /Type /Annot /Subtype /Widget /Parent 5 0 R /AS /Off /Rect [40 250 56 266] /P 3 0 R /AP << /N << /Yes 8 0 R /Off 8 0 R >> >> >>", + "<< /Length 0 >>\nstream\n\nendstream", + ]; + let dir = workdir("radio-shared-state"); + std::fs::write(dir.join("form.pdf"), pdf_of(&objs)).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "form.pdf" }))["doc"].as_u64().unwrap(); + ok(&mut a, "form_fill", json!({ "doc": doc, "values": { "pick": { "widget": 2 } } })); + let after = ok(&mut a, "form_fields", json!({ "doc": doc })); + assert_eq!((&after["fields"][0]["value"], &after["fields"][0]["on_widgets"]), (&json!("Yes"), &json!([2])), "{after}"); + for bad in [json!({ "widget": 0 }), json!({ "widget": 3 }), json!({ "widget": -1 }), json!({ "widget": "1" }), json!({})] { + assert!(matches!(a.call("form_fill", &json!({ "doc": doc, "values": { "pick": bad } })), Err(ToolError::InvalidArgs(_))), "{bad}"); + } +} + +/// `form_fill` ticks one kid of a check box whose kids have different on states by naming it +/// (ISO 32000-2 §12.7.5.2.3, Table 230), and `form_fields` lists the names and the ticked one. +#[test] +fn form_fill_ticks_one_kid_of_a_check_box_by_its_on_state() { + let objs = [ + "<< /Type /Catalog /Pages 2 0 R /AcroForm 4 0 R >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 400 300] >>", + "<< /Type /Page /Parent 2 0 R /Annots [6 0 R 7 0 R] >>", + "<< /Fields [5 0 R] /DA (/Helv 0 Tf 0 g) >>", + "<< /FT /Btn /T (pair) /V /Off /Kids [6 0 R 7 0 R] >>", + "<< /Type /Annot /Subtype /Widget /Parent 5 0 R /AS /Off /Rect [10 250 26 266] /P 3 0 R /AP << /N << /A 8 0 R /Off 8 0 R >> >> >>", + "<< /Type /Annot /Subtype /Widget /Parent 5 0 R /AS /Off /Rect [40 250 56 266] /P 3 0 R /AP << /N << /B 8 0 R /Off 8 0 R >> >> >>", + "<< /Length 0 >>\nstream\n\nendstream", + ]; + let dir = workdir("check-box-kids"); + std::fs::write(dir.join("form.pdf"), pdf_of(&objs)).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "form.pdf" }))["doc"].as_u64().unwrap(); + let before = ok(&mut a, "form_fields", json!({ "doc": doc })); + assert_eq!((&before["fields"][0]["options"], &before["fields"][0]["checked"]), (&json!(["A", "B"]), &Value::Null), "{before}"); + ok(&mut a, "form_fill", json!({ "doc": doc, "values": { "pair": "A" } })); + ok(&mut a, "form_fill", json!({ "doc": doc, "values": { "pair": "B" } })); + let after = ok(&mut a, "form_fields", json!({ "doc": doc })); + assert_eq!((&after["fields"][0]["value"], &after["fields"][0]["checked"]), (&json!(true), &json!("B")), "{after}"); + ok(&mut a, "form_fill", json!({ "doc": doc, "values": { "pair": false } })); + assert_eq!(ok(&mut a, "form_fields", json!({ "doc": doc }))["fields"][0]["checked"], Value::Null); +} + +/// The options `form_fields` reports fill back through `form_fill` to the kid they name, also +/// where one kid's `/Opt` export is the other kid's on state: B exports `(A)`, and filling `A` +/// still ticks the kid whose on state is A (ISO 32000-2 §12.7.5.2.3, Table 230). +#[test] +fn form_fill_round_trips_the_reported_options_of_a_check_box_with_a_crossed_export() { + let objs = [ + "<< /Type /Catalog /Pages 2 0 R /AcroForm 4 0 R >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 400 300] >>", + "<< /Type /Page /Parent 2 0 R /Annots [6 0 R 7 0 R] >>", + "<< /Fields [5 0 R] /DA (/Helv 0 Tf 0 g) >>", + "<< /FT /Btn /T (pair) /V /Off /Opt [(x) (A)] /Kids [6 0 R 7 0 R] >>", + "<< /Type /Annot /Subtype /Widget /Parent 5 0 R /AS /Off /Rect [10 250 26 266] /P 3 0 R /AP << /N << /A 8 0 R /Off 8 0 R >> >> >>", + "<< /Type /Annot /Subtype /Widget /Parent 5 0 R /AS /Off /Rect [40 250 56 266] /P 3 0 R /AP << /N << /B 8 0 R /Off 8 0 R >> >> >>", + "<< /Length 0 >> +stream + +endstream", + ]; + let dir = workdir("check-box-crossed-export-fill"); + std::fs::write(dir.join("form.pdf"), pdf_of(&objs)).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "form.pdf" }))["doc"].as_u64().unwrap(); + let before = ok(&mut a, "form_fields", json!({ "doc": doc })); + let options = before["fields"][0]["options"].clone(); + assert_eq!(options, json!(["A", "B"]), "{before}"); + for option in ["A", "B"] { + ok(&mut a, "form_fill", json!({ "doc": doc, "values": { "pair": option } })); + let after = ok(&mut a, "form_fields", json!({ "doc": doc })); + assert_eq!(after["fields"][0]["checked"], json!(option), "filling {option}: {after}"); + } +} + +/// `form_fields` on a check box's kids: one option per distinct on state, named by its `/Opt` +/// export unless two states share one, and `checked` follows the kid whose `/AS` is on, which +/// ISO 32000-2 §12.7.5.2.3 makes decide where `/AS` and `/V` differ. +#[test] +fn form_fields_lists_a_check_box_by_on_state_and_reports_the_kid_that_is_on() { + // Each kid is (on state, /AS). + let listed = |name: &str, field: &str, (sa, asa): (&str, &str), (sb, asb): (&str, &str)| { + let kid = |on: &str, state: &str, x: u32| { + format!( + "<< /Type /Annot /Subtype /Widget /Parent 5 0 R /AS /{state} /Rect [{x} 250 {} 266] /P 3 0 R /AP << /N << /{on} 8 0 R /Off 8 0 R >> >> >>", + x + 16 + ) + }; + let (wa, wb) = (kid(sa, asa, 10), kid(sb, asb, 40)); + let objs = [ + "<< /Type /Catalog /Pages 2 0 R /AcroForm 4 0 R >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 400 300] >>", + "<< /Type /Page /Parent 2 0 R /Annots [6 0 R 7 0 R] >>", + "<< /Fields [5 0 R] /DA (/Helv 0 Tf 0 g) >>", + field, + &wa, + &wb, + "<< /Length 0 >>\nstream\n\nendstream", + ]; + let dir = workdir(name); + std::fs::write(dir.join("form.pdf"), pdf_of(&objs)).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "form.pdf" }))["doc"].as_u64().unwrap(); + let r = ok(&mut a, "form_fields", json!({ "doc": doc })); + (r["fields"][0]["options"].clone(), r["fields"][0]["checked"].clone()) + }; + // Kids sharing the on state Yes tick together, whatever their exports: not two options. + let shared = + listed("check-box-shared-state", "<< /FT /Btn /T (pair) /V /Off /Opt [(x) (y)] /Kids [6 0 R 7 0 R] >>", ("Yes", "Off"), ("Yes", "Off")); + assert_eq!(shared, (Value::Null, Value::Null)); + // /V names B, but A's /AS is on: A is the one ticked. + let differ = listed("check-box-as-wins", "<< /FT /Btn /T (pair) /V /B /Kids [6 0 R 7 0 R] >>", ("A", "A"), ("B", "Off")); + assert_eq!(differ, (json!(["A", "B"]), json!("A"))); + // Two on states with one export: the states name the options. + let same = listed("check-box-same-export", "<< /FT /Btn /T (pair) /V /B /Opt [(x) (x)] /Kids [6 0 R 7 0 R] >>", ("A", "Off"), ("B", "B")); + assert_eq!(same, (json!(["A", "B"]), json!("B"))); + // B's export is A's on state, so naming it would tick A: the states name the options. + let crossed = listed("check-box-crossed-export", "<< /FT /Btn /T (pair) /V /B /Opt [(x) (A)] /Kids [6 0 R 7 0 R] >>", ("A", "Off"), ("B", "B")); + assert_eq!(crossed, (json!(["A", "B"]), json!("B"))); +} + /// A form whose fields are only page widgets (the `/Fields` list is empty) still lists, fills and /// resets through the tools, and the leniency shows up as a repair note. #[test] @@ -2969,6 +3783,15 @@ fn bates_numbers_continue_across_files_and_report_preflight_and_permission_failu {"input":"a.pdf","output":"a-numbered.pdf"} ]); let mut a = auto(&dir); + for invalid_start in [json!(0), json!(-1), json!(1.5), json!("17")] { + let error = err(&mut a, "doc_bates_multi", json!({"files":files,"start_number":invalid_start})); + assert!(matches!(error, ToolError::InvalidArgs(_))); + } + let unicode = ok(&mut a, "doc_bates_multi", json!({"files":files,"prefix":"é".repeat(65),"dry_run":true})); + assert_eq!(unicode["files"].as_array().unwrap().len(), 2); + assert!(matches!(err(&mut a, "doc_bates_multi", json!({"files":files,"prefix":"CASE>>"})), ToolError::InvalidArgs(_))); + assert!(matches!(err(&mut a, "doc_bates_multi", json!({"files":files,"suffix":"A>>B"})), ToolError::InvalidArgs(_))); + let allocation = ok(&mut a, "doc_bates_multi", json!({"files":files,"start_number":50,"dry_run":true})); assert_eq!(allocation["files"][0]["first"], 50); assert_eq!(allocation["files"][0]["last"], 51); @@ -2987,15 +3810,24 @@ fn bates_numbers_continue_across_files_and_report_preflight_and_permission_failu let overflow = err(&mut a, "doc_bates_multi", json!({"files":[{"input":"b.pdf","output":"overflow.pdf"}],"start_number":u64::MAX})); assert!(overflow.to_string().contains("overflows")); assert!(!dir.join("overflow.pdf").exists()); - // A range may end at u64::MAX itself. - let at_max = ok(&mut a, "doc_bates_multi", json!({"files":[{"input":"b.pdf","output":"max.pdf"}],"start_number":u64::MAX - 1,"dry_run":true})); - assert_eq!(at_max["files"][0]["last"], u64::MAX); - for bad in - [json!({"start_number":-3}), json!({"start_number":2.5}), json!({"start_number":"7"}), json!({"prefix":"CASE>>"}), json!({"suffix":"A>>B"})] + std::fs::write(dir.join("one.pdf"), fixture(1)).unwrap(); + let max_number = + ok(&mut a, "doc_bates_multi", json!({"files":[{"input":"one.pdf","output":"one-numbered.pdf"}],"start_number":u64::MAX,"dry_run":true})); + assert_eq!(max_number["files"][0]["first"], u64::MAX); + assert_eq!(max_number["files"][0]["last"], u64::MAX); + + #[cfg(windows)] { - let mut args = json!({"files":[{"input":"b.pdf","output":"bad.pdf"}],"dry_run":true}); - args.as_object_mut().unwrap().extend(bad.as_object().unwrap().clone()); - assert!(a.call("doc_bates_multi", &args).is_err(), "{bad}"); + let collision = err( + &mut a, + "doc_bates_multi", + json!({"files":[ + {"input":"b.pdf","output":"Case.pdf"}, + {"input":"a.pdf","output":"CASE.pdf"} + ]}), + ); + assert!(collision.to_string().contains("same output path")); + assert!(!dir.join("Case.pdf").exists()); } use pdfcraft_engine::{Changes, Edit, Protection, Session}; @@ -3362,6 +4194,59 @@ trailer << /Root 1 0 R >> assert_eq!(saved.iter().find(|t| t.kind == "TH").and_then(|t| t.scope.as_deref()), Some("Column")); } +/// A layout table becomes plain blocks through `accessibility_layout_table_to_blocks` (#4105): the +/// table findings go, the content keeps its order, and the change is undone and saved. +#[test] +fn a_layout_table_becomes_blocks_through_automation() { + let dir = workdir("a11y-layout-table"); + let content = "/TD << /MCID 0 >> BDC 10 150 40 20 re f EMC /TD << /MCID 1 >> BDC 60 150 40 20 re f EMC"; + let pdf = format!( + "%PDF-1.7 +1 0 obj << /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 5 0 R >> endobj +2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj +3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Contents 4 0 R /StructParents 0 >> endobj +4 0 obj << /Length {} >> stream +{content} +endstream endobj +5 0 obj << /Type /StructTreeRoot /K 6 0 R /ParentTree 11 0 R >> endobj +6 0 obj << /S /Document /P 5 0 R /K [7 0 R] >> endobj +7 0 obj << /S /Table /P 6 0 R /K [8 0 R] >> endobj +8 0 obj << /S /TR /P 7 0 R /K [9 0 R 10 0 R] >> endobj +9 0 obj << /S /TD /P 8 0 R /Pg 3 0 R /K 0 >> endobj +10 0 obj << /S /TD /P 8 0 R /Pg 3 0 R /K 1 >> endobj +11 0 obj << /Nums [0 [9 0 R 10 0 R]] >> endobj +trailer << /Root 1 0 R >> +%%EOF", + content.len() + ); + std::fs::write(dir.join("layout.pdf"), pdf).unwrap(); + let mut a = auto(&dir); + let fd = ok(&mut a, "doc_open", json!({ "path": "layout.pdf" }))["doc"].as_u64().unwrap(); + let table_rules = |a: &mut Automation| -> Vec { + let r = ok(a, "accessibility_check", json!({ "doc": fd, "rules": ["table-summary", "table-headers"] })); + ["table-summary", "table-headers"] + .map(|id| r["results"].as_array().unwrap().iter().find(|x| x["rule"] == id).unwrap()["status"].as_str().unwrap().to_owned()) + .to_vec() + }; + assert_eq!(table_rules(&mut a), ["failed", "failed"]); + let types = |tags: &Value| tags["tags"].as_array().unwrap().iter().map(|t| t["type"].as_str().unwrap().to_owned()).collect::>(); + let tags = ok(&mut a, "accessibility_layout_table_to_blocks", json!({ "doc": fd, "object": 7, "generation": 0 })); + assert_eq!(types(&tags), ["Document", "Div", "Div", "P", "P"]); + assert_eq!(table_rules(&mut a), ["passed", "passed"]); + let content = |a: &mut Automation, object: u32| { + ok(a, "accessibility_tag_content", json!({ "doc": fd, "object": object, "generation": 0 }))["content"][0]["mcid"].clone() + }; + assert_eq!((content(&mut a, 9), content(&mut a, 10)), (json!(0), json!(1)), "each cell keeps its content"); + // Only a table: a cell is refused. + assert!(matches!(err(&mut a, "accessibility_layout_table_to_blocks", json!({ "doc": fd, "object": 9, "generation": 0 })), ToolError::Failed(_))); + assert_eq!(ok(&mut a, "edit_undo", json!({ "doc": fd }))["undone"], "Change tag properties"); + assert_eq!(types(&ok(&mut a, "accessibility_tags", json!({ "doc": fd }))), ["Document", "Table", "TR", "TD", "TD"]); + ok(&mut a, "edit_redo", json!({ "doc": fd })); + ok(&mut a, "doc_save", json!({ "doc": fd, "path": "blocks.pdf" })); + let saved = pdfcraft_cos::Document::open(std::sync::Arc::new(std::fs::read(dir.join("blocks.pdf")).unwrap())).unwrap(); + assert_eq!(pdfcraft_engine::a11y::tags(&saved).into_iter().map(|t| t.kind).collect::>(), ["Document", "Div", "Div", "P", "P"]); +} + #[test] fn marking_a_leaf_tag_as_an_artifact_through_automation_is_undoable_and_saved() { let dir = workdir("a11y-artifact"); @@ -4694,12 +5579,8 @@ fn rotating_a_field_through_tools() { ok(&mut a, "form_set_props", json!({ "doc": doc, "field": "City", "rotation": 90 })); let (rot, turned) = rect_of(&mut a); assert_eq!(rot, 90); - let (bw, bh) = (before[2] - before[0], before[3] - before[1]); - let (tw, th) = (turned[2] - turned[0], turned[3] - turned[1]); - assert!((tw - bh).abs() < 0.2 && (th - bw).abs() < 0.2, "swapped {before:?} -> {turned:?}"); - let center = |r: &[f64]| ((r[0] + r[2]) / 2.0, (r[1] + r[3]) / 2.0); - let (bc, tc) = (center(&before), center(&turned)); - assert!((bc.0 - tc.0).abs() < 0.2 && (bc.1 - tc.1).abs() < 0.2, "center moved {bc:?} -> {tc:?}"); + // The field stays where it is; only its content turns (#4296). + assert!(turned.iter().zip(&before).all(|(t, b)| (t - b).abs() < 0.2), "the field moved: {before:?} -> {turned:?}"); assert_eq!(ok(&mut a, "edit_undo", json!({ "doc": doc }))["undone"], "Change field properties"); let (rot, back) = rect_of(&mut a); assert_eq!((rot, back), (0, before)); @@ -4818,6 +5699,95 @@ fn redacting_through_tools() { assert!(matches!(a.call("redact_mark", &json!({ "doc": doc })), Err(ToolError::InvalidArgs(_)))); } +/// #2539 review: `redact_apply` says where the removed text is still held that applying leaves as +/// it is (a bookmark title and a note's text here), and changes neither. +#[test] +fn redact_apply_reports_places_that_still_hold_the_removed_text() { + let dir = workdir("redact-still-holding"); + std::fs::write(dir.join("memo.txt"), "Codename Bluefin report\nPublic line").unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_create", json!({ "from": "text", "path": "memo.txt" }))["doc"].as_u64().unwrap(); + ok(&mut a, "bookmark_add", json!({ "doc": doc, "title": "Bluefin plans", "page": 1 })); + ok(&mut a, "bookmark_add", json!({ "doc": doc, "title": "Annual figures", "page": 1 })); + ok(&mut a, "comment_add", json!({ "doc": doc, "page": 1, "type": "note", "at": [300, 400], "contents": "Ask about bluefin" })); + ok(&mut a, "comment_add", json!({ "doc": doc, "page": 1, "type": "note", "at": [300, 500], "contents": "Looks good" })); + ok(&mut a, "redact_mark", json!({ "doc": doc, "find": "Bluefin" })); + let r = ok(&mut a, "redact_apply", json!({ "doc": doc })); + assert_eq!(r["still_holding"], json!({ "bookmarks": 1, "attachments": 0, "comments": 1, "xmp": 0, "total": 2, "unsearched": false }), "{r}"); + // Said in the warnings too, with the other edit notices. + assert_eq!( + r["warnings"], + json!([ + "the redacted text also appears in 1 bookmark title and 1 comment, which redaction doesn't change (Remove hidden information removes them)" + ]), + "{r}" + ); + assert!(!page_text(&mut a, doc)[0].contains("Bluefin")); + let titles: Vec = ok(&mut a, "bookmark_list", json!({ "doc": doc }))["bookmarks"] + .as_array() + .unwrap() + .iter() + .map(|b| b["title"].as_str().unwrap().to_string()) + .collect(); + assert_eq!(titles, ["Bluefin plans", "Annual figures"], "reported, not changed"); + assert_eq!(r["identification"], json!({ "pdfa": "none", "pdfua": "none" }), "{r}"); +} + +/// #2539 review: when `redact_apply` removes the document's XMP packet because it names the +/// redacted word, `identification` says the PDF/A-2b and PDF/UA-1 identification it held was +/// written back, and the warnings say it too. +#[test] +fn redact_apply_reports_the_identification_written_back() { + let dir = workdir("redact-identification"); + let xmp = r#"Bluefin report"#; + let body = "BT /F1 18 Tf 20 150 Td (Codename BLUEFIN report) Tj ET"; + let content = format!("<< /Length {} >>\nstream\n{body}\nendstream", body.len()); + let metadata = format!("<< /Type /Metadata /Subtype /XML /Length {} >>\nstream\n{xmp}\nendstream", xmp.len()); + let objs = [ + "<< /Type /Catalog /Pages 2 0 R /Metadata 6 0 R >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>", + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Resources << /Font << /F1 4 0 R >> >> /Contents 5 0 R >>", + "<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>", + &content, + &metadata, + ]; + std::fs::write(dir.join("pdfa.pdf"), pdf_of(&objs)).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "pdfa.pdf" }))["doc"].as_u64().unwrap(); + ok(&mut a, "redact_mark", json!({ "doc": doc, "find": "BLUEFIN" })); + let r = ok(&mut a, "redact_apply", json!({ "doc": doc })); + assert_eq!(r["identification"], json!({ "pdfa": "kept", "pdfua": "kept" }), "{r}"); + assert_eq!( + r["warnings"], + json!([ + "the document's XMP metadata held the redacted text and was removed; its PDF/A and PDF/UA identification was written back on its own" + ]), + "{r}" + ); +} + +#[test] +fn applying_redactions_proposes_the_name_with_redacted() { + let dir = workdir("redact-save-as"); + std::fs::write(dir.join("memo.txt"), "Call 555-123-4567 today").unwrap(); + let mut a = auto(&dir); + let made = ok(&mut a, "doc_create", json!({ "from": "text", "path": "memo.txt" }))["doc"].as_u64().unwrap(); + ok(&mut a, "doc_save", json!({ "doc": made, "path": "contract.pdf" })); + let original = std::fs::read(dir.join("contract.pdf")).unwrap(); + let doc = ok(&mut a, "doc_open", json!({ "path": "contract.pdf" }))["doc"].as_u64().unwrap(); + ok(&mut a, "redact_mark", json!({ "doc": doc, "pattern": "phone" })); + let r = ok(&mut a, "redact_apply", json!({ "doc": doc })); + assert_eq!(r["save_as"], "contract_Redacted.pdf"); + ok(&mut a, "doc_save", json!({ "doc": doc, "path": r["save_as"] })); + assert_eq!(std::fs::read(dir.join("contract.pdf")).unwrap(), original, "the opened file is left as it was"); + let bytes = std::fs::read(dir.join("contract_Redacted.pdf")).unwrap(); + assert!(!bytes.windows(4).any(|w| w == b"4567"), "the copy has no trace of the number"); + // Redacting the redacted copy again proposes the same name, with one suffix. + let copy = ok(&mut a, "doc_open", json!({ "path": "contract_Redacted.pdf" }))["doc"].as_u64().unwrap(); + ok(&mut a, "redact_mark", json!({ "doc": copy, "find": "today" })); + assert_eq!(ok(&mut a, "redact_apply", json!({ "doc": copy }))["save_as"], "contract_Redacted.pdf"); +} + #[test] fn redacting_with_a_caller_regex_through_tools() { let dir = workdir("redact-regex"); @@ -5041,6 +6011,40 @@ fn removing_hidden_information_through_tools() { assert!(matches!(&refused, Err(ToolError::Failed(m)) if m.contains("can't be edited")), "{refused:?}"); } +/// `removed` counts what went from the listed categories. Flattening a field whose appearance +/// draws invisible text puts that text on the page, where hidden text (not listed) counts it; +/// a count over every category took it off what was removed (#5873). +#[test] +fn removed_hidden_information_counts_the_listed_categories() { + let appearance = "BT /Helv 12 Tf 2 4 Td (Ada) Tj 3 Tr (Secret) Tj ET"; + let appearance = format!( + "<< /Type /XObject /Subtype /Form /BBox [0 0 140 18] /Resources << /Font << /Helv 7 0 R >> >> /Length {} >>\nstream\n{appearance}\nendstream", + appearance.len() + ); + let objs = [ + "<< /Type /Catalog /Pages 2 0 R /AcroForm 4 0 R >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 400 300] >>", + "<< /Type /Page /Parent 2 0 R /Annots [5 0 R] /Resources << /Font << /Helv 7 0 R >> >> >>", + "<< /Fields [5 0 R] /DA (/Helv 12 Tf 0 g) /DR << /Font << /Helv 7 0 R >> >> >>", + "<< /Type /Annot /Subtype /Widget /FT /Tx /T (name) /V (Ada) /Rect [10 250 150 268] /P 3 0 R /AP << /N 6 0 R >> >>", + &appearance, + "<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>", + ]; + let dir = workdir("hidden-listed"); + std::fs::write(dir.join("form.pdf"), pdf_of(&objs)).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "form.pdf" }))["doc"].as_u64().unwrap(); + let count = |a: &mut Automation, c: &str| { + let info = ok(a, "doc_hidden_info", json!({ "doc": doc })); + info["categories"].as_array().unwrap().iter().find(|x| x["category"] == c).unwrap()["count"].as_u64().unwrap() + }; + assert_eq!((count(&mut a, "form-fields"), count(&mut a, "hidden-text")), (1, 0)); + let r = ok(&mut a, "doc_remove_hidden", json!({ "doc": doc, "categories": ["form-fields"] })); + assert_eq!(r["removed"], 1, "{r}"); + assert_eq!(count(&mut a, "form-fields"), 0); + assert_eq!(count(&mut a, "hidden-text"), 6, "the flattened appearance's invisible word is page content now"); +} + /// A one-page PDF whose text "TOP SECRET" sits under an opaque black rectangle. fn faux_redacted_fixture() -> Vec { helvetica_page("BT /F1 24 Tf 100 700 Td (TOP SECRET) Tj ET\n0 g\n90 688 220 32 re f") @@ -6183,6 +7187,80 @@ fn links_through_tools() { assert!(matches!(a.call("link_add", &json!({ "doc": doc, "page": 1, "rect": [0, 0, 50, 20] })), Err(ToolError::InvalidArgs(_)))); } +#[test] +fn link_zoom_through_tools() { + let dir = workdir("link-zoom"); + std::fs::write(dir.join("notes.txt"), "Contents").unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_create", json!({ "from": "text", "path": "notes.txt" }))["doc"].as_u64().unwrap(); + let link = |a: &mut Automation| ok(a, "link_list", json!({ "doc": doc }))["links"][0].clone(); + // A page link opens at Fit Page unless told otherwise. + ok(&mut a, "link_add", json!({ "doc": doc, "page": 1, "rect": [72, 300, 200, 320], "to_page": 1 })); + assert_eq!((link(&mut a)["to_page"].clone(), link(&mut a)["zoom"].clone()), (json!(1), json!("fit_page"))); + let index = link(&mut a)["index"].clone(); + // A zoom alone keeps the page; a page alone keeps the zoom. + ok(&mut a, "link_edit", json!({ "doc": doc, "page": 1, "index": index, "zoom": 200 })); + let l = link(&mut a); + assert_eq!((l["to_page"].clone(), l["zoom"].clone()), (json!(1), json!(200.0))); + assert_eq!(l["destination"]["zoom"], 2.0, "the viewer reads the zoom the link writes: {l}"); + ok(&mut a, "link_edit", json!({ "doc": doc, "page": 1, "index": index, "zoom": "fit_width" })); + ok(&mut a, "link_edit", json!({ "doc": doc, "page": 1, "index": index, "to_page": 1 })); + assert_eq!(link(&mut a)["zoom"], "fit_width"); + for (name, zoom) in [("actual", json!(100.0)), ("inherit", json!("inherit")), ("fit_visible", json!("fit_visible"))] { + ok(&mut a, "link_edit", json!({ "doc": doc, "page": 1, "index": index, "zoom": name })); + assert_eq!(link(&mut a)["zoom"], zoom, "{name}"); + } + // A zoom needs a page to show: refused with a web address, unknown, or not positive. + for args in [ + json!({ "doc": doc, "page": 1, "rect": [72, 400, 200, 420], "url": "https://example.org", "zoom": "fit_page" }), + json!({ "doc": doc, "page": 1, "rect": [72, 400, 200, 420], "zoom": "fit_page" }), + json!({ "doc": doc, "page": 1, "rect": [72, 400, 200, 420], "to_page": 1, "zoom": "sideways" }), + json!({ "doc": doc, "page": 1, "rect": [72, 400, 200, 420], "to_page": 1, "zoom": 0 }), + json!({ "doc": doc, "page": 1, "rect": [72, 400, 200, 420], "to_page": 1, "zoom": [2] }), + ] { + assert!(matches!(a.call("link_add", &args), Err(ToolError::InvalidArgs(_))), "{args}"); + } + ok(&mut a, "link_edit", json!({ "doc": doc, "page": 1, "index": index, "url": "https://example.org" })); + assert!(matches!(a.call("link_edit", &json!({ "doc": doc, "page": 1, "index": index, "zoom": 150 })), Err(ToolError::InvalidArgs(_)))); +} + +/// `link_edit` with a page and no zoom keeps the link's zoom, and with it the link's own `/XYZ` +/// factor, even outside the 1–6400% a chosen zoom is written within: 0.5% and 10000% stay so, in +/// link_list and in the viewer's destination. Naming the link's own page changes nothing, so its +/// position stays too. A zoom chosen anew is still kept within 1–6400%. +#[test] +fn link_edit_keeps_a_links_own_zoom_outside_the_range_it_writes() { + let (_base, root) = sandbox("link-own-zoom"); + let pdf = pdf_of(&[ + "<< /Type /Catalog /Pages 2 0 R >>", + "<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 /MediaBox [0 0 300 400] >>", + "<< /Type /Page /Parent 2 0 R /Annots [5 0 R 6 0 R 7 0 R] >>", + "<< /Type /Page /Parent 2 0 R >>", + "<< /Type /Annot /Subtype /Link /Rect [10 10 100 30] /Border [0 0 0] /Dest [3 0 R /XYZ null null 0.005] >>", + "<< /Type /Annot /Subtype /Link /Rect [10 50 100 70] /Border [0 0 0] /Dest [3 0 R /XYZ null null 100] >>", + "<< /Type /Annot /Subtype /Link /Rect [10 90 100 110] /Border [0 0 0] /Dest [3 0 R /XYZ 72 300 2] >>", + ]); + std::fs::write(root.join("zooms.pdf"), pdf).unwrap(); + let mut a = auto(&root); + let doc = ok(&mut a, "doc_open", json!({ "path": "zooms.pdf" }))["doc"].as_u64().unwrap(); + let link = |a: &mut Automation, index: u64| { + ok(a, "link_list", json!({ "doc": doc }))["links"].as_array().unwrap().iter().find(|l| l["index"] == index).unwrap().clone() + }; + for (index, percent, factor) in [(1, 0.5, 0.005), (2, 10000.0, 100.0)] { + ok(&mut a, "link_edit", json!({ "doc": doc, "page": 1, "index": index, "to_page": 2 })); + let l = link(&mut a, index); + assert_eq!((l["to_page"].clone(), l["zoom"].clone()), (json!(2), json!(percent)), "{l}"); + let shown = l["destination"]["zoom"].as_f64().unwrap(); + assert!((shown - factor).abs() <= factor * 1e-6, "the viewer's zoom is the link's own: {l}"); + } + ok(&mut a, "link_edit", json!({ "doc": doc, "page": 1, "index": 3, "to_page": 1 })); + let l = link(&mut a, 3); + // `/XYZ 72 300` on a 400 pt page: 72 from the left and 100 from the top, as the tools measure. + assert_eq!((&l["zoom"], &l["destination"]["left"], &l["destination"]["top"]), (&json!(200.0), &json!(72.0), &json!(100.0)), "{l}"); + ok(&mut a, "link_edit", json!({ "doc": doc, "page": 1, "index": 1, "zoom": 10000 })); + assert_eq!(link(&mut a, 1)["zoom"], json!(6400.0)); +} + /// #4168: a Launch action whose file is a PDF (here with Windows `\` separators, relative to the /// source as a remote link is) opens that PDF in PdfCraft through link_open, within the root; one /// naming a program is listed but never followed. @@ -6210,6 +7288,40 @@ fn a_launch_link_to_a_pdf_opens_it_like_a_remote_link() { assert_eq!(a.session().docs().len(), before, "nothing was opened or launched"); } +/// A bookmark's action is reported as a link's target is, and only a go-to names a page here: a +/// remote go-to's `[1 /Fit]` is the other file's page 2, so doc_info and bookmark_list give this +/// file's no page for it. +#[test] +fn bookmarks_report_their_actions_and_a_remote_go_to_names_no_page_here() { + let (_base, root) = sandbox("bookmark-actions"); + let pdf = pdf_of(&[ + "<< /Type /Catalog /Pages 2 0 R /Outlines 5 0 R >>", + "<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 /MediaBox [0 0 300 400] >>", + "<< /Type /Page /Parent 2 0 R >>", + "<< /Type /Page /Parent 2 0 R >>", + "<< /Type /Outlines /First 6 0 R /Last 8 0 R /Count 3 >>", + "<< /Title (Web) /Parent 5 0 R /Next 7 0 R /A << /S /URI /URI (https://example.org/manual) >> >>", + "<< /Title (Other file) /Parent 5 0 R /Prev 6 0 R /Next 8 0 R /A << /S /GoToR /F (other.pdf) /D [1 /Fit] >> >>", + "<< /Title (Here) /Parent 5 0 R /Prev 7 0 R /A << /S /GoTo /D [4 0 R /Fit] >> >>", + ]); + std::fs::write(root.join("bookmarks.pdf"), pdf).unwrap(); + let mut a = auto(&root); + let doc = ok(&mut a, "doc_open", json!({ "path": "bookmarks.pdf" }))["doc"].as_u64().unwrap(); + let info = ok(&mut a, "doc_info", json!({ "doc": doc })); + let outline = &info["outline"]; + assert_eq!((&outline[0]["page"], &outline[0]["action"]), (&Value::Null, &json!({ "uri": "https://example.org/manual" })), "{info}"); + assert_eq!(outline[1]["page"], Value::Null, "{info}"); + assert_eq!( + (&outline[1]["action"]["action"], &outline[1]["action"]["remote_document"], &outline[1]["action"]["destination"]["page"]), + (&json!("GoToR"), &json!("other.pdf"), &json!(2)), + "{info}" + ); + assert_eq!((&outline[2]["page"], &outline[2]["action"]), (&json!(2), &Value::Null), "{info}"); + let marks = ok(&mut a, "bookmark_list", json!({ "doc": doc }))["bookmarks"].clone(); + let pages: Vec<_> = marks.as_array().unwrap().iter().map(|b| (b["title"].clone(), b["page"].clone())).collect(); + assert_eq!(pages, [(json!("Web"), Value::Null), (json!("Other file"), Value::Null), (json!("Here"), json!(2))], "{marks}"); +} + #[test] fn local_gotor_link_open_refuses_directories_devices_and_non_pdf_targets() { let (_base, root) = sandbox("gotor-not-a-file"); @@ -6754,6 +7866,7 @@ fn object_has_name(obj: &pdfcraft_cos::Object, name: &[u8]) -> bool { } } +/// Polygons, polylines and other drawing comments are added and listed through the tools. #[test] fn drawing_comments_through_tools() { let dir = workdir("drawing"); @@ -6783,11 +7896,35 @@ fn drawing_comments_through_tools() { a.call("comment_add", &json!({ "doc": doc, "page": 1, "type": "polygon", "points": [[1, 1]] })), Err(ToolError::Failed(_) | ToolError::InvalidArgs(_)) )); - // Replace Text: a strikeout over the found text and a grouped caret. - ok(&mut a, "comment_add", json!({ "doc": doc, "page": 2, "type": "replace", "find": "page 2", "contents": "second page" })); + // Replace Text: a strikeout over the found text, grouped under a caret holding the + // replacement (the group's primary, ISO 32000-2 §12.5.6.2). + ok(&mut a, "comment_add", json!({ "doc": doc, "page": 2, "type": "replace", "find": "page 2", "contents": "second page", "color": "green" })); let list = ok(&mut a, "comment_list", json!({ "doc": doc, "page": 2 })); - let strike = list["comments"].as_array().unwrap().iter().find(|c| c["type"] == "StrikeOut").unwrap().clone(); - assert_eq!(strike["replies"][0]["contents"], "second page", "the caret threads under the strikeout"); + let caret = list["comments"].as_array().unwrap().iter().find(|c| c["type"] == "Caret").unwrap().clone(); + // One comment, the caret (the group's primary, ISO 32000-2 §12.5.6.2), holding the new text; + // the strikeout is in its group, not a reply or a comment of its own. + assert_eq!(caret["contents"], "second page", "{list}"); + assert_eq!(caret["replies"], json!([]), "{list}"); + assert_eq!(caret["group"][0]["type"], "StrikeOut", "{list}"); + assert!(!list["comments"].as_array().unwrap().iter().any(|c| c["type"] == "StrikeOut"), "{list}"); + // Review on #5689 (CodeRabbit): the requested colour is the group's, so the primary has it. + // The list gives a group the primary's colour alone, so the strikeout's own (`/C`, what it is + // drawn in) is read from an XFDF export. + let pair = |a: &mut Automation| { + let list = ok(a, "comment_list", json!({ "doc": doc, "page": 2 })); + let c = list["comments"].as_array().unwrap().iter().find(|c| c["type"] == "Caret").unwrap().clone(); + ok(a, "doc_export_data", json!({ "doc": doc, "path": "pair.xfdf", "what": "comments" })); + let xfdf = std::fs::read_to_string(dir.join("pair.xfdf")).unwrap(); + let strike = xfdf.split("').next()).unwrap_or_default().to_owned(); + let color = strike.split("color=\"").nth(1).and_then(|v| v.split('"').next()).map(|v| json!(v)).unwrap_or(Value::Null); + (c["color"].clone(), color) + }; + assert_eq!(pair(&mut a), (json!("#2E9E5C"), json!("#2E9E5C")), "the requested green on both, not the caret's default blue: {caret}"); + // Restyling the caret restyles the proposal, its strikeout too, as one undo step. + ok(&mut a, "comment_edit", json!({ "doc": doc, "page": 2, "index": caret["index"], "color": "purple" })); + assert_eq!(pair(&mut a), (json!("#8A3FD1"), json!("#8A3FD1")), "the strikeout follows its caret"); + ok(&mut a, "edit_undo", json!({ "doc": doc })); + assert_eq!(pair(&mut a), (json!("#2E9E5C"), json!("#2E9E5C")), "one undo puts both back"); assert!(matches!(a.call("comment_add", &json!({ "doc": doc, "page": 2, "type": "replace", "find": "page 2" })), Err(ToolError::InvalidArgs(_)))); // A file attached as a comment. std::fs::write(dir.join("notes.txt"), b"remember").unwrap(); @@ -6836,6 +7973,8 @@ fn lines_and_corners_are_held_level_or_plumb_through_the_tool() { assert!(matches!(refused, Err(ToolError::InvalidArgs(_))), "ortho is for lines and corners: {refused:?}"); } +/// A visible signature drawn from an image, typed text or strokes through sign_document; bad +/// graphics are refused before anything is written. #[test] fn signing_with_an_image_typed_or_drawn_graphic_through_tools() { let dir = workdir("signing-graphic"); @@ -6905,6 +8044,111 @@ fn signing_with_an_image_typed_or_drawn_graphic_through_tools() { assert!(inked(60, 110), "the strokes are drawn"); } +/// Signing a dynamic form's signature box through sign_document runs its XFA signing scripts only +/// when asked (run_form_scripts): preSign first, so what it changes is signed, listed field by field, +/// and postSign after, left as unsaved changes; the result carries their messages. Not asked, they +/// don't run and the result says the box has them; a box without them reports nothing. +#[test] +fn signing_an_xfa_box_runs_its_signing_scripts_through_tools() { + let dir = workdir("xfa-sign-events"); + std::fs::write(dir.join("form.pdf"), pdfcraft_xfa::fixtures::shell(&pdfcraft_xfa::fixtures::signing_events_template())).unwrap(); + let mut a = auto(&dir); + ok(&mut a, "sign_id_create", json!({ "name": "Ada Lovelace", "key": "p256", "password": "secret12", "path": "ada.p12" })); + let doc = ok(&mut a, "doc_open", json!({ "path": "form.pdf" }))["doc"].as_u64().unwrap(); + let fields = ok(&mut a, "form_fields", json!({ "doc": doc })); + let name = |n: &str| { + fields["fields"] + .as_array() + .unwrap() + .iter() + .filter_map(|f| f["name"].as_str()) + .find(|f| *f == n || f.ends_with(&format!(".{n}[0]"))) + .unwrap() + .to_string() + }; + // Not asked: the scripts don't run, and the result says the box has them. + let r = ok(&mut a, "sign_document", json!({ "doc": doc, "id": "ada.p12", "password": "secret12", "field": name("sign"), "out": "unasked.pdf" })); + assert_eq!(r["form_scripts"]["skipped"], json!(true), "{r}"); + let unasked = ok(&mut a, "doc_open", json!({ "path": "unasked.pdf" }))["doc"].as_u64().unwrap(); + let unasked = ok(&mut a, "form_fields", json!({ "doc": unasked })); + assert!( + unasked["fields"] + .as_array() + .unwrap() + .iter() + .any(|f| f["name"].as_str().is_some_and(|n| n == "when" || n.ends_with(".when[0]")) && f["value"].is_null()), + "{unasked}" + ); + let doc = ok(&mut a, "doc_open", json!({ "path": "form.pdf" }))["doc"].as_u64().unwrap(); + // A bad argument is refused before the preSign scripts run. + let bad = a.call( + "sign_document", + &json!({ "doc": doc, "id": "ada.p12", "password": "secret12", "field": name("sign"), "out": "bad.pdf", "run_form_scripts": true, "reason": 5 }), + ); + assert!(matches!(bad, Err(ToolError::InvalidArgs(_))), "{bad:?}"); + let now = ok(&mut a, "form_fields", json!({ "doc": doc })); + assert!( + now["fields"] + .as_array() + .unwrap() + .iter() + .any(|f| f["name"].as_str().is_some_and(|n| n == "when" || n.ends_with(".when[0]")) && f["value"].is_null()), + "{now}" + ); + // A signed file that can't be written (`out` is a folder): not signed, and what preSign did + // is taken back. + std::fs::create_dir_all(dir.join("taken")).unwrap(); + let failed = a.call( + "sign_document", + &json!({ "doc": doc, "id": "ada.p12", "password": "secret12", "field": name("sign"), "out": "taken", "run_form_scripts": true }), + ); + assert!(failed.is_err(), "{failed:?}"); + let now = ok(&mut a, "form_fields", json!({ "doc": doc })); + assert!( + now["fields"] + .as_array() + .unwrap() + .iter() + .any(|f| f["name"].as_str().is_some_and(|n| n == "when" || n.ends_with(".when[0]")) && f["value"].is_null()), + "{now}" + ); + let r = ok( + &mut a, + "sign_document", + json!({ "doc": doc, "id": "ada.p12", "password": "secret12", "field": name("sign"), "out": "signed.pdf", "run_form_scripts": true }), + ); + let scripts = &r["form_scripts"]; + let changed = scripts["changed"].as_array().unwrap(); + assert_eq!(changed.len(), 1, "only what preSign changed: {r}"); + assert_eq!((&changed[0]["after"], &changed[0]["read_only"]), (&json!(["signed"]), &json!(true)), "{r}"); + assert_eq!((scripts["before_signing"].as_bool(), scripts["after_signing"].as_bool()), (Some(true), Some(true)), "{r}"); + assert_eq!(scripts["alerts"], json!([{ "message": "Signing notice", "title": null, "icon": "error", "buttons": "ok" }]), "{r}"); + assert_eq!(scripts["unsaved_changes"], json!(true), "the postSign change isn't in the signed file: {r}"); + // The signed file holds what the preSign script did, not what the postSign one did. + let signed = ok(&mut a, "doc_open", json!({ "path": "signed.pdf" }))["doc"].as_u64().unwrap(); + let values = ok(&mut a, "form_fields", json!({ "doc": signed })); + let value = |n: &str| { + values["fields"] + .as_array() + .unwrap() + .iter() + .find(|f| f["name"].as_str().is_some_and(|f| f == n || f.ends_with(&format!(".{n}[0]")))) + .map(|f| f["value"].clone()) + }; + assert_eq!(value("when"), Some(json!("signed")), "{values}"); + assert_eq!(value("after"), Some(json!(null)), "{values}"); + let list = ok(&mut a, "sign_list", json!({ "doc": signed })); + assert!(list["signatures"].as_array().unwrap().iter().any(|s| s["signed"] == true && s["status"] == "unknown"), "intact: {list}"); + // A box without signing scripts: nothing reported. + let r = ok( + &mut a, + "sign_document", + json!({ "doc": signed, "id": "ada.p12", "password": "secret12", "field": name("plain"), "out": "both.pdf", "run_form_scripts": true }), + ); + assert!(r.get("form_scripts").is_none(), "{r}"); + let _ = std::fs::remove_dir_all(&dir); +} + /// Signing a laid-out XFA form's signature box through the tools signs the field named by its /// SOM path, which is how another viewer, laying the form out again, finds the signature; the /// signed file reopens with the signature intact and on that field. @@ -6927,6 +8171,8 @@ fn a_dynamic_xfa_forms_signature_box_is_signed_under_its_som_path() { let _ = std::fs::remove_dir_all(&dir); } +/// Digital IDs are created and opened, documents signed with them and their signatures validated, +/// all through the tools. #[test] fn named_layer_verdicts_are_reported_and_rendered_through_tools() { use pdfcraft_cos::{Dict, Document, Lexer, ObjRef, Object, SaveOptions, Stream}; @@ -7136,6 +8382,36 @@ fn signing_with_translated_labels_through_tools() { assert!(stamped.contains("(Genehmigt)") && !stamped.contains("Reason") && !stamped.contains("Date"), "{stamped}"); } +#[test] +fn optimizing_discards_an_embedded_search_index_through_tools() { + use pdfcraft_cos::{Dict, Object}; + let dir = workdir("optimize-search-index"); + let mut cos = pdfcraft_cos::Document::open(std::sync::Arc::new(fixture(1))).unwrap(); + let mut info = Dict::new(); + for name in ["IBM-ODIndexes", "Illustrator"] { + let mut data = Dict::new(); + data.set(b"Private".to_vec(), Object::Int(1)); + info.set(name.as_bytes().to_vec(), Object::Dict(data)); + } + let root = cos.root().unwrap(); + cos.update_dict(root, |c| c.set(b"PieceInfo".to_vec(), Object::Dict(info))).unwrap(); + std::fs::write(dir.join("indexed.pdf"), pdfcraft_cos::write_full(&cos, &pdfcraft_cos::SaveOptions::default()).unwrap()).unwrap(); + let piece_info = |name: &str| { + let d = pdfcraft_cos::Document::open(std::sync::Arc::new(std::fs::read(dir.join(name)).unwrap())).unwrap(); + let cat = d.get(d.root().unwrap()).as_dict().cloned().unwrap(); + let info = d.dict(cat.get(b"PieceInfo").unwrap()).unwrap(); + info.iter().map(|(k, _)| String::from_utf8_lossy(k).into_owned()).collect::>() + }; + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "indexed.pdf" }))["doc"].as_u64().unwrap(); + let kept = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "kept.pdf" })); + assert_eq!(kept["search_indexes"], 0, "{kept}"); + assert_eq!(piece_info("kept.pdf"), ["IBM-ODIndexes", "Illustrator"]); + let r = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "small.pdf", "discard_search_index": true })); + assert_eq!(r["search_indexes"], 1, "{r}"); + assert_eq!(piece_info("small.pdf"), ["Illustrator"]); +} + #[test] fn optimizing_through_tools() { let dir = workdir("optimize"); @@ -7209,6 +8485,32 @@ fn optimizing_through_tools() { assert!(matches!(a.call("doc_optimize", &json!({ "doc": doc, "path": "x.pdf", "discard": ["everything"] })), Err(ToolError::InvalidArgs(_)))); } +#[test] +fn optimizing_discards_form_actions_through_tools() { + let dir = workdir("optimize-form-actions"); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "a.pdf" }))["doc"].as_u64().unwrap(); + ok(&mut a, "form_add_field", json!({ "doc": doc, "page": 1, "type": "button", "rect": [20, 20, 120, 42], "name": "Send" })); + ok( + &mut a, + "form_set_actions", + json!({ "doc": doc, "field": "Send", "actions": [ + { "trigger": "mouse_up", "submit": "https://example.com/submit" }, + { "trigger": "mouse_down", "reset": [] }, + { "trigger": "mouse_enter", "javascript": "app.alert(1);" } + ] }), + ); + let kept = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "kept.pdf" })); + assert_eq!(kept["form_actions"], 0, "{kept}"); + let r = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "small.pdf", "discard_form_actions": true })); + assert_eq!(r["form_actions"], 2, "{r}"); + let small = ok(&mut a, "doc_open", json!({ "path": "small.pdf" }))["doc"].as_u64().unwrap(); + let fields = ok(&mut a, "form_fields", json!({ "doc": small })); + assert!(fields.to_string().contains("Send"), "the button stays: {fields}"); + let actions = ok(&mut a, "form_actions", json!({ "doc": small, "field": "Send" })); + assert_eq!(actions["actions"], json!([{ "trigger": "mouse_enter", "javascript": "app.alert(1);" }]), "{actions}"); +} + /// #3227 review: retaining the version, a catalog `/Version` below the 1.5 an object stream /// needs is not what viewers read: the header is raised, and the later of the two is reported. #[test] @@ -7229,6 +8531,35 @@ fn retained_pdf_version_reports_the_header_raised_for_object_streams() { assert_eq!(r["pdf_version"].as_str(), Some("1.5"), "{r}"); } +/// #3227: retaining the version, the later of the written header and the catalog `/Version` is +/// reported: a catalog below the header cannot hide it, and equal versions are reported as is. +#[test] +fn retained_pdf_version_is_the_later_of_header_and_catalog() { + let dir = workdir("optimize-retain-version-later"); + let mut a = auto(&dir); + let blank = ok(&mut a, "doc_create", json!({ "from": "blank", "width": 300, "height": 400 }))["doc"].as_u64().unwrap(); + ok(&mut a, "doc_optimize", json!({ "doc": blank, "path": "v14.pdf", "compatibility": "1.4" })); + let mut cos = pdfcraft_cos::Document::open(std::sync::Arc::new(std::fs::read(dir.join("v14.pdf")).unwrap())).unwrap(); + let root = cos.root().unwrap(); + for (label, header, catalog, saved, reported) in [ + ("higher-header", "1.7", "1.4", "1.7", "1.7"), + // PDF versions compare as major/minor numbers: 2.0 is newer than 1.7. + // Retain keeps the header. The catalog /Version only sets the reported version. + ("higher-catalog", "1.7", "2.0", "1.7", "2.0"), + ("equal", "1.4", "1.4", "1.4", "1.4"), + ] { + cos.update_dict(root, |c| c.set(b"Version".to_vec(), pdfcraft_cos::Object::name(catalog))).unwrap(); + cos.set_version(header); + let input = pdfcraft_cos::write_full(&cos, &pdfcraft_cos::SaveOptions { object_streams: false, ..Default::default() }).unwrap(); + std::fs::write(dir.join(format!("{label}-input.pdf")), input).unwrap(); + let doc = ok(&mut a, "doc_open", json!({ "path": format!("{label}-input.pdf") }))["doc"].as_u64().unwrap(); + let out = format!("{label}-output.pdf"); + let r = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": out, "compatibility": "retain", "object_streams": false })); + assert!(std::fs::read(dir.join(&out)).unwrap().starts_with(format!("%PDF-{saved}\n").as_bytes()), "{label}: written header"); + assert_eq!(r["pdf_version"].as_str(), Some(reported), "{label}: {r}"); + } +} + /// `data` as LZW with one 9-bit literal code per byte (clear-table first, EOD last): valid for /// fewer than 250 bytes, before the code width grows. fn lzw_literals(data: &[u8]) -> Vec { @@ -8227,6 +9558,22 @@ fn action_run_keeps_outputs_with_duplicate_input_basenames() { assert_eq!(std::fs::read(first_path).unwrap(), first_bytes, "an earlier output must not be replaced"); } +#[cfg(unix)] +#[test] +fn action_run_never_replaces_an_entry_that_exists_check_misses() { + let dir = workdir("action-noclobber"); + std::fs::create_dir_all(dir.join("results")).unwrap(); + std::fs::write(dir.join("report.pdf"), fixture(1)).unwrap(); + // A dangling link is invisible to `exists()` but is still an occupied name. + std::os::unix::fs::symlink(dir.join("nowhere.pdf"), dir.join("results/report.pdf")).unwrap(); + let mut a = auto(&dir); + let args = json!({ "steps": [{ "step": "set_title", "arg": "Processed" }], "paths": ["report.pdf"], "folder": "results" }); + let result = ok(&mut a, "action_run", args); + let output = result["files"][0]["output"].as_str().unwrap(); + assert!(output.ends_with("report (2).pdf"), "{output}"); + assert!(dir.join("results/report.pdf").symlink_metadata().unwrap().file_type().is_symlink(), "the existing entry must stay"); +} + #[test] fn pdfa_through_tools() { let dir = workdir("pdfa"); @@ -8521,6 +9868,119 @@ BT /F1 12 Tf 72 640 Td (A paragraph further down.) Tj ET"; assert_eq!(paragraphs.len(), 2, "{xml}"); } +/// A one-page PDF (612 x 792 pt) drawing `content`, with the standard fonts `fonts` as /F1, /F2 +/// and so on, each in its built-in encoding. A font given `bfchar` pairs (" ") reads +/// its codes through that ToUnicode map, as a symbol font Word embeds does. +fn standard_fonts_pdf(content: &str, fonts: &[(&str, Option<&str>)]) -> Vec { + let names: String = (1..=fonts.len()).map(|i| format!("/F{i} {} 0 R ", i + 4)).collect(); + let mut objs = vec![ + "<< /Type /Catalog /Pages 2 0 R >>".to_string(), + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>".into(), + format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 612 792] /Contents 4 0 R /Resources << /Font << {names}>> >> >>"), + format!("<< /Length {} >>\nstream\n{content}\nendstream", content.len()), + ]; + let mut maps = Vec::new(); + for (font, map) in fonts { + let to_unicode = map.map_or(String::new(), |map| { + let cmap = format!( + "/CIDInit /ProcSet findresource begin 12 dict begin begincmap /CMapType 2 def 1 begincodespacerange <00> endcodespacerange \ +1 beginbfchar {map} endbfchar endcmap CMapName currentdict /CMap defineresource pop end end" + ); + maps.push(format!("<< /Length {} >>\nstream\n{cmap}\nendstream", cmap.len())); + format!(" /ToUnicode {} 0 R", fonts.len() + 4 + maps.len()) + }); + objs.push(format!("<< /Type /Font /Subtype /Type1 /BaseFont /{font}{to_unicode} >>")); + } + objs.extend(maps); + let mut pdf = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offsets.push(pdf.len()); + pdf.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes()); + } + let xref = pdf.len(); + pdf.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offsets { + pdf.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + pdf.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes()); + pdf +} + +/// The text of each paragraph that holds any in `word/document.xml` of the .docx at `path`. +fn word_paragraph_texts(path: &Path) -> Vec { + let zip = std::fs::read(path).unwrap(); + let mut at = 0; + let mut xml = String::new(); + while zip[at..].starts_with(b"PK\x03\x04") { + let size = u32::from_le_bytes(zip[at + 18..at + 22].try_into().unwrap()) as usize; + let len = u16::from_le_bytes(zip[at + 26..at + 28].try_into().unwrap()) as usize; + let data = at + 30 + len; + if &zip[at + 30..data] == b"word/document.xml" { + std::io::Read::read_to_string(&mut flate2::read::DeflateDecoder::new(&zip[data..data + size]), &mut xml).unwrap(); + break; + } + at = data + size; + } + xml.split("") + .skip(1) + .map(|p| { + let p = p.split_once("").map_or(p, |(inside, _)| inside); + p.split("").skip(1).filter_map(|t| t.split_once("").map(|(t, _)| t)).collect::() + }) + .filter(|text| !text.is_empty()) + .collect() +} + +/// #4158: a sentence that wraps onto the next lines after a bold name and a slanted one reaches +/// Word as one paragraph, though reading splits each of its lines where the style changes. +#[test] +fn exporting_to_word_keeps_a_wrapped_sentence_in_one_paragraph() { + let dir = workdir("wrapped-sentence-word"); + let content = "BT /F1 10 Tf 72 700 Td (The parties to this lease are the owner, ) Tj /F2 10 Tf (Harbour Lane Properties) Tj \ +/F1 10 Tf (, of 4 Quay Street, and the) Tj ET \ +BT /F1 10 Tf 72 686 Td (tenant named in the schedule, ) Tj /F3 10 Tf (Blue Finch Studio) Tj \ +/F1 10 Tf (, who agree as follows to rent the upper) Tj ET \ +BT /F1 10 Tf 72 672 Td (rooms for the whole of the term.) Tj ET \ +BT /F1 10 Tf 72 640 Td (Rent is paid monthly in advance.) Tj ET"; + let fonts = [("Helvetica", None), ("Helvetica-Bold", None), ("Helvetica-Oblique", None)]; + std::fs::write(dir.join("lease.pdf"), standard_fonts_pdf(content, &fonts)).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "lease.pdf" }))["doc"].as_u64().unwrap(); + ok(&mut a, "doc_export_office", json!({ "doc": doc, "path": "lease.docx" })); + let sentence = "The parties to this lease are the owner, Harbour Lane Properties, of 4 Quay Street, and the tenant named in the \ +schedule, Blue Finch Studio, who agree as follows to rent the upper rooms for the whole of the term."; + assert_eq!(word_paragraph_texts(&dir.join("lease.docx")), [sentence, "Rent is paid monthly in advance."]); +} + +/// A bulleted list drawn as Word draws one, each bullet in the Symbol font (read as U+F0B7, as +/// Word maps it) a tab stop before its item's text, reaches Word as one paragraph per item and +/// not as a two-column table. +#[test] +fn exporting_to_word_keeps_a_bulleted_list_as_paragraphs() { + let dir = workdir("bulleted-list-word"); + let mut content = String::from("BT /F1 10 Tf 72 700 Td (The tenant looks after:) Tj ET\n"); + for (n, item) in ["the stairs and the landing;", "the shared yard and its gate;", "the bins, every week."].iter().enumerate() { + let y = 686 - 14 * n; + content.push_str(&format!("BT /F2 10 Tf 90 {y} Td (\\267) Tj ET BT /F1 10 Tf 108 {y} Td ({item}) Tj ET\n")); + } + content.push_str("BT /F1 10 Tf 72 630 Td (Rent is paid monthly in advance.) Tj ET"); + std::fs::write(dir.join("list.pdf"), standard_fonts_pdf(&content, &[("Helvetica", None), ("Symbol", Some(" "))])).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "list.pdf" }))["doc"].as_u64().unwrap(); + ok(&mut a, "doc_export_office", json!({ "doc": doc, "path": "list.docx" })); + assert_eq!( + word_paragraph_texts(&dir.join("list.docx")), + [ + "The tenant looks after:", + "\u{F0B7} the stairs and the landing;", + "\u{F0B7} the shared yard and its gate;", + "\u{F0B7} the bins, every week.", + "Rent is paid monthly in advance." + ] + ); +} + /// Hundreds of calculations run on opening and again after a change through the tools. The /// time limits are raised so that a busy machine does not stop them (the engine's tests check /// that the limits stop them). @@ -8670,6 +10130,46 @@ fn links_in_xfa_rich_text_are_listed_through_tools() { assert_eq!(urls(&mut a, doc), ["mailto:forms@example.com"], "one link after a save and a fresh open, not two"); } +/// What a date typed into an XFA date field leaves in the form's data, for each kind of picture: +/// what another viewer kept for the same typing on a synthetic form. A date one of the edit +/// picture's patterns reads is kept as an ISO date; anything else, ISO text included, as typed, +/// and nothing is refused. +#[test] +fn xfa_date_fields_keep_typed_dates_as_another_viewer_does() { + let dir = workdir("xfa-typed-dates"); + std::fs::write(dir.join("dates.pdf"), pdfcraft_xfa::fixtures::shell(&pdfcraft_xfa::fixtures::date_pictures_template())).unwrap(); + let mut a = auto(&dir); + // typed, then the data for display, edit, either and bare. + let seen = [ + ("2016-10-18", ["2016-10-18", "2016-10-18", "2016-10-18", "2016-10-18"]), + ("10/18/2016", ["10/18/2016", "2016-10-18", "10/18/2016", "10/18/2016"]), + ("20161018", ["20161018", "20161018", "2016-10-18", "20161018"]), + ("18OCT16", ["18OCT16", "18OCT16", "2016-10-18", "18OCT16"]), + ("26 10 18", ["26 10 18", "26 10 18", "26 10 18", "26 10 18"]), + ("xyz", ["xyz", "xyz", "xyz", "xyz"]), + ]; + for (typed, kept) in seen { + for (field, want) in ["display", "edit", "either", "bare"].into_iter().zip(kept) { + let doc = ok(&mut a, "doc_open", json!({ "path": "dates.pdf" }))["doc"].as_u64().unwrap(); + ok(&mut a, "form_fill", json!({ "doc": doc, "values": { field: typed } })); + ok(&mut a, "doc_export_data", json!({ "doc": doc, "path": "data.xml", "what": "fields" })); + let xml = std::fs::read_to_string(dir.join("data.xml")).unwrap(); + assert!(xml.contains(&format!("<{field}>{want}")), "{typed} in {field}: want {want}, got {xml}"); + ok(&mut a, "doc_close", json!({ "doc": doc, "discard_changes": true })); + } + } + // What the fields show: an ISO date in the field's own date pattern (MM/DD/YYYY, YYYYMMDD), + // read from the appearances flattening draws into the page. + let doc = ok(&mut a, "doc_open", json!({ "path": "dates.pdf" }))["doc"].as_u64().unwrap(); + ok(&mut a, "form_fill", json!({ "doc": doc, "values": { "edit": "2016-10-18", "either": "2016-10-18" } })); + ok(&mut a, "doc_flatten", json!({ "doc": doc, "comments": false, "fields": true })); + let text = ok(&mut a, "text_extract", json!({ "doc": doc }))["pages"][0]["text"].as_str().unwrap().to_string(); + assert!(text.contains("10/18/2016") && text.contains("20161018") && !text.contains("2016-10-18"), "{text}"); + let _ = std::fs::remove_dir_all(&dir); +} + +/// A dynamic XFA form opens laid out from its template through the tools, renders, takes values +/// and saves them. #[test] fn dynamic_xfa_forms_open_render_fill_and_save_through_tools() { let dir = workdir("xfa"); @@ -8746,7 +10246,8 @@ fn cut_stack_printing_through_tools() { ); assert_eq!(r["pages"], 5); let printed = ok(&mut a, "doc_open", json!({"path": "range.pdf"}))["doc"].as_u64().unwrap(); - assert_eq!(page_text(&mut a, printed), ["Page 10\nPage 4", "Page 8\nPage 2", "Page 6"]); + // Two pages side by side: their labels share a baseline, so a line. + assert_eq!(page_text(&mut a, printed), ["Page 10 Page 4", "Page 8 Page 2", "Page 6"]); assert_eq!(page_text(&mut a, doc), before); assert_eq!(ok(&mut a, "doc_info", json!({"doc": doc}))["document"]["dirty"], false); for duplex in ["long-edge", "short-edge"] { @@ -8769,6 +10270,49 @@ fn cut_stack_printing_through_tools() { let _ = std::fs::remove_dir_all(dir); } +#[test] +fn custom_pages_per_sheet_prints_the_grid_as_given() { + // Two narrow pages side by side on portrait A4: 2 columns by 1 row (#6551). + let dir = workdir("custom-grid"); + std::fs::write(dir.join("narrow.pdf"), fixture(4)).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({"path": "narrow.pdf"}))["doc"].as_u64().unwrap(); + for orientation in ["auto", "portrait"] { + let r = ok( + &mut a, + "doc_print", + json!({"doc": doc, "layout": "multiple", "cols": 2, "rows": 1, "paper": "A4", "orientation": orientation, "path": "grid.pdf"}), + ); + assert_eq!(r["sheets"], 2, "{orientation}"); + let printed = ok(&mut a, "doc_open", json!({"path": "grid.pdf"}))["doc"].as_u64().unwrap(); + let info = ok(&mut a, "doc_info", json!({"doc": printed})); + let sheet = &info["pages"][0]; + assert!(sheet["width"].as_f64().unwrap() < sheet["height"].as_f64().unwrap(), "portrait sheet with {orientation}: {sheet}"); + // Side by side: the second page level with the first, to its right. + let rows = ok(&mut a, "text_extract", json!({"doc": printed, "rect": [0, 0, 600, 850]})); + let rows: Vec<&str> = rows["pages"].as_array().unwrap().iter().map(|p| p["text"].as_str().unwrap().trim()).collect(); + assert_eq!(rows, ["Page 1 Page 2", "Page 3 Page 4"], "{orientation}"); + ok(&mut a, "doc_close", json!({"doc": printed})); + } + for bad in [ + json!({"cols": 2}), + json!({"rows": 1}), + json!({"cols": 0, "rows": 1}), + json!({"cols": 2, "rows": 100}), + json!({"cols": -1, "rows": 2}), + json!({"cols": 2, "rows": 1, "per_sheet": 2}), + json!({"cols": 2, "rows": 1, "layout": "fit"}), + ] { + let mut args = json!({"doc": doc, "layout": "multiple", "path": "refused.pdf"}); + for (k, v) in bad.as_object().unwrap() { + args[k] = v.clone(); + } + assert!(matches!(a.call("doc_print", &args), Err(ToolError::InvalidArgs(_))), "{bad}"); + } + assert!(!dir.join("refused.pdf").exists()); + let _ = std::fs::remove_dir_all(dir); +} + #[test] fn measurements_calibrate_draw_save_reopen_and_export() { let dir = workdir("measurements"); @@ -8914,6 +10458,94 @@ fn measurement_snap_tool_covers_all_targets() { let _ = std::fs::remove_dir_all(dir); } +/// An XFA button click through js_run returns only what the click produced, not the +/// alerts of an earlier fill. +#[test] +fn an_xfa_click_through_js_run_does_not_return_an_earlier_fills_alerts() { + let dir = workdir("xfa-click-output"); + std::fs::write(dir.join("scripted.pdf"), pdfcraft_xfa::fixtures::shell(&pdfcraft_xfa::fixtures::scripted_template())).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "scripted.pdf" }))["doc"].as_u64().unwrap(); + // 500 fails qty's validation, which shows its message. + ok(&mut a, "form_fill", json!({ "doc": doc, "values": { "qty": "500" } })); + let r = ok(&mut a, "js_run", json!({ "doc": doc, "script": "", "field": "hello" })); + assert_eq!(r["alerts"], json!([{ "message": "Hello 500", "title": null, "icon": "error", "buttons": "ok" }]), "{r}"); + let _ = std::fs::remove_dir_all(dir); +} + +/// form_set_script returns only the console lines its own edit produced, not those of an +/// earlier fill. +#[test] +fn form_set_script_does_not_return_an_earlier_fills_console_lines() { + let dir = workdir("set-script-output"); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "a.pdf" }))["doc"].as_u64().unwrap(); + for (i, name) in ["a", "b"].into_iter().enumerate() { + let y = 20 + 30 * i; + ok(&mut a, "form_add_field", json!({ "doc": doc, "page": 1, "type": "text", "rect": [20, y, 180, y + 22], "name": name })); + } + ok(&mut a, "form_set_script", json!({ "doc": doc, "field": "a", "event": "validate", "script": "console.println('validated ' + event.value);" })); + ok(&mut a, "form_fill", json!({ "doc": doc, "values": { "a": "7" } })); + let r = ok(&mut a, "form_set_script", json!({ "doc": doc, "field": "b", "event": "format", "script": "event.value = event.value;" })); + assert_eq!(r["console"], json!([]), "{r}"); + assert_eq!(r["errors"], json!([]), "{r}"); + let _ = std::fs::remove_dir_all(dir); +} + +/// js_run returns the output of the field scripts that its field changes ran, with the +/// script's own. +#[test] +fn js_run_returns_the_output_of_the_calculations_its_changes_ran() { + let dir = workdir("js-run-calc-output"); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "a.pdf" }))["doc"].as_u64().unwrap(); + for (i, name) in ["a", "b"].into_iter().enumerate() { + let y = 20 + 30 * i; + ok(&mut a, "form_add_field", json!({ "doc": doc, "page": 1, "type": "text", "rect": [20, y, 180, y + 22], "name": name })); + } + let calc = "console.println('calculated ' + getField('a').value); event.value = getField('a').value;"; + ok(&mut a, "form_set_script", json!({ "doc": doc, "field": "b", "event": "calculate", "script": calc })); + let r = ok(&mut a, "js_run", json!({ "doc": doc, "script": "getField('a').value = '4'; console.println('set');" })); + let console: Vec<&str> = r["console"].as_array().unwrap().iter().filter_map(Value::as_str).collect(); + assert!(console.first().is_some_and(|l| l.starts_with("set")), "{r}"); + assert!(console.iter().any(|l| l.starts_with("calculated 4")), "{r}"); + let _ = std::fs::remove_dir_all(dir); +} + +/// The tools that edit a document without going through form_fill's path report the values of +/// an XFA form that could not be written to its data in `warnings`, once, and leave nothing for +/// the next tool to report as its own. +#[test] +fn edits_report_the_xfa_data_they_could_not_write_once() { + let dir = workdir("xfa-data-warnings"); + // The data holds structured content where the price goes, so its value can't be written. + let tpl = pdfcraft_xfa::fixtures::scripted_template().replace( + "", + "

7

\n", + ); + std::fs::write(dir.join("data.pdf"), pdfcraft_xfa::fixtures::shell(&tpl)).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "data.pdf" }))["doc"].as_u64().unwrap(); + let structured = |out: &Value, key: &str| { + out[key].as_array().map_or(0, |w| w.iter().filter(|w| w.as_str().is_some_and(|w| w.contains("structured content"))).count()) + }; + let r = ok(&mut a, "js_set_document_script", json!({ "doc": doc, "name": "init", "script": "var ready = true;" })); + assert_eq!(structured(&r, "warnings"), 1, "{r}"); + assert_eq!(r["scripts"][0]["name"], "init", "{r}"); + let r = ok( + &mut a, + "form_set_actions", + json!({ "doc": doc, "field": "hello", "actions": [{ "trigger": "mouse_enter", "javascript": "var entered = true;" }] }), + ); + assert_eq!(structured(&r, "warnings"), 1, "{r}"); + let r = ok(&mut a, "js_run", json!({ "doc": doc, "script": "getField('qty').value = 3;" })); + assert_eq!(structured(&r, "warnings"), 1, "{r}"); + // form_set_script lists its edit's own warning in errors, and the earlier tools' are not repeated. + let r = ok(&mut a, "form_set_script", json!({ "doc": doc, "field": "qty", "event": "format", "script": "event.value = event.value;" })); + assert_eq!(structured(&r, "errors"), 1, "{r}"); + let _ = std::fs::remove_dir_all(dir); +} + #[test] fn growable_xfa_text_paginates_after_fill_save_and_print() { let dir = workdir("xfa-growable-text"); @@ -8947,6 +10579,8 @@ fn growable_xfa_text_paginates_after_fill_save_and_print() { let _ = std::fs::remove_dir_all(dir); } +/// A print button repeated on every page of an XFA form runs its own XFA click script (a print +/// request) through js_run, from each copy, before and after saving and reopening. #[test] fn repeated_xfa_print_buttons_work_through_tools_and_after_reopen() { let dir = workdir("xfa-print-buttons"); @@ -8974,6 +10608,93 @@ fn repeated_xfa_print_buttons_work_through_tools_and_after_reopen() { check(&mut a, doc); } +/// form_focus runs a field's XFA enter and exit scripts on every focus change; scripts that +/// change nothing leave no edit, a radio button's own scripts run for its widget, and unknown +/// fields, widgets and events are refused. +#[test] +fn xfa_focus_scripts_run_through_form_focus() { + let dir = workdir("xfa-focus"); + std::fs::write(dir.join("focus.pdf"), pdfcraft_xfa::fixtures::shell(&pdfcraft_xfa::fixtures::focus_template())).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "focus.pdf" }))["doc"].as_u64().unwrap(); + let value = |a: &mut Automation, n: &str| { + let f = ok(a, "form_fields", json!({ "doc": doc })); + f["fields"].as_array().unwrap().iter().find(|f| f["name"] == n).map(|f| f["value"].clone()).unwrap() + }; + for _ in 0..3 { + assert_eq!(ok(&mut a, "form_focus", json!({ "doc": doc, "field": "probe", "event": "enter" }))["changed"], true); + assert_eq!(ok(&mut a, "form_focus", json!({ "doc": doc, "field": "probe", "event": "exit" }))["changed"], true); + } + assert_eq!((value(&mut a, "enters"), value(&mut a, "exits")), (json!("3"), json!("3"))); + ok(&mut a, "edit_undo", json!({ "doc": doc })); + assert_eq!(value(&mut a, "exits"), "2", "each run is one undo step"); + // A script that only shows a message changes nothing: no edit (the document stays saved and + // what was undone can still be redone), and the message comes back. + ok(&mut a, "doc_save", json!({ "doc": doc, "path": "saved.pdf" })); + let quiet = ok(&mut a, "form_focus", json!({ "doc": doc, "field": "quiet", "event": "exit" })); + assert_eq!( + (quiet["changed"].clone(), quiet["alerts"].clone()), + (json!(false), json!([{ "message": "left quiet", "title": null, "icon": "error", "buttons": "ok" }])), + "{quiet}" + ); + let state = &ok(&mut a, "doc_info", json!({ "doc": doc }))["document"]; + assert_eq!((state["dirty"].clone(), state["redo"].clone()), (json!(false), json!("Run form script")), "{state}"); + // The second radio button has an enter script of its own. + let fields = ok(&mut a, "form_fields", json!({ "doc": doc })); + let group = fields["fields"].as_array().unwrap().iter().find(|f| f["type"] == "radio").unwrap()["name"].clone(); + assert_eq!(ok(&mut a, "form_focus", json!({ "doc": doc, "field": group, "event": "enter" }))["changed"], false); + assert_eq!(ok(&mut a, "form_focus", json!({ "doc": doc, "field": group, "event": "enter", "widget": 2 }))["changed"], true); + assert_eq!(value(&mut a, "picks"), "1"); + for bad in [ + json!({ "doc": doc, "field": "nowhere", "event": "enter" }), + json!({ "doc": doc, "field": group, "event": "enter", "widget": 3 }), + json!({ "doc": doc, "field": "probe", "event": "enter", "widget": 0 }), + json!({ "doc": doc, "field": "probe", "event": "click" }), + ] { + assert!(a.call("form_focus", &bad).is_err(), "{bad}"); + } +} + +/// form_focus refuses a field with no widget, which nothing can focus, instead of reporting a +/// focus change that ran nothing. +#[test] +fn form_focus_refuses_a_field_with_no_widget() { + let dir = workdir("focus-no-widget"); + let objs = [ + "<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [4 0 R 5 0 R] >> >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 300 400] >>", + "<< /Type /Page /Parent 2 0 R /Annots [4 0 R] >>", + "<< /Type /Annot /Subtype /Widget /FT /Tx /T (name) /Rect [60 60 200 80] /P 3 0 R >>", + "<< /FT /Tx /T (total) /V (3) >>", + ]; + let mut bytes = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offsets.push(bytes.len()); + bytes.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes()); + } + let xref = bytes.len(); + bytes.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offsets { + bytes.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + bytes.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes()); + std::fs::write(dir.join("total.pdf"), bytes).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "total.pdf" }))["doc"].as_u64().unwrap(); + let fields = ok(&mut a, "form_fields", json!({ "doc": doc })); + assert!(fields["fields"].as_array().unwrap().iter().any(|f| f["name"] == "total"), "the field without a widget is listed: {fields}"); + for args in [json!({ "doc": doc, "field": "total", "event": "enter" }), json!({ "doc": doc, "field": "total", "event": "enter", "widget": 1 })] { + let e = err(&mut a, "form_focus", args.clone()).to_string(); + assert!(e.contains("\"total\" has no widget to focus"), "{args}: {e}"); + } + assert_eq!( + ok(&mut a, "form_focus", json!({ "doc": doc, "field": "name", "event": "enter" }))["changed"], + false, + "a field with a widget still takes it" + ); +} + /// A dynamic XFA form's scripts run on open, on fills and for buttons through the tools, with undo, /// and `js_run` reports their messages. #[test] @@ -9089,6 +10810,38 @@ fn doc_info_describes_set_layer_links() { }) ); assert_eq!(info["layers"], json!([{ "name": "Red", "visible": true, "initial": true }, { "name": "Green", "visible": false, "initial": false }])); + // No /Order: the Layers panel presents none (ISO 32000-2 Table 99), though both are layers. + assert_eq!(info["layer_order"], json!([])); +} + +/// `doc_info`'s `layer_order` follows the default configuration's /Order (ISO 32000-2 Table 99): +/// a group by its 1-based position in `layers` and its name, a nested array one level deeper, +/// a label that starts one, and a group /Order does not list left out. +#[test] +fn doc_info_lists_the_layer_order_tree() { + let dir = workdir("layer-order"); + let pdf = pdf_of(&[ + "<< /Type /Catalog /Pages 2 0 R /OCProperties << /OCGs [4 0 R 5 0 R 6 0 R 7 0 R] /D << /Order [6 0 R [(Inks) 4 0 R [5 0 R]]] >> >> >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>", + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] >>", + "<< /Type /OCG /Name (Red) >>", + "<< /Type /OCG /Name (Green) >>", + "<< /Type /OCG /Name (Blue) >>", + "<< /Type /OCG /Name (Unlisted) >>", + ]); + std::fs::write(dir.join("order.pdf"), pdf).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "order.pdf" }))["doc"].as_u64().unwrap(); + let info = ok(&mut a, "doc_info", json!({ "doc": doc })); + assert_eq!( + info["layer_order"], + json!([ + { "depth": 0, "layer": 3, "name": "Blue" }, + { "depth": 0, "label": "Inks" }, + { "depth": 1, "layer": 1, "name": "Red" }, + { "depth": 2, "layer": 2, "name": "Green" }, + ]) + ); } mod close_argument_tests { @@ -9445,6 +11198,50 @@ fn thai_text_through_tools_saves_reopens_extracts_and_renders() { assert!(page_text(&mut a, reopened)[0].contains("แก้ไข น้ำ")); } +/// Thai typed into Fill & Sign text, a text box, a form field, a footer and a watermark (#2354) is +/// drawn from PdfCraft's Thai face, as Add text draws it: no tool warns of `?`, and after a save +/// each one's text is found as typed and the page renders. A build without a Thai face refuses +/// the text in each, saying what to do, rather than writing `?????`. +#[test] +fn thai_in_fill_sign_text_comments_fields_and_page_marks_is_drawn_and_found() { + let dir = workdir("thai-everywhere"); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({"path": "a.pdf"}))["doc"].as_u64().unwrap(); + let thai = "บุญฤทธิ์ น้ำ ปั้น"; + let field = ok(&mut a, "form_add_field", json!({"doc": doc, "page": 1, "type": "text", "rect": [20, 120, 220, 142]}))["field"] + .as_str() + .unwrap() + .to_string(); + let calls = [ + ("fill_sign_add", json!({"doc": doc, "page": 1, "type": "text", "at": [20, 40], "text": thai, "font_size": 14})), + ("comment_add", json!({"doc": doc, "page": 1, "type": "textbox", "rect": [20, 160, 200, 190], "contents": thai})), + ("form_fill", json!({"doc": doc, "values": {field.as_str(): thai}})), + ("doc_header_footer", json!({"doc": doc, "pages": [1], "footer_center": format!("{thai} <<1>>")})), + ("doc_watermark", json!({"doc": doc, "pages": [1], "text": thai})), + ]; + if pdfcraft_fonts::document_thai_font().is_none() { + for (tool, args) in calls { + let err = a.call(tool, &args).unwrap_err().to_string(); + assert!(err.contains("Thai") && err.contains("CRAFT_FONTS_DIR"), "{tool}: {err}"); + } + assert_eq!(ok(&mut a, "comment_list", json!({"doc": doc}))["comments"].as_array().map(Vec::len), Some(0)); + return; + } + for (tool, args) in calls { + let r = ok(&mut a, tool, args); + assert!(r.get("warnings").is_none(), "{tool}: {r}"); + } + ok(&mut a, "doc_save", json!({"doc": doc, "path": "thai-marks.pdf"})); + let reopened = ok(&mut a, "doc_open", json!({"path": "thai-marks.pdf"}))["doc"].as_u64().unwrap(); + let found = ok(&mut a, "text_find", json!({"doc": reopened, "query": "บุญฤทธิ์"})); + assert_eq!(found["count"], 5, "{found}"); + let rendered = a.call("page_render", &json!({"doc": reopened, "page": 1, "dpi": 72})).unwrap(); + assert!(rendered.iter().any(|c| matches!(c, Content::Png { data, .. } if !data.is_empty()))); + // Flattened, every one of them is page text, as typed. + ok(&mut a, "doc_flatten", json!({"doc": reopened})); + assert_eq!(page_text(&mut a, reopened)[0].matches(thai).count(), 5, "{:?}", page_text(&mut a, reopened)); +} + #[test] fn hebrew_text_through_tools_saves_reopens_extracts_and_renders() { let dir = workdir("hebrew-content"); @@ -9503,6 +11300,75 @@ fn mixed_objects_move_through_tools_with_generation_and_one_undo() { assert!(page_text(&mut a, reopened)[0].contains("Added label")); } +/// Edit a PDF's ⌘A, ⌘C / ⌘X and ⌘V headlessly: every object of a page copied, pasted into +/// another document and onto its own page (stepped down and right), and cut as one undo step. +#[test] +fn objects_copy_cut_and_paste_through_tools_within_and_across_documents() { + let dir = workdir("object-copy-paste"); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({"path":"a.pdf"}))["doc"].as_u64().unwrap(); + let other = ok(&mut a, "doc_open", json!({"path":"b.pdf"}))["doc"].as_u64().unwrap(); + ok(&mut a, "page_add_text", json!({"doc":doc,"page":1,"text":"Added label","rect":[20,20,130,45]})); + assert!(matches!(a.call("object_paste", &json!({"doc":doc,"page":1})), Err(ToolError::Failed(_))), "nothing copied yet"); + let listed = ok(&mut a, "object_list", json!({"doc":doc,"page":1})); + assert_eq!(listed["count"], 2); + let copied = ok(&mut a, "object_copy", json!({"doc":doc,"page":1,"all":true,"generation":listed["generation"]})); + assert_eq!(copied["copied"], 2); + let text = copied["text"].as_str().unwrap(); + assert!(text.contains("Added label") && text.contains("Page 1"), "{text}"); + assert_eq!(ok(&mut a, "object_list", json!({"doc":doc,"page":1})), listed, "a copy changes nothing"); + + // Into another document: the same objects, where they were, as its own page text. + let before = ok(&mut a, "object_list", json!({"doc":other,"page":2})); + assert_eq!(ok(&mut a, "object_paste", json!({"doc":other,"page":2}))["pasted"], 2); + let pasted = ok(&mut a, "object_list", json!({"doc":other,"page":2})); + assert_eq!(pasted["count"], 3); + let rect_of = + |list: &Value, text: &str| list["objects"].as_array().unwrap().iter().find(|o| o["text"] == text).map(|o| o["rect"].clone()).unwrap(); + assert_eq!(rect_of(&pasted, "Page 1"), rect_of(&listed, "Page 1")); + // The added text box comes as the page's own text, drawn inside the box it had. + let (label, box_) = (rect_of(&pasted, "Added label"), rect_of(&listed, "Added label")); + assert!((0..2).all(|i| label[i].as_f64() >= box_[i].as_f64()) && (2..4).all(|i| label[i].as_f64() <= box_[i].as_f64()), "{label} in {box_}"); + assert!(page_text(&mut a, other)[1].contains("Added label")); + ok(&mut a, "doc_save", json!({"doc":other,"path":"pasted.pdf"})); + let reopened = ok(&mut a, "doc_open", json!({"path":"pasted.pdf"}))["doc"].as_u64().unwrap(); + assert!(page_text(&mut a, reopened)[1].contains("Page 1")); + ok(&mut a, "edit_undo", json!({"doc":other})); + assert_eq!(ok(&mut a, "object_list", json!({"doc":other,"page":2}))["objects"], before["objects"], "one undo step"); + + // Onto its own page: 18 points down and to the right. + ok(&mut a, "object_paste", json!({"doc":doc,"page":1})); + let both = ok(&mut a, "object_list", json!({"doc":doc,"page":1})); + assert_eq!(both["count"], 4); + let original = rect_of(&listed, "Page 1"); + let moved = + both["objects"].as_array().unwrap().iter().filter(|o| o["text"] == "Page 1").map(|o| o["rect"].clone()).find(|r| *r != original).unwrap(); + for i in 0..4 { + assert!((moved[i].as_f64().unwrap() - original[i].as_f64().unwrap() - 18.0).abs() < 0.02, "{moved} {original}"); + } + + // Cut: copied, then deleted as one undo step. + let cut = ok(&mut a, "object_copy", json!({"doc":doc,"page":1,"objects":[{"kind":"text","index":1}],"cut":true})); + assert_eq!(cut["copied"], 1); + assert_eq!(ok(&mut a, "object_list", json!({"doc":doc,"page":1}))["count"], 3); + ok(&mut a, "edit_undo", json!({"doc":doc})); + assert_eq!(ok(&mut a, "object_list", json!({"doc":doc,"page":1}))["objects"], both["objects"]); + + for args in [ + json!({"doc":doc,"page":1,"all":true,"objects":[{"kind":"text","index":1}]}), + json!({"doc":doc,"page":1}), + json!({"doc":doc,"page":1,"objects":[{"kind":"text","index":99}]}), + json!({"doc":doc,"page":9,"all":true}), + ] { + assert!(a.call("object_copy", &args).is_err(), "{args}"); + } + assert!( + matches!(a.call("object_copy", &json!({"doc":doc,"page":1,"all":true,"generation":listed["generation"]})), Err(ToolError::Failed(_))), + "stale" + ); + assert!(a.call("object_paste", &json!({"doc":doc,"page":9})).is_err()); +} + #[test] fn object_move_rejects_invalid_nested_references_without_any_change() { let dir = workdir("object-move-refusals"); @@ -9555,6 +11421,22 @@ fn a_deleted_image_is_not_kept_by_reduce_or_optimize() { assert_eq!(reopened["pages"], 1); } +#[test] +fn optimizing_discards_links_to_other_documents_through_tools() { + let dir = workdir("optimize-external-refs"); + std::fs::write(dir.join("remote.pdf"), remote_link_pdf("other.pdf", None)).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "remote.pdf" }))["doc"].as_u64().unwrap(); + let kept = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "kept.pdf" })); + assert_eq!(kept["external_links"], 0, "{kept}"); + let kept = ok(&mut a, "doc_open", json!({ "path": "kept.pdf" }))["doc"].as_u64().unwrap(); + assert_eq!(ok(&mut a, "link_list", json!({ "doc": kept }))["count"], 1); + let r = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "small.pdf", "discard_external_refs": true })); + assert_eq!((r["external_links"].as_u64(), r["external_bookmarks"].as_u64()), (Some(1), Some(0)), "{r}"); + let small = ok(&mut a, "doc_open", json!({ "path": "small.pdf" }))["doc"].as_u64().unwrap(); + assert_eq!(ok(&mut a, "link_list", json!({ "doc": small }))["count"], 0); +} + /// A two-page PDF with the same three-row table ("Name Qty / Apple 12 / Pear 7"); the second /// page is turned with /Rotate 90. fn table_pdf() -> Vec { @@ -10229,6 +12111,84 @@ fn a_root_less_write_output_is_atomic_too() { let _ = std::fs::remove_dir_all(&dir); } +/// Printer marks through the tools: added around the trim box (the page widened to make room), +/// removable by kind. +#[test] +fn printer_marks_through_tools() { + let dir = workdir("printer-marks"); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "a.pdf" }))["doc"].as_u64().unwrap(); + let width_before = ok(&mut a, "doc_info", json!({ "doc": doc }))["pages"][0]["width"].as_f64().unwrap_or(0.0); + let r = ok(&mut a, "doc_printer_marks", json!({ "doc": doc, "pages": [1], "color_bars": false, "line_weight": 0.5 })); + assert_eq!(r["undo"], "Add printer marks"); + let info = ok(&mut a, "doc_info", json!({ "doc": doc })); + let width_after = info["pages"][0]["width"].as_f64().unwrap_or(0.0); + assert!(width_after > width_before + 70.0 || width_before == 0.0, "widened to make room: {width_before} → {width_after}; {info}"); + let r = ok(&mut a, "doc_remove_marks", json!({ "doc": doc, "kind": "printer_marks" })); + assert_eq!(r["undo"], "Remove printer marks"); + for bad in [json!({ "line_weight": -1 }), json!({ "line_weight": 0 }), json!({ "line_weight": 10000 }), json!({ "make_room": 1000000 })] { + let mut args = bad.clone(); + args["doc"] = json!(doc); + assert!(matches!(a.call("doc_printer_marks", &args), Err(ToolError::Failed(_) | ToolError::InvalidArgs(_))), "{bad}"); + } + assert!(matches!( + a.call( + "doc_printer_marks", + &json!({ "doc": doc, "trim": false, "bleed": false, "registration": false, "color_bars": false, "page_info": false }) + ), + Err(ToolError::Failed(_) | ToolError::InvalidArgs(_)) + )); +} + +/// Convert colours through the tools: counts, converts, reports nothing to do. +#[test] +fn convert_colours_through_tools() { + let dir = workdir("convert-colours"); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "a.pdf" }))["doc"].as_u64().unwrap(); + // The fixture sets no colour of its own: nothing to convert yet. + let r = ok(&mut a, "doc_convert_colours", json!({ "doc": doc, "to": "gray" })); + assert_eq!((r["colours"].as_u64(), r["undo"].is_null()), (Some(0), true), "{r}"); + // Red text added as page content converts. + ok(&mut a, "page_add_text", json!({ "doc": doc, "page": 1, "text": "Red", "at": [20, 20], "color": "#FF0000" })); + let r = ok(&mut a, "doc_convert_colours", json!({ "doc": doc, "to": "gray", "pages": [1] })); + assert!(r["colours"].as_u64().unwrap_or(0) >= 1 && r["pages"].as_u64() == Some(1), "{r}"); + assert_eq!(r["undo"], "Convert colours"); + assert_eq!(r["loses_conformance"], json!([]), "{r}"); + assert!(matches!(a.call("doc_convert_colours", &json!({ "doc": doc, "to": "lab" })), Err(ToolError::InvalidArgs(_)))); + // A file that declares a standard is told what it may have lost. + ok(&mut a, "pdfa_convert", json!({ "doc": doc, "level": "3b" })); + ok(&mut a, "page_add_text", json!({ "doc": doc, "page": 1, "text": "Blue", "at": [20, 40], "color": "#0000FF" })); + let r = ok(&mut a, "doc_convert_colours", json!({ "doc": doc, "to": "cmyk", "pages": [1] })); + let loses = r["loses_conformance"].as_array().unwrap(); + assert_eq!(loses[0], "PDF/A-3b", "{r}"); + assert!(loses[1].as_str().unwrap().starts_with("output intent "), "{r}"); +} + +/// Fix hairlines through the tools: counts what changed, and reports zero without an edit. +#[test] +fn fix_hairlines_through_tools() { + let dir = workdir("hairlines"); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "a.pdf" }))["doc"].as_u64().unwrap(); + // The fixture's text pages have no strokes. + let r = ok(&mut a, "doc_fix_hairlines", json!({ "doc": doc })); + assert_eq!((r["fixed"].as_u64(), r["undo"].is_null()), (Some(0), true)); + // A zero-width rule drawn by the page content tools counts as a hairline. + ok(&mut a, "comment_add", json!({ "doc": doc, "page": 1, "type": "line", "from": [10, 10], "to": [100, 10], "width": 0 })); + let r = ok(&mut a, "doc_fix_hairlines", json!({ "doc": doc, "pages": [1] })); + // Comments are annotations, not page content: still nothing in the page's own content. + assert_eq!(r["fixed"].as_u64(), Some(0), "{r}"); + assert_eq!(r["elsewhere"].as_u64(), Some(0), "{r}"); + for bad in + [json!({ "width": 0 }), json!({ "width": -1 }), json!({ "width": 10000 }), json!({ "narrower_than": -1 }), json!({ "narrower_than": 10000 })] + { + let mut args = bad.clone(); + args["doc"] = json!(doc); + assert!(matches!(a.call("doc_fix_hairlines", &args), Err(ToolError::InvalidArgs(_))), "{bad}"); + } +} + #[test] fn attaching_and_deleting_files_through_the_tools() { let dir = workdir("attach"); @@ -10269,6 +12229,80 @@ fn attaching_and_deleting_files_through_the_tools() { assert!(a.call("attachment_add", &json!({ "doc": doc, "path": "missing.txt" })).is_err(), "a file that is not there is refused"); } +#[test] +fn attachment_descriptions_can_be_edited_cleared_undone_and_saved() { + let dir = workdir("attachment-description"); + let bytes = b"original attachment bytes"; + std::fs::write(dir.join("notes.txt"), bytes).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({"path": "a.pdf"}))["doc"].as_u64().unwrap(); + ok(&mut a, "attachment_add", json!({"doc": doc, "path": "notes.txt", "description": "Original"})); + + ok(&mut a, "attachment_edit", json!({"doc": doc, "name": "notes.txt", "description": "Résumé 東京"})); + assert_eq!(ok(&mut a, "doc_info", json!({"doc": doc}))["attachments"][0]["description"], "Résumé 東京"); + ok(&mut a, "edit_undo", json!({"doc": doc})); + assert_eq!(ok(&mut a, "doc_info", json!({"doc": doc}))["attachments"][0]["description"], "Original"); + ok(&mut a, "edit_redo", json!({"doc": doc})); + assert_eq!(ok(&mut a, "doc_info", json!({"doc": doc}))["attachments"][0]["description"], "Résumé 東京"); + + ok(&mut a, "attachment_edit", json!({"doc": doc, "name": "notes.txt", "description": ""})); + assert!(ok(&mut a, "doc_info", json!({"doc": doc}))["attachments"][0]["description"].is_null()); + ok(&mut a, "edit_undo", json!({"doc": doc})); + ok(&mut a, "doc_save", json!({"doc": doc, "path": "edited.pdf", "full": true})); + let saved = std::fs::read(dir.join("edited.pdf")).unwrap(); + let info = pdfcraft_render::inspect(std::sync::Arc::new(saved.clone()), None).unwrap(); + let attachment = info.attachments.iter().find(|item| item.name == "notes.txt").unwrap(); + assert_eq!(attachment.description.as_deref(), Some("Résumé 東京")); + assert_eq!(pdfcraft_render::attachment_data(&saved, None, attachment).unwrap(), bytes); + let reopened = ok(&mut a, "doc_open", json!({"path": "edited.pdf"}))["doc"].as_u64().unwrap(); + assert_eq!(ok(&mut a, "doc_info", json!({"doc": reopened}))["attachments"][0]["description"], "Résumé 東京"); +} + +#[test] +fn attachment_edit_resolves_the_listed_name_to_its_name_tree_key() { + let dir = workdir("attachment-name-tree-key"); + std::fs::write(dir.join("payload.txt"), b"payload").unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({"path":"a.pdf"}))["doc"].as_u64().unwrap(); + ok(&mut a, "attachment_add", json!({"doc":doc,"path":"payload.txt","name":"storage-key.txt"})); + ok(&mut a, "doc_save", json!({"doc":doc,"path":"seed.pdf","full":true})); + let mut source = pdfcraft_cos::Document::open(std::sync::Arc::new(std::fs::read(dir.join("seed.pdf")).unwrap())).unwrap(); + let key = "storage-key.txt"; + let root = source.root().unwrap(); + let catalog = source.get(root).as_dict().unwrap().clone(); + let names = source.resolve(catalog.get(b"Names").unwrap()).as_dict().unwrap().clone(); + let tree = source.resolve(names.get(b"EmbeddedFiles").unwrap()).as_dict().unwrap().clone(); + let pairs = source.resolve(tree.get(b"Names").unwrap()).as_array().unwrap().clone(); + let spec_ref = pairs.get(1).and_then(pdfcraft_cos::Object::as_ref).unwrap(); + source + .update_dict(spec_ref, |spec| { + let name = pdfcraft_cos::Object::String(pdfcraft_cos::PdfString::text("visible-name.txt")); + spec.set(b"F".to_vec(), name.clone()); + spec.set(b"UF".to_vec(), name); + }) + .unwrap(); + let bytes = pdfcraft_cos::write_full(&source, &pdfcraft_cos::SaveOptions::default()).unwrap(); + std::fs::write(dir.join("named.pdf"), bytes).unwrap(); + + let doc = ok(&mut a, "doc_open", json!({"path":"named.pdf"}))["doc"].as_u64().unwrap(); + let attachment = &ok(&mut a, "doc_info", json!({"doc":doc}))["attachments"][0]; + assert_eq!(attachment["name"], "visible-name.txt"); + assert_eq!(attachment["key"], key); + ok(&mut a, "attachment_edit", json!({"doc":doc,"key":key,"description":"Updated"})); + assert_eq!(ok(&mut a, "doc_info", json!({"doc":doc}))["attachments"][0]["description"], "Updated"); + assert!(matches!( + a.call("attachment_edit", &json!({"doc":doc,"name":"missing.txt","description":"x"})), + Err(ToolError::InvalidArgs(message)) if message.contains("no document attachment named") + )); + ok(&mut a, "doc_save", json!({"doc":doc,"path":"edited.pdf","full":true})); + let saved = pdfcraft_cos::Document::open(std::sync::Arc::new(std::fs::read(dir.join("edited.pdf")).unwrap())).unwrap(); + let catalog = saved.get(saved.root().unwrap()).as_dict().unwrap().clone(); + let names = saved.resolve(catalog.get(b"Names").unwrap()).as_dict().unwrap().clone(); + let tree = saved.resolve(names.get(b"EmbeddedFiles").unwrap()).as_dict().unwrap().clone(); + let pairs = saved.resolve(tree.get(b"Names").unwrap()).as_array().unwrap().clone(); + assert_eq!(pairs.first().and_then(pdfcraft_cos::Object::as_string).map(pdfcraft_cos::PdfString::to_text).as_deref(), Some(key)); +} + #[test] fn fdf_nested_comment_values_survive_export_import_and_save_through_tools() { use pdfcraft_cos::{Dict, Document, Object, PdfString, SaveOptions, write_full}; @@ -10359,6 +12393,49 @@ fn text_box_and_callout_fill_and_border_through_tools() { } } +/// A custom stamp can carry a dynamic line (#1741): `{name}`, `{date}` and `{time}` are filled in +/// when it is placed and drawn under the picture, which keeps its shape; the stamp widens to fit +/// the line; a line too long is cut. +#[test] +fn a_custom_stamp_line_is_filled_in_when_placed() { + let dir = workdir("stamp_line"); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "a.pdf" }))["doc"].as_u64().unwrap(); + ok(&mut a, "stamp_custom", json!({ "doc": doc, "page": 2, "path": "b.pdf", "file_page": 2, "at": [100, 150], "name": "Plain" })); + ok( + &mut a, + "stamp_custom", + json!({ "doc": doc, "page": 1, "path": "b.pdf", "file_page": 2, "at": [100, 150], "name": "Logo", "author": "Ada", "line": "Received {date} {time} by {name}" }), + ); + // The clock is 2023-11-14 22:13:20, taken as local time. + let text = page_text(&mut a, doc)[0].clone(); + assert!(text.contains("Received Nov 14, 2023 10:13 pm by Ada"), "{text}"); + let list = ok(&mut a, "comment_list", json!({ "doc": doc })); + let rect = |page: u64| { + let c = list["comments"].as_array().unwrap().iter().find(|c| c["page"] == page).unwrap(); + let r: Vec = c["rect"].as_array().unwrap().iter().map(|v| v.as_f64().unwrap()).collect(); + (r[2] - r[0], r[3] - r[1]) + }; + let ((pw, ph), (lw, lh)) = (rect(2), rect(1)); + // The stamp stays within the natural 200 pt. Its line, at its usual size, is wider than the + // picture, so the stamp is wider than the picture's shape with the band added. + assert!(lw / lh > pw / ph * (1.0 - 0.22) + 0.01, "widened for the line: {pw}x{ph} vs {lw}x{lh}"); + // A template within the schema's 200 characters whose filled-in line runs past the limit + // (33 times of 8 characters) is cut to exactly 200, not refused. + let long = "{time}".repeat(33); + assert!(long.chars().count() <= 200); + ok(&mut a, "stamp_custom", json!({ "doc": doc, "page": 3, "path": "b.pdf", "file_page": 2, "at": [100, 150], "line": long })); + ok(&mut a, "doc_save", json!({ "doc": doc, "path": "stamped.pdf" })); + let saved = pdfcraft_cos::Document::open(std::sync::Arc::new(std::fs::read(dir.join("stamped.pdf")).unwrap())).unwrap(); + let lines: Vec = saved + .scan_objects() + .filter_map(|(_, o)| o.as_dict()?.get(b"PCByLine")?.as_string().map(pdfcraft_cos::PdfString::to_text)) + .filter(|l| l.starts_with("10:13 pm")) + .collect(); + assert_eq!(lines.len(), 1, "{lines:?}"); + assert_eq!(lines[0].chars().count(), 200, "{:?}", lines[0]); +} + /// Three paragraphs: Alpha (two rows), Bravo (one row) and Charlie (two rows), at the leading /// of #2516. Alpha's last baseline is 35.6 pt above Bravo's; moving Alpha down 21.2 pt leaves /// 14.4 pt, Alpha's own leading, so the next read groups the two as one paragraph. @@ -11168,6 +13245,146 @@ fn simple_rule_scan_drawn(content: &[u8]) -> Vec { write_full(&doc, &SaveOptions::default()).unwrap() } +/// text_extract says when a page's text is incomplete because its content was cut at the +/// renderer's limits (#623); a page read whole carries no such mark. +#[test] +fn text_extract_says_when_a_pages_content_was_cut() { + use std::io::Write as _; + let dir = workdir("cut-content"); + // 288 MiB of whitespace, past the renderer's 256 MiB per-stream limit, as one Flate stream. + let mut z = flate2::write::ZlibEncoder::new(Vec::new(), flate2::Compression::best()); + let chunk = vec![0u8; 1 << 20]; + for _ in 0..288 { + z.write_all(&chunk).unwrap(); + } + let stream = z.finish().unwrap(); + let mut pdf = b"%PDF-1.7\n1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj\n2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj\n3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 200 300] /Contents 4 0 R >> endobj\n".to_vec(); + pdf.extend_from_slice(format!("4 0 obj << /Filter /FlateDecode /Length {} >> stream\n", stream.len()).as_bytes()); + pdf.extend_from_slice(&stream); + pdf.extend_from_slice(b"\nendstream endobj\ntrailer << /Root 1 0 R >>\n%%EOF\n"); + std::fs::write(dir.join("cut.pdf"), pdf).unwrap(); + let mut a = auto(&dir); + let cut = ok(&mut a, "doc_open", json!({ "path": "cut.pdf" }))["doc"].as_u64().unwrap(); + let out = ok(&mut a, "text_extract", json!({ "doc": cut })); + assert_eq!(out["pages"][0]["truncated"], json!(true), "{out}"); + let whole = ok(&mut a, "doc_open", json!({ "path": "a.pdf" }))["doc"].as_u64().unwrap(); + let out = ok(&mut a, "text_extract", json!({ "doc": whole, "pages": [1] })); + assert!(out["pages"][0].get("truncated").is_none(), "{out}"); + let _ = std::fs::remove_dir_all(&dir); +} + +// Contributor-original PDF: a Square on page 1 and, on page 2, a Circle with `/IRT` to it and +// `/RT /Group`. +const SPREAD_GROUP_FIXTURE: &[u8] = b"%PDF-1.7 +1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj +2 0 obj << /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 >> endobj +3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 300 200] /Annots [5 0 R] >> endobj +4 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 300 200] /Annots [6 0 R] >> endobj +5 0 obj << /Type /Annot /Subtype /Square /Rect [20 20 100 80] /NM (sq-primary) /T (Alice) /Contents (Primary) /C [1 0 0] >> endobj +6 0 obj << /Type /Annot /Subtype /Circle /Rect [180 120 260 180] /NM (circ-other-page) /T (Bob) /Contents (Member on page 2) /C [0 0 1] /IRT 5 0 R /RT /Group >> endobj +trailer << /Root 1 0 R >> +%%EOF"; + +/// A group is one page's, as Acrobat reads it: a subordinate whose primary is on another page +/// lists as a comment of its own, with its own author and text, not as part of the group or a +/// reply, and deleting the primary leaves it. ISO 32000-2 §12.5.6.2 doesn't limit a group to a +/// page; Acrobat (desktop and iPad) and PDF-XChange all delete the primary without such a +/// subordinate, and Acrobat desktop lists it on its own. +#[test] +fn a_group_member_on_another_page_is_a_comment_of_its_own() { + let dir = workdir("rt-group-spread"); + std::fs::write(dir.join("group.pdf"), SPREAD_GROUP_FIXTURE).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "group.pdf" }))["doc"].as_u64().unwrap(); + let check = |list: &Value, when: &str| { + assert_eq!(list["count"], 2, "{when}: the Square and the Circle, each a comment: {list}"); + let circle = list["comments"].as_array().unwrap().iter().find(|c| c["id"] == "circ-other-page").cloned().unwrap(); + assert_eq!((circle["author"].as_str(), circle["contents"].as_str()), (Some("Bob"), Some("Member on page 2")), "{when}"); + let square = list["comments"].as_array().unwrap().iter().find(|c| c["id"] == "sq-primary").cloned().unwrap(); + assert!(square["group"].as_array().unwrap().is_empty() && square["replies"].as_array().unwrap().is_empty(), "{when}: {square}"); + }; + check(&ok(&mut a, "comment_list", json!({ "doc": doc })), "opened"); + ok(&mut a, "comment_add", json!({ "doc": doc, "page": 1, "type": "note", "at": [280, 20], "contents": "Other" })); + let after = ok(&mut a, "comment_list", json!({ "doc": doc })); + let kept: Vec = after["comments"].as_array().unwrap().iter().filter(|c| c["contents"] != "Other").cloned().collect(); + check(&json!({ "count": kept.len(), "comments": kept }), "edited"); + ok(&mut a, "comment_delete", json!({ "doc": doc, "id": "sq-primary" })); + let left = ok(&mut a, "comment_list", json!({ "doc": doc })); + assert!(left.to_string().contains("circ-other-page"), "deleting the primary leaves the Circle on page 2: {left}"); +} + +/// #1152 headless: comment_list gives a callout's arrow point, knee and text box; comment_edit's +/// `points` moves the arrow point and knee, and `rect` resizes the text box with the arrow left +/// where it is. +#[test] +fn a_callouts_parts_are_listed_and_changed_through_the_tools() { + let dir = workdir("callout-parts"); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "a.pdf" }))["doc"].as_u64().unwrap(); + ok( + &mut a, + "comment_add", + json!({ "doc": doc, "page": 1, "type": "callout", "rect": [110, 200, 190, 240], "to": [40, 160], "knee": [80, 220], "contents": "Look" }), + ); + let first = |a: &mut Automation| ok(a, "comment_list", json!({ "doc": doc }))["comments"][0].clone(); + let near = |v: &Value, want: &[f64]| v.as_array().unwrap().iter().zip(want).all(|(x, w)| (x.as_f64().unwrap() - w).abs() < 0.6); + let c = first(&mut a); + assert!(near(&c["points"][0], &[40.0, 160.0]) && near(&c["points"][1], &[80.0, 220.0]), "{c}"); + assert!(near(&c["text_box"], &[110.0, 200.0, 190.0, 240.0]), "{c}"); + let (page, index) = (c["page"].clone(), c["index"].clone()); + ok(&mut a, "comment_edit", json!({ "doc": doc, "page": page, "index": index, "points": [[30, 150], [80, 220]] })); + let c = first(&mut a); + assert!(near(&c["points"][0], &[30.0, 150.0]) && near(&c["text_box"], &[110.0, 200.0, 190.0, 240.0]), "{c}"); + ok(&mut a, "comment_edit", json!({ "doc": doc, "page": page, "index": index, "rect": [110, 200, 230, 260] })); + let c = first(&mut a); + assert!(near(&c["text_box"], &[110.0, 200.0, 230.0, 260.0]) && near(&c["points"][0], &[30.0, 150.0]), "{c}"); + // Three points, or none, aren't a callout's. + assert!(matches!( + a.call("comment_edit", &json!({ "doc": doc, "page": page, "index": index, "points": [[1, 1], [2, 2], [3, 3]] })), + Err(ToolError::Failed(_) | ToolError::InvalidArgs(_)) + )); + ok(&mut a, "doc_save", json!({ "doc": doc, "path": "callout.pdf" })); + let saved = ok(&mut a, "doc_open", json!({ "path": "callout.pdf" }))["doc"].as_u64().unwrap(); + let c = ok(&mut a, "comment_list", json!({ "doc": saved }))["comments"][0].clone(); + assert!(near(&c["points"][0], &[30.0, 150.0]) && near(&c["text_box"], &[110.0, 200.0, 230.0, 260.0]), "after reopening: {c}"); + // Without a knee, comment_add puts it where Acrobat does: 12 pt out from the middle of the side + // facing the point. + ok( + &mut a, + "comment_add", + json!({ "doc": saved, "page": 1, "type": "callout", "rect": [300, 200, 380, 240], "to": [250, 160], "contents": "Auto" }), + ); + let list = ok(&mut a, "comment_list", json!({ "doc": saved }))["comments"].clone(); + let auto = list.as_array().unwrap().iter().find(|c| c["contents"] == "Auto").unwrap(); + assert!(near(&auto["points"][1], &[288.0, 220.0]), "{auto}"); +} + +/// comment_popup opens, closes and moves a comment's pop-up, comment_list reports it, and each +/// change is one undo step with its own label. +#[test] +fn comment_popups_are_opened_moved_and_listed() { + let dir = workdir("popup"); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "a.pdf" }))["doc"].as_u64().unwrap(); + ok(&mut a, "comment_add", json!({ "doc": doc, "page": 1, "type": "note", "at": [30.0, 40.0], "contents": "Look here" })); + let listed = ok(&mut a, "comment_list", json!({ "doc": doc })); + let note = &listed["comments"][0]; + assert_eq!(note["popup"]["open"], json!(false), "{listed}"); + let id = note["id"].as_str().unwrap().to_string(); + let opened = ok(&mut a, "comment_popup", json!({ "doc": doc, "id": id, "open": true })); + assert_eq!(opened["undo"], json!("Open comment pop-up"), "{opened}"); + let listed = ok(&mut a, "comment_list", json!({ "doc": doc })); + assert_eq!(listed["comments"][0]["popup"]["open"], json!(true)); + let moved = ok(&mut a, "comment_popup", json!({ "doc": doc, "id": id, "rect": [60.0, 60.0, 180.0, 140.0] })); + assert_eq!(moved["undo"], json!("Move comment pop-up")); + let listed = ok(&mut a, "comment_list", json!({ "doc": doc })); + assert_eq!(listed["comments"][0]["popup"]["rect"], json!([60.0, 60.0, 180.0, 140.0]), "{listed}"); + let closed = ok(&mut a, "comment_popup", json!({ "doc": doc, "id": id, "open": false })); + assert_eq!(closed["undo"], json!("Close comment pop-up")); + assert!(matches!(err(&mut a, "comment_popup", json!({ "doc": doc, "id": id })), ToolError::InvalidArgs(_))); + let _ = std::fs::remove_dir_all(&dir); +} + /// #3005: a table and a small signature drawn inside one transparency-group Form XObject (how /// HTML-to-PDF converters write a block with a translucent picture). Edit a PDF sees one piece /// of artwork and no text; `image_edit ungroup` draws it in place with the same pixels, its @@ -11863,6 +14080,49 @@ fn comments_copy_and_paste_between_documents_through_tools() { assert_eq!(ok(&mut a, "comment_list", json!({ "doc": src }))["comments"].as_array().unwrap().len(), 2, "the source is untouched"); } +/// A bitmap Type 3 font drawn with a flipped font matrix under a flipped text matrix (as TeX's +/// bitmap fonts are): `text_lines` boxes a line by its glyphs' `d1` boxes (ISO 32000-2 §9.6.4), +/// as high as the ink, not a 0.09-point sliver. +#[test] +fn text_lines_box_a_flipped_type3_line_by_its_glyphs() { + let content = "BT /F1 1 Tf 0.15 0 0 -0.15 10 20 Tm (ab) Tj ET"; + let glyph_a = "46 0 3 -1 42 57 d1 3 -1 39 58 re f"; + let glyph_b = "42 0 2 -18 40 38 d1 2 -18 38 56 re f"; + let objs: Vec = vec![ + "<< /Type /Catalog /Pages 2 0 R >>".into(), + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>".into(), + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 50] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >>".into(), + format!("<< /Length {} >>\nstream\n{content}\nendstream", content.len()), + "<< /Type /Font /Subtype /Type3 /FontMatrix [1 0 0 -1 0 0] /FontBBox [-4 -18 51 57] /FirstChar 97 /LastChar 98 \ + /Widths [46 42] /Encoding << /Differences [97 /a /b] >> /CharProcs << /a 6 0 R /b 7 0 R >> >>" + .into(), + format!("<< /Length {} >>\nstream\n{glyph_a}\nendstream", glyph_a.len()), + format!("<< /Length {} >>\nstream\n{glyph_b}\nendstream", glyph_b.len()), + ]; + let mut bytes = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offsets.push(bytes.len()); + bytes.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes()); + } + let xref = bytes.len(); + bytes.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offsets { + bytes.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + bytes.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes()); + let dir = workdir("type3-text-lines"); + std::fs::write(dir.join("type3.pdf"), bytes).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "type3.pdf" }))["doc"].as_u64().unwrap(); + let lines = ok(&mut a, "text_lines", json!({ "doc": doc, "page": 1 })); + // Glyph space y -18..57, turned over twice: 57 × 0.15 above the baseline at y 20 (8.55) and + // 18 × 0.15 below it (2.7). Top-left origin on a 50-point page: 21.45 down to 32.7. + assert_eq!(lines["count"], 1, "{lines}"); + assert_eq!(lines["lines"][0]["text"], "ab"); + assert_eq!(lines["lines"][0]["rect"], json!([10.0, 21.45, 23.2, 32.7]), "{lines}"); +} + #[test] fn explicit_bundled_thai_font_choice_layout_edit_undo_and_reopen() { let temp = tempfile::tempdir().unwrap(); diff --git a/crates/automation/tests/it/find_text.rs b/crates/automation/tests/it/find_text.rs new file mode 100644 index 0000000000..4ba2441305 --- /dev/null +++ b/crates/automation/tests/it/find_text.rs @@ -0,0 +1,84 @@ +//! `text_find` on a ligature glyph named in the encoding and on a word divided by a hyphen at a +//! line end. + +use std::path::PathBuf; + +use pdfcraft_automation::{Automation, Content}; +use serde_json::{Value, json}; + +/// A fresh directory for one test. +fn tmp(name: &str) -> PathBuf { + let d = std::env::temp_dir().join(format!("pdfcraft-find-text-{}-{name}", std::process::id())); + let _ = std::fs::remove_dir_all(&d); + std::fs::create_dir_all(&d).unwrap(); + d +} + +fn ok(a: &mut Automation, tool: &str, args: Value) -> Value { + match a.call(tool, &args) { + Ok(mut c) => match c.remove(0) { + Content::Json(v) => v, + other => panic!("{tool}: expected JSON, got {other:?}"), + }, + Err(e) => panic!("{tool} {args}: {e}"), + } +} + +/// One page: `ofce` in a Helvetica whose code 128 is named `/fi`, and `exam-` at the end of +/// one line with `ple` starting the next. +fn find_pdf() -> Vec { + let body = "BT /F2 14 Tf 90 700 Td (of\\200ce) Tj ET\nBT /F1 14 Tf 90 650 Td (exam-) Tj 0 -16 Td (ple) Tj ET\n"; + let objs = [ + "<< /Type /Catalog /Pages 2 0 R >>".to_string(), + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_string(), + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 612 792] /Contents 4 0 R /Resources << /Font << /F1 5 0 R /F2 6 0 R >> >> >>".to_string(), + format!("<< /Length {} >>\nstream\n{body}\nendstream", body.len()), + "<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>".to_string(), + "<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding << /Type /Encoding /BaseEncoding /WinAnsiEncoding /Differences [128 /fi] >> >>" + .to_string(), + ]; + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offsets.push(out.len()); + out.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes()); + } + let xref = out.len(); + out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offsets { + out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes()); + out +} + +/// "office" finds the ligature, and "example" both halves of the divided word, highlighted line by +/// line; the hyphenated form is still found as written. +#[test] +fn text_find_matches_ligatures_and_words_divided_at_a_line_end() { + let dir = tmp("forms"); + std::fs::write(dir.join("find.pdf"), find_pdf()).unwrap(); + let mut a = Automation::new().with_root(&dir).unwrap(); + let doc = ok(&mut a, "doc_open", json!({ "path": "find.pdf" }))["doc"].as_u64().unwrap(); + let find = |a: &mut Automation, q: &str| ok(a, "text_find", json!({ "doc": doc, "query": q })); + + let office = find(&mut a, "office"); + assert_eq!(office["count"], 1, "{office}"); + let rects = office["matches"][0]["rects"].as_array().unwrap(); + assert_eq!(rects.len(), 1); + assert!((rects[0][0].as_f64().unwrap() - 90.0).abs() < 0.5, "from the left edge of the o: {rects:?}"); + + let example = find(&mut a, "example"); + assert_eq!(example["count"], 1, "{example}"); + assert_eq!(example["matches"][0]["text"], "exam-\nple"); + let rects = example["matches"][0]["rects"].as_array().unwrap(); + assert_eq!(rects.len(), 2, "one highlight on each line: {rects:?}"); + let top = |k: usize| rects[k][1].as_f64().unwrap(); + assert!(top(1) > top(0) + 10.0, "the second on the next line down: {rects:?}"); + + for q in ["exam-ple", "Exam- ple", "exam-"] { + assert_eq!(find(&mut a, q)["count"], 1, "{q}"); + } + assert_eq!(find(&mut a, "examples")["count"], 0); + let _ = std::fs::remove_dir_all(&dir); +} diff --git a/crates/automation/tests/it/hostile_args.rs b/crates/automation/tests/it/hostile_args.rs index c017ff78d3..9835231c63 100644 --- a/crates/automation/tests/it/hostile_args.rs +++ b/crates/automation/tests/it/hostile_args.rs @@ -11,9 +11,10 @@ //! //! 1. **Nothing panics.** A panic in a tool is a crashed MCP server or CLI (AGENTS.md §4), so //! every call must come back as `Ok` or as a `ToolError` the agent can read. -//! 2. **Nothing escapes `--root`.** Every path argument is tried with `..` traversal, absolute -//! paths, Windows drive and UNC paths, and a planted symbolic link. The call must fail, and -//! afterwards nothing outside the root may have been created. +//! 2. **Nothing escapes `--root` or the automation roots.** Every path argument is tried with `..` +//! traversal, absolute paths, Windows drive and UNC paths, and a planted symbolic link, under +//! both `--root` and `--automation-read-root`/`--automation-write-root` (#4003). The call must +//! fail, and afterwards nothing outside the root may have been created. //! //! The second is the mechanized form of the D3 audit. Reading the code says every handler resolves //! its own paths and that no tool dispatches another internally; this keeps that true as tools are @@ -22,6 +23,7 @@ use std::panic::{AssertUnwindSafe, catch_unwind}; use std::path::{Path, PathBuf}; +use pdfcraft_automation::file_access::AutomationRoots; use pdfcraft_automation::{Automation, ToolError, tools}; use serde_json::{Value, json}; @@ -84,6 +86,7 @@ fn zoo() -> Vec { /// bug can only touch what the test made. `linked`: the root holds `link-out`, a symbolic link /// to `base`. fn escapes(base: &std::path::Path, linked: bool) -> Vec { + let root = base.join("root"); let mut out: Vec = vec![ "../outside.pdf".into(), "../../outside.pdf".into(), @@ -93,13 +96,16 @@ fn escapes(base: &std::path::Path, linked: bool) -> Vec { base.join("outside.pdf").to_string_lossy().into_owned(), base.join("new-outside.pdf").to_string_lossy().into_owned(), base.join("sub").join("new-outside.pdf").to_string_lossy().into_owned(), + root.join("..").join("outside.pdf").to_string_lossy().into_owned(), ]; if linked { out.push("link-out/outside.pdf".into()); + out.push(root.join("link-out").join("outside.pdf").to_string_lossy().into_owned()); } // Backslashes separate only on Windows; elsewhere this is an ordinary file name in the root. if cfg!(windows) { out.push(r"..\..\outside.pdf".into()); + out.push(r"\\.\pipe\pdfcraft-hostile-escape.pdf".into()); } out } @@ -220,6 +226,20 @@ fn empty_dir(dir: &Path) { } } +/// `action_run` keeps going when one input path is refused and reports that refusal on the file. +/// That is still a confinement refusal: every file has an error naming the boundary, and none +/// has an output. +fn per_file_confinement_refusal(content: &[pdfcraft_automation::Content]) -> bool { + let Some(files) = content.iter().find_map(|c| match c { + pdfcraft_automation::Content::Json(v) => v.get("files").and_then(Value::as_array), + pdfcraft_automation::Content::Png { .. } => None, + }) else { + return false; + }; + !files.is_empty() + && files.iter().all(|file| file.get("output").is_none() && file.get("error").and_then(Value::as_str).is_some_and(|e| e.contains("outside"))) +} + /// Real values for the arguments a tool checks before the others, so a hostile value reaches the /// argument it is meant for instead of failing on a placeholder. `sign_document` checks its /// digital ID first and its document second: this ID opens with its password, and each call gets @@ -234,6 +254,12 @@ fn fixture(a: &mut Automation, root: &Path, tool: &str) -> serde_json::Map { + let args = json!({ "steps": [{ "step": "set_title", "arg": "Confined" }] }); + args.as_object().cloned().unwrap_or_default() + } _ => serde_json::Map::new(), } } @@ -313,28 +339,34 @@ fn no_tool_escapes_the_root() { } }; - let mut a = Automation::new().with_root(&root).expect("root"); - - // A real document, so tools reach their path arguments instead of failing on `doc` first. - let doc = blank_doc(&mut a); + let roots = AutomationRoots::new(Some(&root), Some(&root)).expect("roots"); + let modes = [("--root", Automation::new().with_root(&root).expect("root")), ("automation roots", Automation::new().with_automation_roots(roots))]; let mut leaked = Vec::new(); let mut tried = 0usize; let mut refused = 0usize; - for def in tools() { - for (prop, is_list) in path_props(&def) { - for escape in escapes(&base, linked) { - let mut obj = plausible_with(&def, Some(doc)); - obj.insert(prop.clone(), if is_list { json!([escape]) } else { json!(escape) }); - let args = Value::Object(obj); - tried += 1; - match catch_unwind(AssertUnwindSafe(|| a.call(def.name, &args))) { - // A path that leaves the root must be refused. Succeeding means the tool - // either used it or ignored an argument it declares — both are faults. - Ok(Ok(_)) => leaked.push(format!("{} accepted {prop}={escape:?}", def.name)), - Ok(Err(_)) => refused += 1, - Err(_) => leaked.push(format!("{} panicked on {prop}={escape:?}", def.name)), + for (mode, mut a) in modes { + // A real document, so tools reach their path arguments instead of failing on `doc` first. + let doc = blank_doc(&mut a); + + for def in tools() { + for (prop, is_list) in path_props(&def) { + for escape in escapes(&base, linked) { + let mut obj = plausible_with(&def, Some(doc)); + obj.insert(prop.clone(), if is_list { json!([escape]) } else { json!(escape) }); + let args = Value::Object(obj); + tried += 1; + match catch_unwind(AssertUnwindSafe(|| a.call(def.name, &args))) { + // A path that leaves the root must be refused. Succeeding means the tool + // either used it or ignored an argument it declares — both are faults. + // action_run reports a refused input inside an otherwise successful + // result, one error per file, and writes nothing for that file. + Ok(Ok(content)) if per_file_confinement_refusal(&content) => refused += 1, + Ok(Ok(_)) => leaked.push(format!("{mode}: {} accepted {prop}={escape:?}", def.name)), + Ok(Err(_)) => refused += 1, + Err(_) => leaked.push(format!("{mode}: {} panicked on {prop}={escape:?}", def.name)), + } } } } diff --git a/crates/automation/tests/it/list_selection.rs b/crates/automation/tests/it/list_selection.rs new file mode 100644 index 0000000000..2c2dc041ef --- /dev/null +++ b/crates/automation/tests/it/list_selection.rs @@ -0,0 +1,81 @@ +//! Which options a list box has selected, through the tool table (ISO 32000-2 Table 234, `/I`). + +use pdfcraft_automation::{Automation, Content}; +use serde_json::{Value, json}; + +fn tool(a: &mut Automation, name: &str, args: Value) -> Value { + match a.call(name, &args).unwrap().remove(0) { + Content::Json(value) => value, + other => panic!("expected JSON: {other:?}"), + } +} + +/// A PDF of the given objects (numbered from 1). +fn pdf_of(objs: &[&str]) -> Vec { + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offsets.push(out.len()); + out.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes()); + } + let xref = out.len(); + out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offsets { + out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes()); + out +} + +fn open(test: &str) -> (Automation, u64, std::path::PathBuf) { + let dir = std::env::temp_dir().join(format!("pdfcraft-list-selection-{test}-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let bytes = pdf_of(&[ + "<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [4 0 R 6 0 R] /NeedAppearances true /DA (/Helv 10 Tf 0 g) /DR << /Font << /Helv 5 0 R >> >> >> >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 400 400] >>", + "<< /Type /Page /Parent 2 0 R /Annots [4 0 R 6 0 R] >>", + "<< /Type /Annot /Subtype /Widget /FT /Ch /T (D_dup) /Opt [[(1) (One-A)] [(1) (One-B)] [(2) (Two)]] /V (1) /I [1] /Rect [50 250 200 320] /P 3 0 R /F 4 >>", + "<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>", + "<< /Type /Annot /Subtype /Widget /FT /Ch /T (H_twovals) /Opt [[(TW) (Taiwan)] [(JP) (Japan)] [(US) (United States)]] /V [(TW) (JP)] /Rect [50 150 200 220] /P 3 0 R /F 4 >>", + ]); + std::fs::write(dir.join("choice.pdf"), bytes).unwrap(); + let mut a = Automation::new().with_root(&dir).unwrap(); + let id = tool(&mut a, "doc_open", json!({ "path": "choice.pdf" }))["doc"].as_u64().unwrap(); + (a, id, dir) +} + +fn field(a: &mut Automation, doc: u64, name: &str) -> Value { + let all = tool(a, "form_fields", json!({ "doc": doc })); + all["fields"].as_array().unwrap().iter().find(|f| f["name"] == name).unwrap().clone() +} + +/// Two options share the export value `1`: `selected` says which one `/I` picks, and filling the +/// field with an option's label picks that option. +#[test] +fn form_fields_reports_which_of_two_options_with_one_export_value_is_selected() { + let (mut a, doc, dir) = open("dup"); + let d = field(&mut a, doc, "D_dup"); + assert_eq!(d["value"], "1"); + assert_eq!(d["selected"], json!([2]), "One-B, the second option: {d}"); + tool(&mut a, "form_fill", json!({ "doc": doc, "values": { "D_dup": "One-A" } })); + let d = field(&mut a, doc, "D_dup"); + assert_eq!((d["value"].clone(), d["selected"].clone()), (json!("1"), json!([1])), "{d}"); + tool(&mut a, "form_fill", json!({ "doc": doc, "values": { "D_dup": "One-B" } })); + let d = field(&mut a, doc, "D_dup"); + assert_eq!(d["selected"], json!([2])); + let _ = std::fs::remove_dir_all(&dir); +} + +/// A list box without MultiSelect given `/V [(TW) (JP)]` reports Taiwan alone (ISO 32000-2 +/// Table 233: "at most one item shall be selected"), and filling it writes one value. +#[test] +fn form_fields_reports_one_value_for_a_single_select_list_given_two() { + let (mut a, doc, dir) = open("twovals"); + let h = field(&mut a, doc, "H_twovals"); + assert_eq!((h["value"].clone(), h["selected"].clone()), (json!("TW"), json!([1])), "{h}"); + tool(&mut a, "form_fill", json!({ "doc": doc, "values": { "H_twovals": "United States" } })); + let h = field(&mut a, doc, "H_twovals"); + assert_eq!((h["value"].clone(), h["selected"].clone()), (json!("US"), json!([3])), "{h}"); + let _ = std::fs::remove_dir_all(&dir); +} diff --git a/crates/automation/tests/it/main.rs b/crates/automation/tests/it/main.rs index 92e54c99a6..cdb072a888 100644 --- a/crates/automation/tests/it/main.rs +++ b/crates/automation/tests/it/main.rs @@ -9,12 +9,16 @@ mod batch; mod batch_failures; mod batch_secrets; mod button_caption; +mod find_text; mod form_bulk; mod hostile_args; +mod list_selection; mod model_3d; mod model_render; mod optimize_content_boundaries; +mod portfolio_info; mod redact_after_stray_q_form; mod redact_text_after_q; mod render_warnings; +mod text_copy; mod unreadable_objects; diff --git a/crates/automation/tests/it/model_render.rs b/crates/automation/tests/it/model_render.rs index 615a5346a9..c7dcca1af6 100644 --- a/crates/automation/tests/it/model_render.rs +++ b/crates/automation/tests/it/model_render.rs @@ -110,3 +110,171 @@ fn a_model_outside_the_root_is_never_opened_to_be_drawn() { let doc = open(&mut a, "prc.pdf"); assert!(a.call("model_render", &json!({ "doc": doc, "page": 1 })).is_ok()); } + +#[test] +fn an_orbit_frames_the_model_from_a_named_view_and_its_moves() { + let (_dir, mut a) = setup("orbit"); + let doc = open(&mut a, "prc.pdf"); + let draw = |a: &mut Automation, orbit: Value| orbit_picture(a, doc, false, &orbit); + // The triangle lies in z = 0: the Top view looks down on its face, the Front view along its edge. + let top = draw(&mut a, json!({ "view": "top" })); + assert_eq!(rgb(&top, 32, 24), [190, 192, 200], "the top view sees the face head on"); + let front = draw(&mut a, json!({ "view": "front" })); + assert!((0..48).all(|y| rgb(&front, 32, y) == [255, 255, 255]), "the front view sees only its edge"); + let tipped = draw(&mut a, json!({ "view": "front", "turn": [0, 30] })); + assert_ne!(rgb(&tipped, 32, 24), [255, 255, 255], "tipped down from the front, the face is in the middle"); + // With y up the Front view looks along -z, onto the face. + let y_up = draw(&mut a, json!({ "view": "front", "up": "+y" })); + assert_eq!(rgb(&y_up, 32, 24), [190, 192, 200]); + // Turning the front view down by 90° stops just short of the top view and sees the face. + let turned = draw(&mut a, json!({ "view": "front", "turn": [0, 90] })); + assert_ne!(rgb(&turned, 32, 24), [255, 255, 255]); + // A pan moves the target right, so the model moves left; three radii take it out of the picture. + let (left, right) = extent(&top, 24); + let (moved_left, moved_right) = extent(&draw(&mut a, json!({ "view": "top", "pan": [0.5, 0] })), 24); + assert!(moved_left < left && moved_right < right, "{left}..{right} moved to {moved_left}..{moved_right}"); + let panned = draw(&mut a, json!({ "view": "top", "pan": [3, 0] })); + assert!((0..64).all(|x| rgb(&panned, x, 24) == [255, 255, 255]), "the model has left the picture"); + // Zooming in about the centre keeps the face under it and leaves no background at the corner. + let zoomed = draw(&mut a, json!({ "view": "top", "zoom": 8 })); + assert_eq!(rgb(&zoomed, 32, 24), [190, 192, 200]); + assert_ne!(rgb(&zoomed, 0, 0), [255, 255, 255], "zoomed in, the face fills the corner too"); +} + +/// The first and last columns on row `y` that are not background. +fn extent(data: &[u8], y: usize) -> (usize, usize) { + let drawn: Vec = (0..64).filter(|&x| rgb(data, x, y) != [255, 255, 255]).collect(); + (*drawn.first().expect("the model is on the row"), *drawn.last().expect("the model is on the row")) +} + +/// The first and last rows on column `x` that are not background. +fn rows(data: &[u8], x: usize) -> (usize, usize) { + let drawn: Vec = (0..48).filter(|&y| rgb(data, x, y) != [255, 255, 255]).collect(); + (*drawn.first().expect("the model is on the column"), *drawn.last().expect("the model is on the column")) +} + +/// A 64 by 48 picture of the model through `orbit`. +fn orbit_picture(a: &mut Automation, doc: u64, perspective: bool, orbit: &Value) -> Vec { + let out = + a.call("model_render", &json!({ "doc": doc, "page": 1, "width": 64, "height": 48, "perspective": perspective, "orbit": orbit })).unwrap(); + let Content::Png { data, .. } = &out[0] else { panic!("expected a PNG") }; + data.clone() +} + +#[test] +fn an_empty_orbit_frames_the_isometric_view_in_perspective() { + let (_dir, mut a) = setup("orbit-default"); + let doc = open(&mut a, "prc.pdf"); + let default = orbit_picture(&mut a, doc, false, &json!({})); + assert_ne!(rgb(&default, 32, 24), [255, 255, 255], "the model is centred"); + let (left, right) = extent(&default, 24); + let (high, low) = rows(&default, 32); + assert!(left > 0 && right < 63 && high > 0 && low < 47, "inside the picture: {left}..{right}, {high}..{low}"); + assert!(right - left > 16, "and fills it: {left}..{right}"); + assert_eq!(default, orbit_picture(&mut a, doc, false, &json!({ "view": "iso", "up": "+z", "front": "+y" })), "the defaults"); + let unset = a.call("model_render", &json!({ "doc": doc, "page": 1, "width": 64, "height": 48, "orbit": {} })).unwrap(); + let Content::Png { data: unset, .. } = &unset[0] else { panic!("expected a PNG") }; + assert_eq!(*unset, orbit_picture(&mut a, doc, true, &json!({})), "perspective is the default"); + assert_ne!(*unset, default, "and shows"); +} + +#[test] +fn zoom_at_keeps_the_point_under_it_still() { + let (_dir, mut a) = setup("orbit-zoom-at"); + let doc = open(&mut a, "prc.pdf"); + let (left, right) = extent(&orbit_picture(&mut a, doc, false, &json!({ "view": "top", "zoom": 2 })), 24); + // A point 0.2 picture heights off centre stays there, so what was at the centre moves + // 0.2 x (2 - 1) x 48 = 9.6 pixels the other way. + for (at, sign) in [(0.2, -1.0), (-0.2, 1.0)] { + let (l, r) = extent(&orbit_picture(&mut a, doc, false, &json!({ "view": "top", "zoom": 2, "zoom_at": [at, 0] })), 24); + for (was, now) in [(left, l), (right, r)] { + let moved = now as f64 - was as f64; + assert!((moved - sign * 9.6).abs() <= 1.5, "at {at}: {was} moved to {now}"); + } + } +} + +#[test] +fn the_zoom_is_held_within_its_range() { + let (_dir, mut a) = setup("orbit-zoom-range"); + let doc = open(&mut a, "prc.pdf"); + let far = orbit_picture(&mut a, doc, false, &json!({ "view": "top", "zoom": 0.05 })); + assert_eq!(far, orbit_picture(&mut a, doc, false, &json!({ "view": "top", "zoom": 0.001 })), "the far end"); + assert_ne!(far, orbit_picture(&mut a, doc, false, &json!({ "view": "top", "zoom": 0.2 }))); + // Panned to just inside the near edge, so a nearer view would move the edge in the picture. + let close = |z: f64| json!({ "view": "top", "zoom": z, "pan": [0, -0.7] }); + let near = orbit_picture(&mut a, doc, false, &close(50.0)); + assert_eq!(near, orbit_picture(&mut a, doc, false, &close(1000.0)), "the near end"); + assert_ne!(near, orbit_picture(&mut a, doc, false, &close(25.0))); +} + +#[test] +fn an_orbit_draws_in_perspective_at_the_same_scale() { + let (_dir, mut a) = setup("orbit-perspective"); + let doc = open(&mut a, "prc.pdf"); + let tipped = json!({ "view": "front", "turn": [0, 30] }); + let deep = orbit_picture(&mut a, doc, true, &tipped); + assert_ne!(rgb(&deep, 32, 24), [255, 255, 255]); + assert_ne!(deep, orbit_picture(&mut a, doc, false, &tipped), "foreshortened"); + let top = json!({ "view": "top" }); + let (left, right) = extent(&orbit_picture(&mut a, doc, true, &top), 24); + let (flat_left, flat_right) = extent(&orbit_picture(&mut a, doc, false, &top), 24); + assert!(left.abs_diff(flat_left) <= 2 && right.abs_diff(flat_right) <= 2, "{left}..{right} against {flat_left}..{flat_right}"); +} + +#[test] +fn an_orbit_front_yaw_and_upward_pan_each_move_the_picture() { + let (_dir, mut a) = setup("orbit-moves"); + let doc = open(&mut a, "prc.pdf"); + let mut draw = |orbit: Value| orbit_picture(&mut a, doc, false, &orbit); + // Yawing the Front view by 90 degrees is the Right view, which looks along -x; +x is its opposite. + let right_view = draw(json!({ "view": "right", "turn": [0, 30] })); + assert_eq!(right_view, draw(json!({ "view": "front", "turn": [90, 30] })), "yaw"); + assert_eq!(right_view, draw(json!({ "view": "front", "front": "-x", "turn": [0, 30] })), "front -x"); + assert_ne!(right_view, draw(json!({ "view": "front", "turn": [0, 30] }))); + assert_eq!(draw(json!({ "view": "right", "turn": [180, 30] })), draw(json!({ "view": "front", "front": "+x", "turn": [0, 30] })), "front +x"); + // From above, the triangle's apex is up the picture; a quarter yaw lays it along the row, + // twice as wide there. + let top = draw(json!({ "view": "top" })); + let (left, right) = extent(&top, 24); + let (yaw_left, yaw_right) = extent(&draw(json!({ "view": "top", "turn": [90, 0] })), 24); + assert!(yaw_right - yaw_left > (right - left) * 3 / 2, "{left}..{right} yawed to {yaw_left}..{yaw_right}"); + // A pan up moves the target up, so the model moves down. + let (high, low) = rows(&top, 32); + let (moved_high, moved_low) = rows(&draw(json!({ "view": "top", "pan": [0, 0.25] })), 32); + assert!(moved_high > high && moved_low >= low, "{high}..{low} moved to {moved_high}..{moved_low}"); +} + +#[test] +fn a_wrong_orbit_is_a_clear_error() { + let (_dir, mut a) = setup("orbit-bad"); + let doc = open(&mut a, "prc.pdf"); + for (bad, reason) in [ + (json!({ "orbit": { "view": "side" } }), "view as one of"), + (json!({ "orbit": { "view": 1 } }), "view as a string"), + (json!({ "orbit": { "up": "z" } }), "up as one of"), + (json!({ "orbit": { "up": 1 } }), "up as a string"), + (json!({ "orbit": { "front": "y" } }), "front as one of"), + (json!({ "orbit": { "up": "+z", "front": "-z" } }), "front perpendicular to up"), + (json!({ "orbit": { "turn": [1] } }), "turn as [yaw, pitch]"), + (json!({ "orbit": { "turn": [1, 2, 3] } }), "turn as [yaw, pitch]"), + (json!({ "orbit": { "turn": 1 } }), "turn as [yaw, pitch]"), + (json!({ "orbit": { "zoom": 0 } }), "zoom as a positive number"), + (json!({ "orbit": { "zoom": -2 } }), "zoom as a positive number"), + (json!({ "orbit": { "zoom": "a" } }), "zoom as a positive number"), + (json!({ "orbit": { "zoom_at": [0.1, 0] } }), "with zoom when it has zoom_at"), + (json!({ "orbit": { "zoom": 2, "zoom_at": [1] } }), "zoom_at as [right, up]"), + (json!({ "orbit": { "zoom": 2, "zoom_at": [1, 2, 3] } }), "zoom_at as [right, up]"), + (json!({ "orbit": { "pan": [1, "a"] } }), "pan as [right, up]"), + (json!({ "orbit": { "pan": [1] } }), "pan as [right, up]"), + (json!({ "orbit": { "pan": [1, 2, 3] } }), "pan as [right, up]"), + (json!({ "orbit": { "spin": 1 } }), "without spin"), + (json!({ "orbit": 1 }), "an object"), + (json!({ "orbit": {}, "direction": [0, 0, -1] }), "left out when direction is given"), + ] { + let mut args = json!({ "doc": doc, "page": 1, "width": 16, "height": 16 }); + args.as_object_mut().unwrap().extend(bad.as_object().unwrap().clone()); + let err = a.call("model_render", &args).unwrap_err().to_string(); + assert!(err.contains("orbit must be") && err.contains(reason), "{bad}: {err}"); + } +} diff --git a/crates/automation/tests/it/portfolio_info.rs b/crates/automation/tests/it/portfolio_info.rs new file mode 100644 index 0000000000..750a3e8840 --- /dev/null +++ b/crates/automation/tests/it/portfolio_info.rs @@ -0,0 +1,54 @@ +//! `doc_info` reports a PDF portfolio (`/Collection`, ISO 32000-2 §7.11.6): the attachment it +//! presents first and how its file list first shows. + +use pdfcraft_automation::{Automation, Content}; +use serde_json::{Value, json}; + +/// Call tool `name` and return its JSON result. +fn tool(a: &mut Automation, name: &str, args: Value) -> Value { + match a.call(name, &args).unwrap().remove(0) { + Content::Json(value) => value, + other => panic!("expected JSON: {other:?}"), + } +} + +/// A one-page portfolio of `cover.txt` and `form.pdf` with `collection` as its `/Collection` +/// entries, or none at all. +fn portfolio(collection: Option<&str>) -> Vec { + let collection = collection.map(|c| format!("/Collection << /Type /Collection {c} >>")).unwrap_or_default(); + format!( + "%PDF-1.7 +1 0 obj << /Type /Catalog /Pages 2 0 R {collection} /Names << /EmbeddedFiles << /Names [(cover.txt) 5 0 R (form.pdf) 7 0 R] >> >> >> endobj +2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj +3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] >> endobj +4 0 obj << /Type /EmbeddedFile /Length 5 >> stream +hello +endstream endobj +5 0 obj << /Type /Filespec /F (cover.txt) /UF (cover.txt) /EF << /F 4 0 R >> >> endobj +6 0 obj << /Type /EmbeddedFile /Length 8 >> stream +%PDF-1.7 +endstream endobj +7 0 obj << /Type /Filespec /F (form.pdf) /UF (form.pdf) /EF << /F 6 0 R >> >> endobj +trailer << /Root 1 0 R >> +%%EOF" + ) + .into_bytes() +} + +/// `doc_info` names a portfolio's initial document and view, and reports none for a plain file. +#[test] +fn doc_info_names_the_document_a_portfolio_presents_first() { + let dir = tempfile::tempdir().unwrap(); + std::fs::write(dir.path().join("tile.pdf"), portfolio(Some("/D (form.pdf) /View /T"))).unwrap(); + std::fs::write(dir.path().join("cover.pdf"), portfolio(Some(""))).unwrap(); + std::fs::write(dir.path().join("plain.pdf"), portfolio(None)).unwrap(); + let mut auto = Automation::new().with_root(dir.path()).unwrap(); + let mut info = |path: &str| { + let doc = tool(&mut auto, "doc_open", json!({ "path": path }))["doc"].as_u64().unwrap(); + tool(&mut auto, "doc_info", json!({ "doc": doc }))["portfolio"].clone() + }; + assert_eq!(info("tile.pdf"), json!({ "initial": "form.pdf", "view": "tile" })); + // Without `/D` the cover sheet is the initial document. + assert_eq!(info("cover.pdf"), json!({ "initial": null, "view": "details" })); + assert_eq!(info("plain.pdf"), Value::Null); +} diff --git a/crates/automation/tests/it/render_warnings.rs b/crates/automation/tests/it/render_warnings.rs index ff50c11e16..013c287d5d 100644 --- a/crates/automation/tests/it/render_warnings.rs +++ b/crates/automation/tests/it/render_warnings.rs @@ -96,6 +96,33 @@ fn a_clean_page_renders_to_the_png_alone() { let _ = std::fs::remove_dir_all(dir); } +/// An image whose Indexed lookup table is too short (ISO 32000-2 §8.6.6.3) is drawn recovered, and +/// `page_render` says so in the words `pdfcraft-cli render` prints. +#[test] +fn page_render_reports_a_recovered_malformed_indexed_image() { + let dir = std::env::temp_dir().join(format!("pdfcraft-render-warnings-indexed-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let content = "q 200 0 0 200 0 0 cm /Im Do Q"; + // hival 7 over DeviceRGB needs 24 bytes; the table holds 6. + let pdf = format!( + "%PDF-1.7\n1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj\n\ + 2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj\n\ + 3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Resources << /XObject << /Im 5 0 R >> >> /Contents 4 0 R >> endobj\n\ + 4 0 obj << /Length {} >> stream\n{content}\nendstream endobj\n\ + 5 0 obj << /Type /XObject /Subtype /Image /Width 2 /Height 1 /BitsPerComponent 8 /ColorSpace [/Indexed /DeviceRGB 7 ] /Filter /ASCIIHexDecode /Length 5 >> stream\n0001>\nendstream endobj\n\ + trailer << /Root 1 0 R >>\n%%EOF", + content.len() + ); + std::fs::write(dir.join("indexed.pdf"), pdf).unwrap(); + let mut a = Automation::new().with_root(&dir).unwrap(); + let doc = open(&mut a, "indexed.pdf"); + let out = a.call("page_render", &json!({ "doc": doc, "page": 1, "dpi": 72 })).unwrap(); + let Some(Content::Json(report)) = out.get(1) else { panic!("expected the warnings as JSON: {out:?}") }; + assert_eq!(report["warnings"], json!(["an image with a malformed Indexed colour table may be drawn wrongly or not at all"])); + let _ = std::fs::remove_dir_all(dir); +} + /// Over MCP the warnings are a text block after the image and the call's `structuredContent`. #[cfg(feature = "mcp")] #[test] diff --git a/crates/automation/tests/it/text_copy.rs b/crates/automation/tests/it/text_copy.rs new file mode 100644 index 0000000000..c635fba148 --- /dev/null +++ b/crates/automation/tests/it/text_copy.rs @@ -0,0 +1,106 @@ +//! `text_extract` and `text_find` on a page of the cases text copy has to get right: words drawn out +//! of order, runs of different sizes on one baseline, a ligature glyph named in the encoding and a +//! line turned upwards in a row of text. + +use std::path::PathBuf; + +use pdfcraft_automation::{Automation, Content}; +use serde_json::{Value, json}; + +/// A fresh directory for one test. +fn tmp(name: &str) -> PathBuf { + let d = std::env::temp_dir().join(format!("pdfcraft-text-copy-{}-{name}", std::process::id())); + let _ = std::fs::remove_dir_all(&d); + std::fs::create_dir_all(&d).unwrap(); + d +} + +fn ok(a: &mut Automation, tool: &str, args: Value) -> Value { + match a.call(tool, &args) { + Ok(mut c) => match c.remove(0) { + Content::Json(v) => v, + other => panic!("{tool}: expected JSON, got {other:?}"), + }, + Err(e) => panic!("{tool} {args}: {e}"), + } +} + +/// One page of labelled rows (`/F1` Helvetica, `/F2` Helvetica with code 128 named `/fi`): +/// 01 `ofce`; 06 `exam-` over `ple`; 07 two columns of two lines; 08 "second" drawn before +/// "first", to its right; 11 a 14 pt invisible word and a 9 pt note on one baseline; 12 letter +/// spacing; 13 "up" set upwards; 14 two words placed by `Td`. +fn copy_pdf() -> Vec { + let rows = [ + ("01", "BT /F2 14 Tf 90 {Y} Td (of\\200ce) Tj ET"), + ("06", "BT /F1 14 Tf 90 {Y} Td (exam-) Tj 0 -16 Td (ple) Tj ET"), + ("07", "BT /F1 12 Tf 90 {Y} Td (left one) Tj 0 -14 Td (left two) Tj ET BT /F1 12 Tf 300 {Y} Td (right one) Tj 0 -14 Td (right two) Tj ET"), + ("08", "BT /F1 14 Tf 160 {Y} Td (second) Tj ET BT /F1 14 Tf 90 {Y} Td (first) Tj ET"), + ("11", "BT /F1 14 Tf 3 Tr 90 {Y} Td (hidden) Tj ET BT /F1 9 Tf 0.5 g 160 {Y} Td (<- invisible word here) Tj ET"), + ("12", "BT /F1 14 Tf 8 Tc 90 {Y} Td (SPACED) Tj ET"), + ("13", "BT /F1 14 Tf 0 1 -1 0 110 {Y} Tm (up) Tj ET"), + ("14", "BT /F1 14 Tf 90 {Y} Td (split) Tj 40 0 Td (words) Tj ET"), + ]; + let mut body = String::new(); + let mut y = 740; + for (tag, row) in rows { + body += &format!("BT /F1 9 Tf 0.4 g 40 {y} Td ({tag}) Tj ET 0 g\nq {} Q\n", row.replace("{Y}", &y.to_string())); + y -= match tag { + "06" | "13" => 48, + "07" => 44, + _ => 38, + }; + } + let objs = [ + "<< /Type /Catalog /Pages 2 0 R >>".to_string(), + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_string(), + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 612 792] /Contents 4 0 R /Resources << /Font << /F1 5 0 R /F2 6 0 R >> >> >>".to_string(), + format!("<< /Length {} >>\nstream\n{body}\nendstream", body.len()), + "<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>".to_string(), + "<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding << /Type /Encoding /BaseEncoding /WinAnsiEncoding /Differences [128 /fi] >> >>" + .to_string(), + ]; + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offsets.push(out.len()); + out.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes()); + } + let xref = out.len(); + out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offsets { + out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes()); + out +} + +/// Copy keeps a baseline on one line in left-to-right order, columns column by column, the +/// ligature as its letters and the turned line in its row; Find matches what Copy reads, and its +/// rectangles cover the glyphs it names. +#[test] +fn text_extract_and_find_read_one_line_per_baseline() { + let dir = tmp("baseline"); + std::fs::write(dir.join("copy.pdf"), copy_pdf()).unwrap(); + let mut a = Automation::new().with_root(&dir).unwrap(); + let doc = ok(&mut a, "doc_open", json!({ "path": "copy.pdf" }))["doc"].as_u64().unwrap(); + let text = ok(&mut a, "text_extract", json!({ "doc": doc })); + assert_eq!( + text["pages"][0]["text"], + "01 office\n06 exam-\nple\n07 left one\nleft two\nright one\nright two\n08 first second\n11 hidden <- invisible word here\n12 SPACED\n13\nup\n14 split words" + ); + let find = |a: &mut Automation, q: &str| ok(a, "text_find", json!({ "doc": doc, "query": q })); + let first_second = find(&mut a, "first second"); + assert_eq!(first_second["count"], 1); + assert_eq!(first_second["matches"][0]["text"], "first second"); + // One rectangle across both words, from the left edge of "first" (x = 90) past the right edge of "second". + let rects = first_second["matches"][0]["rects"].as_array().unwrap(); + assert_eq!(rects.len(), 1, "{rects:?}"); + let x = |k: usize| rects[0][k].as_f64().unwrap(); + assert!((x(0) - 90.0).abs() < 0.5 && x(2) > 200.0, "{rects:?}"); + let office = find(&mut a, "office"); + assert_eq!(office["count"], 1); + assert_eq!(office["matches"][0]["text"], "office"); + assert_eq!(find(&mut a, "hidden <- invisible")["count"], 1); + assert_eq!(find(&mut a, "left one right one")["count"], 0, "the columns stay apart"); + let _ = std::fs::remove_dir_all(&dir); +} diff --git a/crates/compare/src/lib.rs b/crates/compare/src/lib.rs index 26a50eaf71..e611b0a59c 100644 --- a/crates/compare/src/lib.rs +++ b/crates/compare/src/lib.rs @@ -411,6 +411,12 @@ mod tests { } } +/// Byte offset of pixel (`x`, `y`) in a `w`-wide RGBA buffer, in wide integers so an image past 4 GiB +/// (its `y * w` or `* 4` would wrap a `u32`) still indexes the right pixel; past `usize` it is out of range. +fn rgba_index(x: u32, y: u32, w: u32) -> usize { + usize::try_from((u128::from(y) * u128::from(w) + u128::from(x)) * 4).unwrap_or(usize::MAX) +} + /// Visual compare: the regions where two renderings of a page differ, as pixel boxes /// `[x0, y0, x1, y1]` (y down) in `a`'s image. Both images are RGBA; `b` is sampled at the /// same relative position when sizes differ. Pixels count as different when a channel differs @@ -421,6 +427,11 @@ pub fn visual_regions(a: (&[u8], u32, u32), b: (&[u8], u32, u32), tolerance: u8) if wa == 0 || ha == 0 || wb == 0 || hb == 0 { return Vec::new(); } + // Declared sizes the buffers cannot hold (a damaged render) would overflow the u32 index maths. + let holds = |px: &[u8], w: u32, h: u32| u128::from(w) * u128::from(h) * 4 <= px.len() as u128; + if !holds(pa, wa, ha) || !holds(pb, wb, hb) { + return Vec::new(); + } const CELL: u32 = 8; let (gw, gh) = (wa.div_ceil(CELL), ha.div_ceil(CELL)); let mut grid = vec![false; (gw * gh) as usize]; @@ -428,9 +439,8 @@ pub fn visual_regions(a: (&[u8], u32, u32), b: (&[u8], u32, u32), tolerance: u8) let yb = (y as u64 * hb as u64 / ha as u64) as u32; for x in 0..wa { let xb = (x as u64 * wb as u64 / wa as u64) as u32; - let ia = ((y * wa + x) * 4) as usize; - let ib = ((yb * wb + xb) * 4) as usize; - let (Some(ca), Some(cb)) = (pa.get(ia..ia + 3), pb.get(ib..ib + 3)) else { continue }; + let (ia, ib) = (rgba_index(x, y, wa), rgba_index(xb, yb, wb)); + let (Some(ca), Some(cb)) = (pa.get(ia..ia.saturating_add(3)), pb.get(ib..ib.saturating_add(3))) else { continue }; if ca.iter().zip(cb).any(|(p, q)| p.abs_diff(*q) > tolerance) { grid[((y / CELL) * gw + x / CELL) as usize] = true; } @@ -500,3 +510,28 @@ mod visual_tests { assert_eq!(visual_regions((&a, 200, 200), (&c, 200, 200), 16).len(), 2); } } + +#[cfg(test)] +mod visual_hostile_tests { + use super::*; + + #[test] + fn absurd_dimensions_with_short_buffers_return_quickly() { + let px = [0u8; 16]; + let r = visual_regions((&px, u32::MAX, u32::MAX), (&px, 2, 2), 0); + assert!(r.is_empty()); + } + + #[test] + fn pixel_offsets_past_four_gib_do_not_wrap() { + // A 32768 x 32769 RGBA image is 4,295,098,368 bytes, so a buffer that large passes the size check; + // its last pixel sits past u32::MAX and must not wrap back onto an early pixel. + let (w, h) = (32768u32, 32769u32); + let last = rgba_index(w - 1, h - 1, w); + assert_eq!(last as u64, (u64::from(h - 1) * u64::from(w) + u64::from(w - 1)) * 4); + assert!(last as u64 > u64::from(u32::MAX)); + assert_eq!(rgba_index(0, 0, w), 0); + // Out of range for any buffer: saturates instead of overflowing. + assert_eq!(rgba_index(u32::MAX, u32::MAX, u32::MAX), usize::MAX); + } +} diff --git a/crates/content/src/lib.rs b/crates/content/src/lib.rs index 4957f86861..fb17a903d1 100644 --- a/crates/content/src/lib.rs +++ b/crates/content/src/lib.rs @@ -67,8 +67,9 @@ pub struct Parsed { /// Bytes that could not be read as operands or operators. pub skipped: usize, /// Inline images the renderer's tokenizer reads no image for (it stops, or finds no `EI` it - /// accepts). Their end here is a guess, so the operators after them may not be the ones the - /// page draws. + /// accepts), so their end here is a guess and the operators after them may not be the ones + /// the page draws; and images it reads whose parameters this parser can't, which are missing + /// from `ops` although the page draws them. pub ambiguous_images: usize, /// For [`Pieces::parse`]: per piece, whether a comment runs into it from the piece before /// and whether one runs out of it into the next. Only boundaries are recorded, so the size @@ -139,8 +140,8 @@ impl<'a> Ops<'a> { self.skipped } - /// Inline images read so far whose end is this parser's guess (as in - /// [`Parsed::ambiguous_images`]). + /// Inline images read so far that are ambiguous (as in [`Parsed::ambiguous_images`]): either + /// their end is this parser's guess, or the renderer reads them and this parser cannot. pub fn ambiguous_images(&self) -> usize { self.ambiguous_images } @@ -291,7 +292,8 @@ struct Inline { image: Option<(Dict, Vec)>, /// The position after its `EI` (or the end of the stream). end: usize, - /// The renderer's tokenizer reads no image here, so `end` is this parser's guess. + /// The renderer's tokenizer reads no image here, so `end` is this parser's guess; or it reads + /// one whose parameters this parser can't, so `image` is `None` for an image the page draws. ambiguous: bool, } @@ -339,7 +341,9 @@ fn inline_image(data: &[u8], bi: usize, pos: usize) -> Inline { { samples = rest; } - return Inline { image: readable.then(|| (dict, samples.to_vec())), end, ambiguous: false }; + // The renderer draws an image whose parameters this parser can't read: it is not + // represented here, so a redaction can't inspect or remove it and must not trust the page. + return Inline { image: readable.then(|| (dict, samples.to_vec())), end, ambiguous: !readable }; } if !readable { // Without an `EI` after it, the image runs to the end of the stream. diff --git a/crates/content/src/tests.rs b/crates/content/src/tests.rs index 9ac9297cd2..9ca12c4536 100644 --- a/crates/content/src/tests.rs +++ b/crates/content/src/tests.rs @@ -209,6 +209,16 @@ fn filtered_unknown_and_damaged_inline_images_end_where_the_renderer_ends_them() } } +/// The renderer reads an image whose parameters this parser can't (a stray operand before `ID`): +/// the page draws an image `ops` doesn't hold, so it is counted as ambiguous rather than dropped +/// silently. +#[test] +fn an_inline_image_read_by_the_renderer_but_not_here_is_ambiguous() { + let parsed = parse(b"q BI /W 1 /H 1 /BPC 8 /CS /G 7 ID \x80 EI Q"); + assert_eq!(parsed.ops.iter().map(|op| op.op.as_slice()).collect::>(), [b"q".as_slice(), b"Q"]); + assert_eq!(parsed.ambiguous_images, 1); +} + /// A zlib stream (one stored block) for 64 DeviceGray samples whose bytes hold " EI\n" at offset 41, /// the way a stored deflate block copies incompressible samples verbatim (#3579). const FLATE_WITH_EI: &str = "789c014000bfffed746159d50788f7ea604984cd08e00b4c8983207f56fe82c77ef8dfb8b9a3a586382045490a3120\ diff --git a/crates/cos/README.md b/crates/cos/README.md index 9f5fb56725..a40e022e29 100644 --- a/crates/cos/README.md +++ b/crates/cos/README.md @@ -20,8 +20,8 @@ let doc = Document::open(bytes)?; // Arc> let doc = Document::open_with_password(bytes, Some("owner"))?; // encrypted files let root = doc.root(); // Option — the catalog -let obj = doc.get(ObjRef { num: 12, gen: 0 }); // load (and cache): Arc -let real = doc.resolve(&obj); // follow one indirect reference +let obj = doc.get(ObjRef::new(12, 0)); // load (and cache): Arc +let real = doc.resolve(&obj); // follow a bounded reference chain let d = doc.dict(&obj); // Option, resolving first let data = stream.decoded()?; // filters applied on demand let data = doc.with_direct_parms(&stream).decoded()?; // /DecodeParms given by reference too diff --git a/crates/cos/src/budget.rs b/crates/cos/src/budget.rs index 1a99885944..56286ce4a0 100644 --- a/crates/cos/src/budget.rs +++ b/crates/cos/src/budget.rs @@ -209,6 +209,11 @@ impl ImageBudget { self.refused } + /// Charges `work` steps paid for out of another allowance, by a thread working on its own. + pub fn charge(&mut self, work: u64) { + self.work = self.work.saturating_sub(work); + } + /// The steps of decoding still allowed. pub fn work_left(&self) -> u64 { self.work @@ -748,6 +753,38 @@ mod image_tests { assert_eq!(budget.work_left(), u64::MAX); } + /// PDFBOX-3932: an image and soft mask declared `/Predictor 15` that their writer never + /// applied, so their rows carry no PNG filter-type byte. Exported, they were "shorter than + /// their size says"; they are read as the samples they are, and tagged rows as before. + #[test] + fn an_image_whose_rows_carry_no_png_tags_decodes_to_its_samples() { + let doc = Document::new_empty(); + let samples: Vec = (0..36u8).map(|i| 200 - i).collect(); + let image = |height: Option, raw: Vec| { + let mut parms = Dict::new(); + parms.set(b"Predictor".to_vec(), Object::Int(15)); + parms.set(b"Colors".to_vec(), Object::Int(3)); + parms.set(b"Columns".to_vec(), Object::Int(3)); + let mut dict = Dict::new(); + dict.set(b"Width".to_vec(), Object::Int(3)); + if let Some(h) = height { + dict.set(b"Height".to_vec(), Object::Int(h)); + } + dict.set(b"Filter".to_vec(), Object::name("FlateDecode")); + dict.set(b"DecodeParms".to_vec(), Object::Dict(parms)); + Stream::from_raw(dict, raw) + }; + let untagged = pdfcraft_filters::encode_flate(&samples); + let mut budget = ImageBudget::with_work(u64::MAX); + assert_eq!(budget.decode(&doc, &image(Some(4), untagged.clone()), 1 << 20).unwrap(), samples); + // Without a height to tell, the predictor is undone, as before. + assert_ne!(budget.decode(&doc, &image(None, untagged.clone()), 1 << 20).ok(), Some(samples.clone())); + assert_ne!(budget.decode(&doc, &image(Some(5), untagged), 1 << 20).ok(), Some(samples.clone())); + let p = pdfcraft_filters::Params { predictor: 15, colors: 3, columns: 3, ..pdfcraft_filters::Params::default() }; + let tagged = pdfcraft_filters::encode(&pdfcraft_filters::Filter::Flate, &p, &samples).unwrap(); + assert_eq!(budget.decode(&doc, &image(Some(4), tagged), 1 << 20).unwrap(), samples); + } + /// The images of a file share its globals, so an image refused doesn't read them: this one's /// can't be read, which is how to tell. #[test] diff --git a/crates/cos/src/document.rs b/crates/cos/src/document.rs index e322f22a15..2bf4bce42b 100644 --- a/crates/cos/src/document.rs +++ b/crates/cos/src/document.rs @@ -48,6 +48,8 @@ pub struct Document { /// `fill_xref_gaps`. located: Arc>, repair_log: Arc>, + /// See [`Self::rebuilt_unlinked_update`]. + unlinked_update: bool, /// What loading each file object gave, a failure as well: a damaged object is otherwise /// loaded again on every read, and a member of an object stream that can't be decoded /// decodes the stream again (`Document::load`). diff --git a/crates/cos/src/document/objects.rs b/crates/cos/src/document/objects.rs index 91b8656a3f..8f9d3a8a5b 100644 --- a/crates/cos/src/document/objects.rs +++ b/crates/cos/src/document/objects.rs @@ -2,7 +2,8 @@ use std::collections::HashSet; use super::reader::find; use super::state::{ - LENGTHS, LoadGuard, ObjStm, ObjectReader, ReadState, Slot, SourceIdentity, loaded_bytes, note_overlay_read, objstm_member, overlay_reads, + LENGTHS, LoadGuard, ObjStm, ObjectReader, ReadState, ScanLengths, Slot, SourceIdentity, loaded_bytes, loading, note_overlay_read, objstm_member, + overlay_reads, }; use super::xref::{Revision, XrefEntry, XrefIndex}; use super::{ @@ -90,6 +91,13 @@ impl Document { &self.repair_log } + /// Whether the file was rebuilt from its object headers because its last update has no `/Prev` + /// and so cut its page tree off the cross-reference chain (ISO 32000-2 §7.5.6, §7.5.8.2). The + /// renderer rebuilds such a file under the same conditions, so the two read the same objects. + pub fn rebuilt_unlinked_update(&self) -> bool { + self.unlinked_update + } + /// Objects of the file that could not be read, lowest number first, as lines such as /// "object 11 0 could not be read: syntax error at byte 512: ...". [`Self::get`] reads each /// as null (ISO 32000-2 §7.3.10), so a document with one opens, but what it held is lost: @@ -479,7 +487,14 @@ impl Document { let data = if *work == 0 { Cow::Borrowed(&[][..]) } else { undo(before, work)? }; return pdfcraft_filters::decode_jbig2_within(&data, globals.as_deref().map(Vec::as_slice), columns, height, max, work).map_err(filter); } - let Some(p) = self.fax_params(s) else { return s.decoded_spending(max, work) }; + let Some(p) = self.fax_params(s) else { + // The last filter gives the image's rows: rows without the tags of the PNG predictor + // they declare are read as they are (`pdfcraft_filters::decode_image_tolerant_within`). + return match height { + Some(rows) => s.decoded_image_rows_spending(max, rows, work), + None => s.decoded_spending(max, work), + }; + }; // As many rows as the image has: a /Rows below /Height would cut the picture short, and // rows past /Height aren't drawn. let p = pdfcraft_filters::CcittParams { rows: 0, ..p }; @@ -691,23 +706,7 @@ impl Document { match self.entries.lookup(num, &self.data) { None | Some(XrefEntry::Free { .. }) => Ok(Object::Null), Some(XrefEntry::InFile { offset, .. }) => { - let resolve = |r: ObjRef| { - if self.overlay.contains_key(&r.num) { - note_overlay_read(); - } - // A length resolved through this document's edits counts as an edit read - // on every reuse. - if let Some((known, edits)) = LENGTHS.with(|m| m.borrow().get(&r).copied()) { - if edits { - note_overlay_read(); - } - return known; - } - let edits_read = overlay_reads(); - let length = self.try_get_reference(r).ok().and_then(|o| o.as_int()); - LENGTHS.with(|m| m.borrow_mut().insert(r, (length, overlay_reads() != edits_read))); - length - }; + let resolve = |r: ObjRef| self.stream_length(r); let off = offset as usize; let at = |off: usize| match parse_indirect_shared(&self.data, off, &resolve, &self.endstreams) { Ok((id, o)) if id.num == num => Some(self.decrypted(id, o)), @@ -781,6 +780,29 @@ impl Document { Ok(stm) } + /// An indirect stream `/Length`, read through the loader's guard against cycles. Kept in + /// [`LENGTHS`] only during a load, which clears it when it ends: a scan of the file from + /// [`Self::objstm`] can come here without one. + fn stream_length(&self, r: ObjRef) -> Option { + if self.overlay.contains_key(&r.num) { + note_overlay_read(); + } + let cached = loading(); + // A length resolved through this document's edits counts as an edit read on every reuse. + if cached && let Some((known, edits)) = LENGTHS.with(|m| m.borrow().get(&r).copied()) { + if edits { + note_overlay_read(); + } + return known; + } + let edits_read = overlay_reads(); + let length = self.try_get_reference(r).ok().and_then(|o| o.as_int()); + if cached { + LENGTHS.with(|m| m.borrow_mut().insert(r, (length, overlay_reads() != edits_read))); + } + length + } + /// Last-resort lookup: scan the file for `num G obj`. What isn't there is remembered: each miss /// reads the whole file, and a file can refer to such an object from everywhere. fn scan_for(&self, num: u32) -> Option<(ObjRef, Object)> { @@ -791,6 +813,8 @@ impl Document { super::tests::FILE_SCANS.with(|scans| scans.set(scans.get() + 1)); let needle = format!("{num} "); let data = &self.data; + let budget = ScanLengths::enter(); + let resolve = |r: ObjRef| if budget.take() { self.stream_length(r) } else { None }; let mut found = None; let mut i = 0; while let Some(p) = find(data, needle.as_bytes(), i, data.len()) { @@ -798,7 +822,11 @@ impl Document { if p > 0 && !is_whitespace(data[p - 1]) { continue; } - if let Ok((id, o)) = parse_indirect_shared(data, p, &|_| None, &self.endstreams) + // An indirect `/Length` is resolved, as at the object's offset, or a stream that ends + // in a misspelt `endstream` is searched on into a later object's. A header that parses + // as far as the `/Length` starts with `num` itself, after white space. A budget shared + // with the scans this leads to bounds how often the file is read again ([`ScanLengths`]). + if let Ok((id, o)) = parse_indirect_shared(data, p, &resolve, &self.endstreams) && id.num == num { found = Some((id, o)); // keep the last (newest) definition diff --git a/crates/cos/src/document/open.rs b/crates/cos/src/document/open.rs index b628cea89c..8caf0b83bc 100644 --- a/crates/cos/src/document/open.rs +++ b/crates/cos/src/document/open.rs @@ -21,6 +21,7 @@ impl Document { xref_oddities: Arc::default(), located: Arc::default(), repair_log: Arc::new(Vec::new()), + unlinked_update: false, cache: Arc::default(), objstms: Arc::default(), unreadable: Arc::default(), @@ -109,6 +110,7 @@ impl Document { xref_oddities: Arc::default(), located: Arc::default(), repair_log: Arc::new(Vec::new()), + unlinked_update: false, cache: Arc::default(), objstms: Arc::default(), unreadable: Arc::default(), @@ -134,6 +136,8 @@ impl Document { } let parsed = doc.read_xref_chain(&mut log); let mut authenticated = false; + // The newest section cut the page tree off from the chain (see `update_cuts_off_the_pages`). + let mut cut_off = false; let ok = match parsed { Ok((entries, trailer, revisions, oddities)) => { doc.entries = Arc::new(entries); @@ -147,17 +151,22 @@ impl Document { doc.authenticate(&enc, &credentials)?; authenticated = true; } - doc.root_is_catalog() + let catalog = doc.root_is_catalog(); + cut_off = catalog && doc.update_cuts_off_the_pages(); + catalog && !cut_off } Err(e) => { log.push(format!("cross-reference data unreadable ({e}); rebuilding from object headers")); false } }; + if cut_off { + log.push(UNLINKED_UPDATE.into()); + doc.unlinked_update = true; + } else if !ok && !doc.entries.is_empty() { + log.push("cross-reference data does not lead to a valid catalog; rebuilding from object headers".into()); + } if !ok { - if !doc.entries.is_empty() { - log.push("cross-reference data does not lead to a valid catalog; rebuilding from object headers".into()); - } doc.reconstruct(&mut log)?; } // Header generations are plaintext, including in encrypted files. Inspect only the @@ -182,6 +191,24 @@ impl Document { Ok(doc) } + /// Whether the chain of cross-reference sections ends at the newest one (no `/Prev`, or one + /// that isn't an offset) although the file holds earlier sections, and the catalog's `/Pages` + /// is not among the objects that one section reaches. ISO 32000-2 §7.5.6 and §7.5.8.2 require + /// the `/Prev`; without it the earlier revisions, page tree and all, are cut off, so the file + /// is rebuilt from its object headers as a damaged one is (the newest definition of each + /// object wins), and the repair log says why. (A file whose `/Root` itself is missing is + /// rebuilt anyway.) + fn update_cuts_off_the_pages(&self) -> bool { + if self.revisions.len() != 1 { + return false; + } + let Some(root) = self.root() else { return false }; + let Ok(catalog) = self.try_get_reference(root) else { return false }; + let Object::Dict(catalog) = &*catalog else { return false }; + // The page tree first: the scan for an earlier `startxref` reads the whole file. + !catalog.get(b"Pages").is_some_and(|pages| matches!(&*self.resolve(pages), Object::Dict(_))) && earlier_sections(&self.data) + } + fn authenticate(&mut self, enc: &Object, credentials: &Credentials<'_>) -> Result<(), CosError> { // Authentication can change how the same input is decoded. self.source_identity = Arc::new(()); @@ -203,13 +230,24 @@ impl Document { // Read here, while nothing is decrypted yet. let members = self.encryption_members(enc); let handler = match credentials { - Credentials::Password(password) => Arc::new(pdfcraft_crypt::SecurityHandler::open(params, &id0, *password).map_err(|e| match e { - pdfcraft_crypt::CryptError::WrongPassword if password.is_none() => CosError::NeedsPassword, - pdfcraft_crypt::CryptError::WrongPassword => CosError::WrongPassword, - other => CosError::Security(other.to_string()), - })?), - // The same encryption as the document whose key this is: that key opens it. - Credentials::Handler { handler, id0: known } if *handler.dict() == params && *known == id0 => Arc::clone(handler), + Credentials::Password(password) => match pdfcraft_crypt::SecurityHandler::open(params.clone(), &id0, *password) { + Ok(handler) => Arc::new(handler), + // No password, or the empty one: what only the embedded files need isn't asked. + Err(pdfcraft_crypt::CryptError::WrongPassword) => match self.without_password(&dict, params) { + Some(handler) if password.is_none_or(str::is_empty) => Arc::new(handler), + _ if password.is_none() => return Err(CosError::NeedsPassword), + _ => return Err(CosError::WrongPassword), + }, + Err(other) => return Err(CosError::Security(other.to_string())), + }, + // The same encryption as the document whose key this is: that key opens it. A handler + // without the key opens only what it would open by itself. + Credentials::Handler { handler, id0: known } if *handler.dict() == params && *known == id0 => { + if !handler.has_key() && !crate::security::efopen_only(self, &dict) { + return Err(CosError::NeedsPassword); + } + Arc::clone(handler) + } Credentials::Handler { .. } => return Err(CosError::NeedsPassword), }; self.security = Some(handler); @@ -222,6 +260,46 @@ impl Document { Ok(()) } + /// The handler for a document opened without a password that only its embedded files need + /// (see [`crate::security::efopen_only`]). Not a repair: ISO 32000-2 §7.6.6 has such a + /// document open so; [`Document::embedded_files_locked`] tells it. + fn without_password(&self, dict: &Dict, params: pdfcraft_crypt::EncryptDict) -> Option { + crate::security::efopen_only(self, dict).then(|| pdfcraft_crypt::SecurityHandler::without_key(params)).flatten() + } + + /// A document opened without the password its embedded files need takes either password + /// later: with the file key its embedded files decrypt, so what was read as stored is read + /// again. Refused while an edited stream holds data read as stored, which the key would + /// encrypt a second time. + fn unlock_embedded_files(&mut self, password: &str) -> Result<(), CosError> { + let Some(current) = self.security.as_deref() else { return Ok(()) }; + if self.overlay_entries().any(|(_, _, o)| matches!(o.as_deref(), Some(Object::Stream(s)) if self.reads_as_stored(s))) { + return Err(CosError::Security("an embedded file was edited before the password was given: save or undo that edit first".into())); + } + let handler = pdfcraft_crypt::SecurityHandler::open(current.dict().clone(), &self.first_id(), Some(password)).map_err(|e| match e { + pdfcraft_crypt::CryptError::WrongPassword => CosError::WrongPassword, + other => CosError::Security(other.to_string()), + })?; + self.security = Some(Arc::new(handler)); + self.source_identity = Arc::new(()); + self.cache = Arc::default(); + self.objstms = Arc::default(); + Ok(()) + } + + /// Whether the document opened without the password its embedded files need (`/AuthEvent + /// /EFOpen`): their data is held, and saved, as stored, and isn't to be read as the files. + pub fn embedded_files_locked(&self) -> bool { + self.security.as_deref().is_some_and(|h| !h.has_key()) + } + + /// Whether `s`, a stream of this document, holds encrypted data that reads as stored because the + /// document opened without the password its embedded files need (see + /// [`Document::embedded_files_locked`]). Such data isn't the file it encrypts. + pub fn reads_as_stored(&self, s: &crate::Stream) -> bool { + self.security.as_deref().is_some_and(|h| crate::security::stream_reads_as_stored(self, h, s)) + } + /// The first file identifier, part of the encryption key (empty when there is none). fn first_id(&self) -> Vec { let id = self.trailer.get(b"ID").map(|id| self.resolve(id)); @@ -268,6 +346,9 @@ impl Document { /// Both passwords unlock the same file key, so nothing read so far is decoded again. pub fn unlock(&mut self, password: &str) -> Result<(), CosError> { let Some(current) = self.security.as_deref() else { return Ok(()) }; + if !current.has_key() { + return self.unlock_embedded_files(password); + } if current.auth() == pdfcraft_crypt::Auth::Owner { return Ok(()); } @@ -411,3 +492,12 @@ impl Document { } } } + +/// The repair-log line for a file rebuilt because its last update has no `/Prev`. +const UNLINKED_UPDATE: &str = "the file's last update has no /Prev link to its earlier revisions, so their objects (the page tree among them) are unreachable through the cross-reference chain; rebuilding from object headers"; + +/// Whether `data` has more than one `startxref`: an update was appended to it, or it is +/// linearized (whose first-page section links to the main one). +fn earlier_sections(data: &[u8]) -> bool { + find(data, b"startxref", 0, data.len()).and_then(|at| find(data, b"startxref", at.saturating_add(9), data.len())).is_some() +} diff --git a/crates/cos/src/document/state.rs b/crates/cos/src/document/state.rs index af71f95359..c5bd85c065 100644 --- a/crates/cos/src/document/state.rs +++ b/crates/cos/src/document/state.rs @@ -11,6 +11,52 @@ thread_local! { pub(super) static LENGTHS: std::cell::RefCell, bool)>> = std::cell::RefCell::new(HashMap::new()); /// Unsaved edits read on this thread (`Document::try_get`, and a `/Length` in `Document::load`). static OVERLAY_READS: std::cell::Cell = const { std::cell::Cell::new(0) }; + /// How deep `Document::scan_for` is nested on this thread, and how many indirect `/Length` + /// values the outermost scan and those inside it have resolved ([`ScanLengths`]). + static SCAN_LENGTHS: std::cell::Cell<(u32, u32)> = const { std::cell::Cell::new((0, 0)) }; +} + +/// The indirect `/Length` values a scan of the file for an object may resolve, shared with the +/// scans it leads to. Each header in the file that starts with the object's number is parsed, and +/// a `/Length` that names an object the cross-reference table cannot place scans the file for that +/// one: a damaged file with many such headers, each naming another missing object, would read the +/// whole file once per header. Past the budget a `/Length` is left unresolved, and the stream is +/// read as before resolution, by its `endstream`. +pub(super) struct ScanLengths; + +impl ScanLengths { + /// A real object is defined once per revision, and its `/Length` once per definition. + pub(super) const MAX: u32 = 32; + + /// Marks a scan as running; the outermost one starts the budget afresh. + pub(super) fn enter() -> Self { + SCAN_LENGTHS.with(|c| { + let (depth, used) = c.get(); + c.set((depth.saturating_add(1), if depth == 0 { 0 } else { used })); + }); + ScanLengths + } + + /// Whether one more `/Length` may be resolved, counting it if so. + pub(super) fn take(&self) -> bool { + SCAN_LENGTHS.with(|c| { + let (depth, used) = c.get(); + if used >= Self::MAX { + return false; + } + c.set((depth, used.saturating_add(1))); + true + }) + } +} + +impl Drop for ScanLengths { + fn drop(&mut self) { + SCAN_LENGTHS.with(|c| { + let (depth, used) = c.get(); + c.set((depth.saturating_sub(1), used)); + }); + } } /// Unsaved edits read so far on this thread: a load during which this changes depended on one. @@ -42,6 +88,12 @@ impl LoadGuard { } } +/// Whether an object is being loaded on this thread: [`LENGTHS`] holds only while one is, since +/// it is cleared when the outermost load ends. +pub(super) fn loading() -> bool { + LOADING.with(|l| !l.borrow().is_empty()) +} + /// What loading a file object gave, kept in the document's cache either way. pub(super) type Loaded = Result, CosError>; diff --git a/crates/cos/src/document/tests.rs b/crates/cos/src/document/tests.rs index f69ffae042..274672bc7c 100644 --- a/crates/cos/src/document/tests.rs +++ b/crates/cos/src/document/tests.rs @@ -30,6 +30,67 @@ fn what_a_scan_of_the_file_did_not_find_is_remembered() { assert_eq!(FILE_SCANS.with(std::cell::Cell::get), 1, "a clone has the same file"); } +/// An object found by a scan of the file, because its cross-reference offset is wrong, has its +/// indirect `/Length` resolved as at the right offset. Unresolved, a stream ending in a misspelt +/// `endstream` (`endstrea`) was searched on into a later object's `endstream`, and took that +/// object's header and data as its own. +#[test] +fn a_scanned_stream_resolves_its_indirect_length() { + let mut out = build( + &[ + "<< /Type /Catalog /Pages 2 0 R >>", + "<< /Type /Pages /Kids [] /Count 0 >>", + "<< /Length 4 0 R >>\nstream\nHELLO\nendstrea", + "5", + "<< /Length 5 >>\nstream\nWORLD\nendstream", + ], + "/Root 1 0 R", + ); + // Point object 3's entry at the catalog, so only the scan finds it. + let at = |needle: &[u8], out: &[u8]| out.windows(needle.len()).position(|w| w == needle).unwrap(); + let catalog = format!("{:010} 00000 n ", at(b"1 0 obj", &out)); + let entry = at(format!("{:010} 00000 n ", at(b"3 0 obj", &out)).as_bytes(), &out); + out.splice(entry..entry + catalog.len(), catalog.bytes()); + let doc = Document::open(Arc::new(out)).unwrap(); + let stream = doc.try_get(3).unwrap(); + let Object::Stream(s) = &*stream else { panic!("stream expected: {stream:?}") }; + assert_eq!(&s.raw[..], b"HELLO"); +} + +/// A scan of the file resolves only so many indirect `/Length` values. Each header that starts +/// with the sought number is parsed, and a `/Length` naming an object the table cannot place scans +/// the file for that one: with a hundred such headers, each naming another misplaced object, one +/// lookup read the whole file a hundred and one times. +#[test] +fn a_scan_of_the_file_resolves_a_bounded_number_of_lengths() { + const HEADERS: u32 = 100; + let mut out = b"%PDF-1.7\n".to_vec(); + let catalog = out.len(); + out.extend_from_slice(b"1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj\n"); + let pages = out.len(); + out.extend_from_slice(b"2 0 obj << /Type /Pages /Kids [] /Count 0 >> endobj\n"); + // Object 3 is defined only by headers inside this string, each with its own misplaced /Length. + out.extend_from_slice(b"4 0 obj (\n"); + for k in 0..HEADERS { + out.extend_from_slice(format!("3 0 obj << /Length {} 0 R >> stream\nX\nendstream endobj\n", 10 + k).as_bytes()); + } + out.extend_from_slice(b") endobj\n"); + let size = 10 + HEADERS; + let xref = out.len(); + out.extend_from_slice(format!("xref\n0 {size}\n0000000000 65535 f \n").as_bytes()); + for num in 1..size { + let at = if num == 2 { pages } else { catalog }; + out.extend_from_slice(format!("{at:010} 00000 n \n").as_bytes()); + } + out.extend_from_slice(format!("trailer << /Size {size} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n").as_bytes()); + let doc = Document::open(Arc::new(out)).unwrap(); + FILE_SCANS.with(|scans| scans.set(0)); + let three = doc.try_get(3).unwrap(); + assert!(matches!(&*three, Object::Stream(_)), "{three:?}"); + let scans = FILE_SCANS.with(std::cell::Cell::get); + assert!(scans <= 1 + super::state::ScanLengths::MAX as usize, "{scans} scans of the file"); +} + /// An object can be nothing but a reference to another. A chain of them is followed, and one that /// goes round ends in what isn't a dictionary or a name, without hanging. #[test] @@ -276,6 +337,10 @@ fn fax_images_decode_with_their_parameters() { let mut understated = parms.clone(); understated.set(b"Rows".to_vec(), Object::Int(1)); assert_eq!(image(&doc, Some(Object::Dict(understated.clone())), 10, 3).unwrap(), bits, "/Height rows, whatever /Rows says"); + // More /Rows than /Height (PDFBOX-771: 1,084,975,104), as the renderer reads it too. + let mut overstated = parms.clone(); + overstated.set(b"Rows".to_vec(), Object::Int(1_000_000_000)); + assert_eq!(image(&doc, Some(Object::Dict(overstated)), 10, 3).unwrap(), bits, "/Height rows, not /Rows past it"); // The parameters as the file gives them, /Rows included, and Table 11's defaults. let mut d = Dict::new(); d.set(b"Filter".to_vec(), Object::name("CCITTFaxDecode")); @@ -2190,6 +2255,69 @@ fn the_editor_cache_charges_a_dictionary_for_the_values_it_holds() { assert_eq!(cache.owned(), sum, "the kept total is the cache's own"); } +/// The width (`/MediaBox` x1) of each of `pages`, in order. +fn page_widths(pages: &[Dict]) -> Vec> { + pages.iter().map(|p| p.get(b"MediaBox").and_then(Object::as_array).and_then(|b| b.get(2)).and_then(Object::as_int)).collect() +} + +/// The pages of `doc` in page-tree order, as the dictionaries the document holds. +fn page_tree_pages(doc: &Document) -> Vec { + let catalog = doc.get(doc.root().expect("root")); + let pages = doc.resolve(catalog.as_dict().and_then(|c| c.get(b"Pages")).expect("/Pages")); + let kids = pages.as_dict().and_then(|p| p.get(b"Kids")).and_then(Object::as_array).cloned().unwrap_or_default(); + kids.iter().filter_map(|k| doc.resolve(k).as_dict().cloned()).collect() +} + +/// ISO 32000-2 §7.5.6 and §7.5.8.2: an update reaches the revisions before it through `/Prev`. +/// One without it cuts them off, page tree and all. The file is rebuilt from its object headers +/// as a damaged one is, so its page tree is found, the newest definition of each object wins, and +/// the repair log says why; a save rewrites it in full, with every page. +#[test] +fn an_update_without_prev_is_rebuilt_with_its_page_tree_and_a_repair_note() { + let doc = Document::open(Arc::new(pdfcraft_testkit::update_without_prev(false))).expect("the unlinked file opens"); + let pages = page_tree_pages(&doc); + assert_eq!(pages.len(), 3); + assert!(pages.first().is_some_and(|p| p.contains(b"Annots")), "the update's page wins: {pages:?}"); + assert!(doc.reconstructed() && doc.rebuilt_unlinked_update()); + assert!(doc.repair_log().iter().any(|l| l.contains("no /Prev")), "{:?}", doc.repair_log()); + let catalog = doc.get(doc.root().expect("root")); + assert!(catalog.as_dict().is_some_and(|c| c.contains(b"Lang")), "the update's catalog wins"); + + let saved = crate::write_incremental(&doc, &crate::SaveOptions::default()).expect("saves"); + let reopened = Document::open(Arc::new(saved)).expect("the saved file opens"); + let pages = page_tree_pages(&reopened); + assert_eq!(page_widths(&pages), [Some(200), Some(300), Some(400)], "every page is saved, in order"); + assert!(pages.first().is_some_and(|p| p.contains(b"Annots")), "the update's page is saved: {pages:?}"); + let catalog = reopened.get(reopened.root().expect("root")); + assert!(catalog.as_dict().is_some_and(|c| c.contains(b"Lang")), "the update's catalog is saved"); + assert!(reopened.repair_log().is_empty(), "{:?}", reopened.repair_log()); + + // The same file with `/Prev` reads through its chain, as before. + let linked = Document::open(Arc::new(pdfcraft_testkit::update_without_prev(true))).expect("the linked file opens"); + assert_eq!(page_tree_pages(&linked).len(), 3); + assert!(!linked.reconstructed() && !linked.rebuilt_unlinked_update()); + assert_eq!(linked.revisions().len(), 2); + assert!(linked.repair_log().is_empty(), "{:?}", linked.repair_log()); +} + +/// A hybrid update (ISO 32000-2 §7.5.8.4) without `/Prev`: its `/XRefStm` stream belongs to the +/// update, so the earlier revision is cut off just the same, and the file is rebuilt. +#[test] +fn a_hybrid_update_without_prev_is_rebuilt_too() { + let doc = Document::open(Arc::new(pdfcraft_testkit::hybrid_update_without_prev(false))).expect("the unlinked file opens"); + let pages = page_tree_pages(&doc); + assert_eq!(pages.len(), 3); + assert!(pages.first().is_some_and(|p| p.contains(b"Annots")), "the update's page wins: {pages:?}"); + assert!(doc.rebuilt_unlinked_update(), "{:?}", doc.repair_log()); + assert!(doc.repair_log().iter().any(|l| l.contains("no /Prev")), "{:?}", doc.repair_log()); + let catalog = doc.get(doc.root().expect("root")); + assert!(catalog.as_dict().is_some_and(|c| c.contains(b"Lang")), "the update's catalog wins"); + let linked = Document::open(Arc::new(pdfcraft_testkit::hybrid_update_without_prev(true))).expect("the linked file opens"); + assert_eq!(page_tree_pages(&linked).len(), 3); + assert!(!linked.rebuilt_unlinked_update() && linked.repair_log().is_empty(), "{:?}", linked.repair_log()); + assert_eq!(linked.revisions().len(), 2); +} + #[test] fn find_agrees_with_a_plain_window_search() { let naive = |hay: &[u8], needle: &[u8], from: usize, to: usize| { diff --git a/crates/cos/src/lib.rs b/crates/cos/src/lib.rs index 32a6f6d8b3..7f7f32fedc 100644 --- a/crates/cos/src/lib.rs +++ b/crates/cos/src/lib.rs @@ -25,12 +25,12 @@ mod writer; pub use budget::{DecodeBudget, ImageBudget, MAX_PAGE_DECODED, MAX_PAGE_FORMS, MAX_REDACTED_IMAGE_PIXELS}; pub use bytes::Bytes; pub use document::{Document, ReadState, Revision, SourceIdentity, XrefEntry}; -pub use object::{Dict, MAX_DECODED, MAX_JBIG2_GLOBALS, Name, ObjRef, Object, PdfString, Stream}; +pub use object::{Dict, MAX_DECODED, MAX_JBIG2_GLOBALS, Name, ObjRef, Object, PdfString, Stream, TextChars}; pub use parser::{Lexer, is_whitespace, parse_indirect}; pub use pdfcraft_crypt::{ Algorithm, Auth, Method as CryptMethod, NewEncryption, Permissions, SecurityHandler, effective_password, pdfdoc_byte, pdfdoc_char, }; -pub use writer::{SaveOptions, pdf_date, reachable_objects, serialize, write_full, write_incremental}; +pub use writer::{SaveOptions, civil_from_days, new_file_id, pdf_date, reachable_objects, serialize, write_full, write_incremental}; #[derive(Debug, thiserror::Error, Clone, PartialEq)] pub enum CosError { diff --git a/crates/cos/src/object.rs b/crates/cos/src/object.rs index 68e8b1383e..97a3b60ab9 100644 --- a/crates/cos/src/object.rs +++ b/crates/cos/src/object.rs @@ -57,19 +57,90 @@ impl PdfString { /// non-ASCII bytes (written by some producers) is read as UTF-8 rather than as /// PDFDocEncoding mojibake, and stray leading U+FEFF (doubled BOMs) are removed. pub fn to_text(&self) -> String { - let s = match self.bytes.as_slice() { + self.text_chars().collect() + } + + /// The characters [`PdfString::to_text`] decodes, one at a time, without building a string. + pub fn text_chars(&self) -> TextChars<'_> { + let decode = match self.bytes.as_slice() { [0xFE, 0xFF, rest @ ..] => { - let units: Vec = rest.as_chunks::<2>().0.iter().map(|c| u16::from_be_bytes(*c)).collect(); - String::from_utf16_lossy(&units) + let units: fn(&[u8; 2]) -> u16 = |c| u16::from_be_bytes(*c); + Decode::Utf16(char::decode_utf16(rest.as_chunks::<2>().0.iter().map(units))) } - [0xEF, 0xBB, 0xBF, rest @ ..] => String::from_utf8_lossy(rest).into_owned(), + [0xEF, 0xBB, 0xBF, rest @ ..] => Decode::Utf8Lossy { chunks: rest.utf8_chunks(), valid: "".chars(), replace: false }, b => match std::str::from_utf8(b) { // Pure ASCII is identical either way; only non-ASCII UTF-8 needs the switch. - Ok(utf8) if !b.is_ascii() => utf8.to_owned(), - _ => b.iter().map(|&c| pdfcraft_crypt::pdfdoc_char(c)).collect(), + Ok(utf8) if !b.is_ascii() => Decode::Utf8(utf8.chars()), + _ => Decode::PdfDoc(b.iter()), }, }; - if s.starts_with('\u{FEFF}') { s.trim_start_matches('\u{FEFF}').to_owned() } else { s } + TextChars { decode, leading: true } + } + + /// Whether the string decodes to `s` ([`PdfString::to_text`]), compared without allocating. + pub fn text_eq(&self, s: &str) -> bool { + self.text_chars().eq(s.chars()) + } +} + +/// The characters of a text string: [`PdfString::text_chars`]. +pub struct TextChars<'a> { + decode: Decode<'a>, + /// Still at the start, where stray U+FEFF (doubled BOMs) are dropped. + leading: bool, +} + +/// The UTF-16BE code units after a BOM (an odd last byte is left out). +type Utf16Units<'a> = std::iter::Map, fn(&[u8; 2]) -> u16>; + +enum Decode<'a> { + Utf16(std::char::DecodeUtf16>), + /// After a UTF-8 BOM: each invalid sequence is one U+FFFD, as `String::from_utf8_lossy` has it. + Utf8Lossy { + chunks: std::str::Utf8Chunks<'a>, + valid: std::str::Chars<'a>, + replace: bool, + }, + Utf8(std::str::Chars<'a>), + PdfDoc(std::slice::Iter<'a, u8>), +} + +impl Iterator for Decode<'_> { + type Item = char; + + fn next(&mut self) -> Option { + match self { + Decode::Utf16(units) => units.next().map(|u| u.unwrap_or(char::REPLACEMENT_CHARACTER)), + Decode::Utf8Lossy { chunks, valid, replace } => loop { + if let Some(c) = valid.next() { + return Some(c); + } + if *replace { + *replace = false; + return Some(char::REPLACEMENT_CHARACTER); + } + let chunk = chunks.next()?; + *valid = chunk.valid().chars(); + *replace = !chunk.invalid().is_empty(); + }, + Decode::Utf8(chars) => chars.next(), + Decode::PdfDoc(bytes) => bytes.next().map(|&c| pdfcraft_crypt::pdfdoc_char(c)), + } + } +} + +impl Iterator for TextChars<'_> { + type Item = char; + + fn next(&mut self) -> Option { + loop { + let c = self.decode.next()?; + if self.leading && c == '\u{FEFF}' { + continue; + } + self.leading = false; + return Some(c); + } } } @@ -280,10 +351,12 @@ impl Stream { } /// Decoded data, tolerating truncated or corrupt encodings the way viewers do (the data - /// decoded before the damage is returned). Streams with image codecs fail with + /// decoded before the damage is returned). Decoding past + /// [`crate::budget::MAX_PAGE_DECODED`] (256 MiB), the most the renderer reads from one + /// stream, is an error. Streams with image codecs fail with /// `CosError::Filter` (keep them encoded). Use `decoded_strict` to detect damage. pub fn decoded(&self) -> Result, CosError> { - self.decoded_within(MAX_DECODED) + self.decoded_within(crate::budget::MAX_PAGE_DECODED) } /// Like [`Stream::decoded`], for a stream whose decoded size has a tighter bound than @@ -319,6 +392,14 @@ impl Stream { pdfcraft_filters::decode_tolerant_within(&chain, &self.raw, max.min(MAX_DECODED), work).map_err(filter) } + /// [`Stream::decoded_spending`] for the samples of an image `rows` high (its `/Height`): + /// [`pdfcraft_filters::decode_image_tolerant_within`]. + pub(crate) fn decoded_image_rows_spending(&self, max: usize, rows: u32, work: &mut u64) -> Result, CosError> { + let filter = |e: pdfcraft_filters::FilterError| CosError::Filter(e.to_string()); + let chain = self.readable_filters().map_err(filter)?; + pdfcraft_filters::decode_image_tolerant_within(&chain, &self.raw, max.min(MAX_DECODED), rows, work).map_err(filter) + } + /// The stream decoded through its filters by `decode`, or copied when it has none, within /// `max` and [`MAX_DECODED`]. fn decode_chain( @@ -594,6 +675,33 @@ mod tests { assert_eq!(PdfString::literal(Vec::new()).to_text(), ""); } + #[test] + fn text_eq_compares_what_to_text_decodes() { + let utf16 = |s: &str| [0xFE, 0xFF].into_iter().chain(s.encode_utf16().flat_map(u16::to_be_bytes)).collect::>(); + // Each invalid UTF-8 sequence is one U+FFFD, as String::from_utf8_lossy has it. + let lossy = [0xEF, 0xBB, 0xBF, 0xC3, 0x28, b'x', 0xF0, 0x9F, 0x92, 0xFF]; + assert_eq!(PdfString::literal(lossy.to_vec()).to_text(), String::from_utf8_lossy(&lossy[3..])); + let cases: Vec> = vec![ + Vec::new(), + b"Plain".to_vec(), + vec![b'C', b'a', b'f', 0xE9, 0x18], + "目录".as_bytes().to_vec(), + utf16("第一章"), + vec![0xFE, 0xFF, 0xD8, 0x00, 0x00, b'A'], + vec![0xFE, 0xFF, 0xFE, 0xFF, 0x00, b'A', 0xFE, 0xFF], + lossy.to_vec(), + ]; + for bytes in cases { + let s = PdfString::literal(bytes); + let text = s.to_text(); + assert!(s.text_eq(&text), "{text:?}"); + assert!(!s.text_eq(&format!("{text}x")), "{text:?}"); + assert!(text.is_empty() || !s.text_eq(&text[..text.len() - text.chars().last().map_or(0, char::len_utf8)]), "{text:?}"); + } + // A U+FEFF after the first character is text and stays. + assert!(PdfString::literal(utf16("A\u{FEFF}")).text_eq("A\u{FEFF}")); + } + #[test] fn pdfdoc_encoding_high_range() { assert_eq!(PdfString::literal(vec![0x84, 0x92, 0xA0, b'A']).to_text(), "—™€A"); @@ -674,6 +782,13 @@ mod tests { assert_eq!(stream.decoded_strict().unwrap(), b"A"); } + #[test] + fn decoded_stops_at_renderer_cap_of_256_mib() { + let data = vec![0u8; (256 << 20) + 1]; + let stream = Stream::flate(Dict::new(), &data); + assert!(matches!(stream.decoded(), Err(CosError::Filter(_)))); + } + #[test] fn first_filter_decodes_only_the_outer_wrapper() { let data = "0 0 m 100 100 l S\n".repeat(64).into_bytes(); diff --git a/crates/cos/src/parser.rs b/crates/cos/src/parser.rs index 709659f537..49793d950c 100644 --- a/crates/cos/src/parser.rs +++ b/crates/cos/src/parser.rs @@ -437,7 +437,7 @@ pub(crate) type EndstreamIndex = std::sync::OnceLock>; fn endstream_offsets(data: &[u8]) -> Vec { #[cfg(test)] ENDSTREAM_SCAN.with(|c| c.set(c.get() + data.len())); - data.windows(9).enumerate().filter(|(_, w)| *w == b"endstream").map(|(i, _)| i).collect() + data.windows(9).enumerate().filter(|(i, w)| *w == b"endstream" && endstream_at(data, *i)).map(|(i, _)| i).collect() } fn parse_indirect_in( @@ -491,25 +491,28 @@ fn parse_indirect_in( Some(Object::Ref(r)) => resolve_length(*r), _ => None, }; - let valid = |len: i64| -> bool { - if len < 0 { - return false; - } - let end = start.saturating_add(len as usize); - if end > data.len() { - return false; - } - let mut after = Lexer::new(data, end); - after.eat_keyword(b"endstream") + // A /Length that fits in the file (not negative, and no wrap on a 32-bit target). + let fits = declared.and_then(|l| usize::try_from(l).ok()).filter(|l| start.checked_add(*l).is_some_and(|end| end <= data.len())); + // What follows the /Length bytes, white space aside. + let after = |len: usize| { + let mut after = Lexer::new(data, start.saturating_add(len)); + after.skip_ws(); + data.get(after.pos..).unwrap_or_default() }; - let len = match declared { - Some(l) if valid(l) => l as usize, - // Wrong or missing /Length: search for "endstream" (§7.3.8.1 recovery). - _ => { - find_endstream(data, start, endstreams) - .ok_or_else(|| CosError::Syntax { offset: start, detail: "stream without endstream".into() })? - - start - } + let len = match fits { + // The /Length ends at `endstream`, or where it would be: misspelt (`endstrea`), or + // left out before `endobj` or the end of the file. The stream is the bytes its /Length + // gives, "how many bytes of the PDF file are used for the stream's data" (ISO 32000-2 + // §7.3.8.2), as other readers take it. A producer that wrote `endstrea` after every + // stream lost every font and every page's content, and searching on would run into a + // later object's `endstream`. + Some(l) if at_stream_end(after(l)) => l, + // Wrong or missing /Length: search for "endstream" (§7.3.8.1 recovery). With none + // left in the file, a /Length that fits is still the stream's extent. + _ => match find_endstream(data, start, endstreams) { + Some(end) => end - start, + None => fits.ok_or_else(|| CosError::Syntax { offset: start, detail: "stream without endstream".into() })?, + }, }; let range = start..start + len; let raw = match shared { @@ -524,6 +527,45 @@ fn parse_indirect_in( Ok((id, obj)) } +/// Whether `rest`, what follows a stream's /Length bytes with white space skipped, is where the +/// stream ends. A whole `endstream` token is: a token runs on through regular characters (ISO +/// 32000-2 §7.2.3), so payload such as `endstreamer` is not one. A misspelt `endstream` +/// (`endstrea`), an `endobj` with the `endstream` left out, or both, end it only where the object +/// ends as well, before the end of the data, another object's `N G obj` header, or `xref`, +/// `trailer` or `startxref`: payload such as `endobj more` or `endobject`, where a wrong /Length +/// stops, is not, and the search for `endstream` runs instead. So does anything else but the end +/// of the data. +fn at_stream_end(rest: &[u8]) -> bool { + let mut lx = Lexer::new(rest, 0); + if lx.eat_keyword(b"endstream") { + return true; + } + let misspelt = lx.eat_keyword(b"endstrea"); + let endobj = lx.eat_keyword(b"endobj"); + lx.skip_ws(); + if lx.pos >= rest.len() { + return true; + } + if !(misspelt || endobj) { + return false; + } + if [&b"xref"[..], b"trailer", b"startxref"].iter().any(|kw| Lexer::new(rest, lx.pos).eat_keyword(kw)) { + return true; + } + let number = |lx: &mut Lexer| { + lx.skip_ws(); + let t = lx.token(); + !t.is_empty() && t.iter().all(u8::is_ascii_digit) + }; + number(&mut lx) && number(&mut lx) && lx.eat_keyword(b"obj") +} + +/// Whether a whole `endstream` token starts at `at` in `data`: one followed by a regular +/// character (`endstreamer`) is payload (ISO 32000-2 §7.2.3). +fn endstream_at(data: &[u8], at: usize) -> bool { + data.get(at..).is_some_and(|d| d.starts_with(b"endstream")) && !data.get(at.saturating_add(9)).copied().is_some_and(is_regular) +} + /// Offset where stream data ends (before the EOL preceding "endstream"). fn find_endstream(data: &[u8], from: usize, endstreams: Option<&EndstreamIndex>) -> Option { let at = match endstreams { @@ -533,7 +575,7 @@ fn find_endstream(data: &[u8], from: usize, endstreams: Option<&EndstreamIndex>) } None => { let hay = data.get(from..)?; - let found = hay.windows(9).position(|w| w == b"endstream"); + let found = hay.windows(9).enumerate().position(|(i, w)| w == b"endstream" && endstream_at(hay, i)); #[cfg(test)] ENDSTREAM_SCAN.with(|c| c.set(c.get() + found.map_or(hay.len(), |p| p + 9))); found? + from @@ -660,6 +702,127 @@ mod tests { assert_eq!(s.dict.int(b"Length"), Some(5)); } + /// With no `endstream` left in the file, as a producer that writes `endstrea` leaves every + /// stream, a stream is the bytes its /Length gives, "how many bytes of the PDF file are used + /// for the stream's data" (ISO 32000-2 §7.3.8.2): it was an error, and the object null, so + /// such a file lost every font and page content. A /Length past the file, or none, is still + /// no stream. + #[test] + fn indirect_stream_without_endstream_is_its_length() { + for end in [&b"\r\nendstrea\rendobj\r\n"[..], b"\nendobj\n", b""] { + let mut data = b"7 0 obj\n<< /Length 5 >>\nstream\r\nHELLO".to_vec(); + data.extend_from_slice(end); + let (_, o) = parse_indirect(&data, 0, &|_| None).unwrap(); + let Object::Stream(s) = o else { panic!("stream expected") }; + assert_eq!(&s.raw[..], b"HELLO", "{end:?}"); + } + assert!(parse_indirect(b"7 0 obj\n<< /Length 6 >>\nstream\nHELLO", 0, &|_| None).is_err(), "past the file"); + assert!(parse_indirect(b"7 0 obj\n<< >>\nstream\nHELLO\nendstrea\nendobj\n", 0, &|_| None).is_err(), "no /Length"); + } + + /// A stream ending in `endstrea` is its /Length even when a later object does have an + /// `endstream`: the search for one would otherwise run on into that object and take its + /// header and data as this stream's. + #[test] + fn a_stream_without_endstream_is_not_read_into_a_later_one() { + let data = b"7 0 obj\n<< /Length 5 >>\nstream\r\nHELLO\r\nendstrea\rendobj\r\n8 0 obj\n<< /Length 5 >>\nstream\nWORLD\nendstream\nendobj\n"; + let second = data.windows(7).position(|w| w == b"8 0 obj").unwrap(); + for (offset, want) in [(0, &b"HELLO"[..]), (second, b"WORLD")] { + let (_, o) = parse_indirect(data, offset, &|_| None).unwrap(); + let Object::Stream(s) = o else { panic!("stream expected at {offset}") }; + assert_eq!(&s.raw[..], want, "at {offset}"); + } + // Through the loader's shared index of `endstream` offsets as well. + let index = EndstreamIndex::new(); + let (_, o) = parse_indirect_in(data, None, 0, &|_| None, Some(&index)).unwrap(); + let Object::Stream(s) = o else { panic!("stream expected") }; + assert_eq!(&s.raw[..], b"HELLO"); + } + + /// The same for a stream whose `endstream` is left out before `endobj`: the later object's + /// `endstream` is not this stream's end. + #[test] + fn a_stream_ending_at_endobj_is_not_read_into_a_later_one() { + let data = b"7 0 obj\n<< /Length 5 >>\nstream\nHELLO\nendobj\n8 0 obj\n<< /Length 5 >>\nstream\nWORLD\nendstream\nendobj\n"; + let (_, o) = parse_indirect(data, 0, &|_| None).unwrap(); + let Object::Stream(s) = o else { panic!("stream expected") }; + assert_eq!(&s.raw[..], b"HELLO"); + } + + /// A wrong /Length that stops before payload bytes beginning like an end marker (`endobject`, + /// `endstreak`, `endstreamer`) is not taken for the stream's end: a token runs on through + /// regular characters (ISO 32000-2 §7.2.3), so the search for `endstream` finds the whole + /// payload, and skips `endstreamer` too, by itself and through the loader's shared index. + #[test] + fn a_length_ending_inside_a_longer_word_is_not_an_end_marker() { + for word in ["endobject", "endstreak", "endstreamer"] { + let pdf = format!("7 0 obj\n<< /Length 5 >>\nstream\nHELLO {word} more\nendstream\nendobj\n"); + let (_, o) = parse_indirect(pdf.as_bytes(), 0, &|_| None).unwrap(); + let Object::Stream(s) = o else { panic!("stream expected") }; + assert_eq!(s.raw[..], *format!("HELLO {word} more").as_bytes(), "{word}"); + let index = EndstreamIndex::new(); + let (_, o) = parse_indirect_in(pdf.as_bytes(), None, 0, &|_| None, Some(&index)).unwrap(); + let Object::Stream(s) = o else { panic!("stream expected") }; + assert_eq!(s.raw[..], *format!("HELLO {word} more").as_bytes(), "{word}, indexed"); + } + // Whole markers still end it, followed by white space, a delimiter or the end of the data. + for tail in ["endobj\n", "endstrea%", "endobj"] { + let pdf = format!("7 0 obj\n<< /Length 5 >>\nstream\nHELLO\n{tail}"); + let (_, o) = parse_indirect(pdf.as_bytes(), 0, &|_| None).unwrap(); + let Object::Stream(s) = o else { panic!("stream expected") }; + assert_eq!(&s.raw[..], b"HELLO", "{tail:?}"); + } + } + + /// A whole `endobj` or `endstrea` token where a wrong /Length stops is payload, not the + /// stream's end, unless the object ends there too: what follows is the end of the data, + /// another object's header, or `xref`, `trailer` or `startxref`. Otherwise the search for + /// `endstream` finds the whole payload. + #[test] + fn an_end_marker_inside_the_payload_does_not_end_the_stream() { + for payload in ["endobj more", "endstrea more", "endstrea endobj more", "endobj 12 more"] { + let pdf = format!("7 0 obj\n<< /Length 5 >>\nstream\nHELLO {payload}\nendstream\nendobj\n"); + let (_, o) = parse_indirect(pdf.as_bytes(), 0, &|_| None).unwrap(); + let Object::Stream(s) = o else { panic!("stream expected") }; + assert_eq!(s.raw[..], *format!("HELLO {payload}").as_bytes(), "{payload}"); + } + // Where the object does end, the /Length is the stream, also with an `endstream` later. + for next in ["8 0 obj\n<< >>", "%c\n12 0 obj", "xref", "trailer", "startxref"] { + for marker in ["endobj", "endstrea\nendobj", "endstrea"] { + let pdf = format!("7 0 obj\n<< /Length 5 >>\nstream\nHELLO\n{marker}\n{next}\nstream\nWORLD\nendstream\nendobj\n"); + let (_, o) = parse_indirect(pdf.as_bytes(), 0, &|_| None).unwrap(); + let Object::Stream(s) = o else { panic!("stream expected") }; + assert_eq!(&s.raw[..], b"HELLO", "{marker:?} then {next:?}"); + } + } + } + /// Without an `endstream`, the /Length is trusted as written when it fits, also when it is + /// wrong, and an indirect one is resolved first; one that doesn't fit, a negative one and one + /// too large for any file are errors rather than wrapped into a short read. + #[test] + fn a_stream_without_endstream_takes_its_length_only_when_it_fits() { + let stream = |length: &str, tail: &str| format!("7 0 obj\n<< /Length {length} >>\nstream\nHELLO{tail}"); + let raw = |pdf: &str, lookup: &dyn Fn(ObjRef) -> Option| match parse_indirect(pdf.as_bytes(), 0, lookup) { + Ok((_, Object::Stream(s))) => Ok(s.raw.to_vec()), + Ok((_, other)) => panic!("{pdf:?} is {other:?}"), + Err(e) => Err(e), + }; + let none = |_: ObjRef| None; + // Wrong but fitting: the bytes it gives, not the bytes up to `endstrea`. + assert_eq!(raw(&stream("3", "\nendstrea\nendobj\n"), &none).unwrap(), b"HEL"); + // Exactly to the end of the file, and nothing at all. + assert_eq!(raw(&stream("5", ""), &none).unwrap(), b"HELLO"); + assert_eq!(raw(&stream("0", "\nendstrea\nendobj\n"), &none).unwrap(), b""); + // Indirect, resolved through the lookup. + let lookup = |r: ObjRef| (r == ObjRef::new(9, 0)).then_some(4); + assert_eq!(raw(&stream("9 0 R", "\nendstrea\nendobj\n"), &lookup).unwrap(), b"HELL"); + assert!(raw(&stream("9 0 R", "\nendstrea\nendobj\n"), &none).is_err(), "an unresolved /Length"); + // One past the end, negative, the largest integer and one beyond it. + for length in ["6", "-1", "-9223372036854775808", "9223372036854775807", "18446744073709551621"] { + assert!(raw(&stream(length, ""), &none).is_err(), "/Length {length}"); + } + } + #[test] fn indirect_stream_with_a_space_before_the_line_break() { // `stream \r\n` as Microsoft Reporting Services writes it: the data starts after the diff --git a/crates/cos/src/security.rs b/crates/cos/src/security.rs index 61c4f92a4c..28813b0ac9 100644 --- a/crates/cos/src/security.rs +++ b/crates/cos/src/security.rs @@ -128,6 +128,46 @@ pub(crate) fn encrypt_dict(doc: &Document, d: &Dict) -> EncryptDict { } } +/// Whether `d`, an `/Encrypt` dictionary, has crypt filters that encrypt (`/CFM` `V2`, `AESV2` or +/// `AESV3`), each authorised only when an embedded file is opened (`/AuthEvent /EFOpen`, ISO +/// 32000-2 §7.6.6, Table 25): what a document asks for its embedded files alone, which it opens +/// without (§7.6.6: "EFOpen Authorization shall be required when accessing embedded files"). The +/// files use them through `/EFF` or through a `/Crypt` filter of their own. Its strings and streams +/// have to be in the clear too, which [`SecurityHandler::without_key`] checks. +pub(crate) fn efopen_only(doc: &Document, d: &Dict) -> bool { + let name = |d: &Dict, k: &[u8]| d.get(k).and_then(|v| doc.resolve(v).as_name().map(<[u8]>::to_vec)); + let Some(cf) = d.get(b"CF").map(|c| doc.resolve(c)).and_then(|c| c.as_dict().cloned()) else { return false }; + let mut encrypting = Vec::new(); + for (k, v) in cf.iter() { + let Some(f) = doc.resolve(v).as_dict().cloned() else { continue }; + if !matches!(name(&f, b"CFM").as_deref(), Some(b"V2" | b"AESV2" | b"AESV3")) { + continue; + } + if name(&f, b"AuthEvent").as_deref() != Some(b"EFOpen") { + return false; + } + encrypting.push(k.clone()); + } + !encrypting.is_empty() +} + +/// Whether `s`, read from a document whose handler has no file key, holds data that handler would +/// decrypt, which therefore reads as stored: still encrypted. +pub(crate) fn stream_reads_as_stored(doc: &Document, h: &SecurityHandler, s: &Stream) -> bool { + if h.has_key() { + return false; + } + let _scope = FollowScope::enter(); + let named = named_crypt_filter(doc, &s.dict); + let ty = s.dict.get(b"Type").map(|t| followed(doc, t)); + let kind = match (&named, ty.as_deref().and_then(Object::as_name)) { + (Some(n), _) => StreamKind::Named(n), + (None, Some(b"EmbeddedFile")) => StreamKind::EmbeddedFile, + _ => StreamKind::Normal, + }; + h.decrypts(kind) +} + /// Whether `f`, an entry of a `/Filter` chain or its only filter, is `/Crypt`. fn is_crypt(doc: &Document, f: &Object) -> bool { followed(doc, f).as_name() == Some(b"Crypt") diff --git a/crates/cos/src/writer.rs b/crates/cos/src/writer.rs index 75d49dbd76..56731f7e39 100644 --- a/crates/cos/src/writer.rs +++ b/crates/cos/src/writer.rs @@ -166,6 +166,24 @@ fn write_stream(s: &Stream, out: &mut Vec) { out.extend_from_slice(b"\nendstream"); } +/// A document opened without the password its embedded files need holds their data as stored and +/// can't encrypt: a save keeps its `/Encrypt` as it is, and writes no edited or added stream that +/// would have to be encrypted (an attached file put in, say), which would be written in the clear. +fn keyless_check(doc: &Document) -> Result<(), CosError> { + if !doc.embedded_files_locked() { + return Ok(()); + } + let refuse = + |what: &str| Err(CosError::Security(format!("{what} without the password of the document's embedded files: open it with the password"))); + if doc.encryption_changed() || doc.trailer().get(b"Encrypt").is_none() { + return refuse("the document's security can't change"); + } + if doc.overlay_entries().any(|(_, _, o)| matches!(o.as_deref(), Some(Object::Stream(s)) if doc.reads_as_stored(s))) { + return refuse("an edited or added embedded file can't be encrypted"); + } + Ok(()) +} + /// The object as written: encrypted with its (output) number when the document is encrypted, /// except the `/Encrypt` dictionary itself (`source_num` is its number in the source document). /// `o` has the document's numbers in its references ([`crate::security::transform`]). @@ -242,6 +260,7 @@ pub fn write_incremental(doc: &Document, opts: &SaveOptions) -> Result, if doc.stream_limited() { return Err(CosError::ReadOnlyLimit); } + keyless_check(doc)?; // Reconstructed files have no chain to append to; added or removed encryption must // rewrite every object; redaction must not leave the old revision behind. All need a full save. if doc.reconstructed() || doc.encryption_changed() || doc.full_save_required() { @@ -384,6 +403,14 @@ pub fn write_full(doc: &Document, opts: &SaveOptions) -> Result, CosErro if doc.stream_limited() { return Err(CosError::ReadOnlyLimit); } + keyless_check(doc)?; + // Renumbering breaks a key that mixes in the object number. + if doc.security().is_some_and(|h| !h.has_key() && !h.survives_renumbering()) { + return Err(CosError::Security( + "the embedded files are encrypted and the document was opened without their password: open it with the password to save it in full" + .into(), + )); + } // A full traversal must not fill the editor's shared caches (which undo snapshots also // own). Parsed objects and decoded object streams are temporary working data here. let mut reader = doc.object_reader(); @@ -762,13 +789,21 @@ fn stamp_mod_date(doc: &mut Document, opts: &SaveOptions) { } } -/// Keep an existing `/ID`, or create one (two equal 16-byte strings, §14.4). +/// Keep any existing two-element `/ID` array as it is, even a malformed one such as `[<> <>]` or `[1 2]` (an ordinary +/// save preserves what it doesn't understand; Save as PDF/A replaces or completes it), or create one with +/// [`new_file_id`] when there is none or it isn't a two-element array. fn ensure_id(trailer: &mut Dict, opts: &SaveOptions, content: &[u8]) { if matches!(trailer.get(b"ID"), Some(Object::Array(a)) if a.len() == 2) { return; } + trailer.set(b"ID".to_vec(), new_file_id(content, opts.id_seed)); +} + +/// A new file identifier for a file whose bytes are `content` (ISO 32000-1 14.4): an array of +/// two equal 16-byte strings, the same for the same content and `seed`. +pub fn new_file_id(content: &[u8], seed: u64) -> Object { // FNV-1a over a sample of the content and the seed: stable, dependency-free. - let mut h: u64 = 0xcbf2_9ce4_8422_2325 ^ opts.id_seed; + let mut h: u64 = 0xcbf2_9ce4_8422_2325 ^ seed; for b in content.iter().step_by((content.len() / 4096).max(1)).chain(content.iter().rev().take(64)) { h ^= *b as u64; h = h.wrapping_mul(0x0100_0000_01b3); @@ -777,7 +812,7 @@ fn ensure_id(trailer: &mut Dict, opts: &SaveOptions, content: &[u8]) { id.extend_from_slice(&h.to_be_bytes()); id.extend_from_slice(&h.rotate_left(29).wrapping_mul(0x9E37_79B9_7F4A_7C15).to_be_bytes()); let s = Object::String(PdfString { bytes: id, hex: true }); - trailer.set(b"ID".to_vec(), Object::Array(vec![s.clone(), s])); + Object::Array(vec![s.clone(), s]) } /// Give a rewritten file a new identifier (§14.4), so that a redacted or sanitized copy does not @@ -919,14 +954,14 @@ pub fn pdf_date(unix_secs: i64) -> String { format!("D:{y:04}{m:02}{d:02}{:02}{:02}{:02}Z", secs / 3600, secs / 60 % 60, secs % 60) } -/// Days since 1970-01-01 → proleptic Gregorian (year, month, day). +/// Days since 1970-01-01 → proleptic Gregorian (year, month, day). Total for any `i64` count: +/// dates past `NaiveDate`'s ±262143-year range keep their proleptic Gregorian year instead of +/// falling back to the epoch. /// -/// A `from_timestamp` epoch fallback would change dates past `NaiveDate`'s ±262143-year range, -/// which the old hand-rolled converter formatted directly. The Gregorian calendar repeats every -/// 400 years (146_097 days), so chrono converts the day within one cycle and whole cycles go back -/// into the year (an `i128` so the extremes can't overflow), keeping chrono as the only calendar -/// logic and reproducing the old result exactly. 1970-01-01 is `num_days_from_ce` 719_163. -fn civil_from_days(days: i64) -> (i64, u32, u32) { +/// The Gregorian calendar repeats every 400 years (146_097 days), so chrono converts the day +/// within one cycle and whole cycles go back into the year (an `i128` so the extremes can't +/// overflow), keeping chrono as the only calendar logic. 1970-01-01 is `num_days_from_ce` 719_163. +pub fn civil_from_days(days: i64) -> (i64, u32, u32) { use chrono::Datelike; let (cycles, rest) = (days.div_euclid(146_097), days.rem_euclid(146_097)); // `rest + 719_163` is 719_163..865_259, always a valid CE day. @@ -1110,6 +1145,12 @@ mod tests { // its sign, exactly as before). assert_eq!(pdf_date(10_000_000_000_000), "D:3188570520174640Z"); assert_eq!(pdf_date(-10_000_000_000_000), "D:-3149180813061320Z"); + assert_eq!(crate::civil_from_days(0), (1970, 1, 1)); + assert_eq!(crate::civil_from_days(-1), (1969, 12, 31)); + assert_eq!(crate::civil_from_days(11_016), (2000, 2, 29)); + assert_eq!(crate::civil_from_days(19_782), (2024, 2, 29)); + assert_eq!(crate::civil_from_days(i64::MAX), (25_252_734_927_768_524, 7, 27)); + assert_eq!(crate::civil_from_days(i64::MIN), (-25_252_734_927_764_585, 6, 7)); } #[test] diff --git a/crates/cos/tests/it/efopen.rs b/crates/cos/tests/it/efopen.rs new file mode 100644 index 0000000000..4bf3470ea2 --- /dev/null +++ b/crates/cos/tests/it/efopen.rs @@ -0,0 +1,224 @@ +//! A document that encrypts only its embedded files, authorised when one is opened (`/AuthEvent +//! /EFOpen`, ISO 32000-2 §7.6.6, Table 25): it opens without a password, its pages read as stored, +//! and its embedded files keep their encrypted bytes, flagged as such, through every save. + +use std::sync::Arc; + +use pdfcraft_cos::{Algorithm, CosError, Document, NewEncryption, ObjRef, Object, SaveOptions, write_full, write_incremental}; +use pdfcraft_crypt::StreamKind; + +const PAGE: &str = "BT /F1 12 Tf 20 100 Td (Visible page) Tj ET"; +const ATTACHED: &[u8] = b"the attached file's own words"; +const PASSWORD: &str = "attach pw"; +const EMBEDDED: ObjRef = ObjRef { num: 6, generation: 0 }; +const CONTENT: ObjRef = ObjRef { num: 4, generation: 0 }; + +fn hex(bytes: &[u8]) -> String { + bytes.iter().map(|b| format!("{b:02X}")).collect() +} + +/// One page in the clear and one attachment encrypted with `/StdCF`, whose `/AuthEvent` is +/// `auth_event`: AES-256 (V5, R6) or AES-128 (V4, R4), `/StmF` and `/StrF` `/Identity`. +fn fixture(algorithm: Algorithm, auth_event: &str) -> Vec { + fixture_with(algorithm, auth_event, false) +} + +/// [`fixture`], the attachment naming `/StdCF` in a `/Crypt` filter of its own when `named`, with +/// no `/EFF` (ISO 32000-2 §7.4.10). +fn fixture_with(algorithm: Algorithm, auth_event: &str, named: bool) -> Vec { + let id0 = [7u8; 16]; + let params = + NewEncryption { algorithm, user_password: PASSWORD, owner_password: "owner pw", permissions: -4, encrypt_metadata: true, seed: [3; 32] }; + let handler = pdfcraft_crypt::create(¶ms, &id0).unwrap(); + let d = handler.dict(); + let kind = if named { StreamKind::Named(b"StdCF") } else { StreamKind::EmbeddedFile }; + let cipher = handler.encrypt_stream(EMBEDDED.num, 0, ATTACHED, kind); + let (eff, crypt) = if named { ("", " /Filter [/Crypt] /DecodeParms [<< /Name /StdCF >>]") } else { (" /EFF /StdCF", "") }; + let (version, cfm, r6) = match algorithm { + Algorithm::Aes256 => ("/V 5 /R 6", "/AESV3 /Length 32", format!("/OE <{}> /UE <{}> /Perms <{}>", hex(&d.oe), hex(&d.ue), hex(&d.perms))), + _ => ("/V 4 /R 4 /Length 128", "/AESV2 /Length 16", String::new()), + }; + let encrypt = format!( + "<< /Filter /Standard {version} /O <{}> /U <{}> {r6} /P {} /CF << /StdCF << /AuthEvent /{auth_event} /CFM {cfm} >> >> /StmF /Identity /StrF /Identity{eff} >>", + hex(&d.o), + hex(&d.u), + d.p + ); + let mut objs: Vec> = vec![ + b"<< /Type /Catalog /Pages 2 0 R /Names << /EmbeddedFiles << /Names [(note.txt) 5 0 R] >> >> >>".to_vec(), + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 300 200] >>".to_vec(), + b"<< /Type /Page /Parent 2 0 R /Contents 4 0 R >>".to_vec(), + format!("<< /Length {} >>\nstream\n{PAGE}\nendstream", PAGE.len()).into_bytes(), + b"<< /Type /Filespec /F (note.txt) /UF (note.txt) /EF << /F 6 0 R >> >>".to_vec(), + ]; + let mut stream = format!("<< /Type /EmbeddedFile{crypt} /Length {} >>\nstream\n", cipher.len()).into_bytes(); + stream.extend_from_slice(&cipher); + stream.extend_from_slice(b"\nendstream"); + objs.push(stream); + objs.push(encrypt.into_bytes()); + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offsets.push(out.len()); + out.extend_from_slice(format!("{} 0 obj\n", i + 1).as_bytes()); + out.extend_from_slice(o); + out.extend_from_slice(b"\nendobj\n"); + } + let xref = out.len(); + out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offsets { + out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + let trailer = format!( + "trailer\n<< /Size {} /Root 1 0 R /Encrypt 7 0 R /ID [<{}> <{}>] >>\nstartxref\n{xref}\n%%EOF\n", + objs.len() + 1, + hex(&id0), + hex(&id0) + ); + out.extend_from_slice(trailer.as_bytes()); + out +} + +fn stream(doc: &Document, r: ObjRef) -> pdfcraft_cos::Stream { + match &*doc.get(r) { + Object::Stream(s) => s.clone(), + other => panic!("object {} isn't a stream: {other:?}", r.num), + } +} + +/// The attachment as the password reads it from `bytes`. +fn attachment_with_password(bytes: Vec) -> Vec { + let doc = Document::open_with_password(Arc::new(bytes), Some(PASSWORD)).unwrap(); + assert!(!doc.embedded_files_locked()); + let embedded = doc.root().and_then(|r| doc.get(r).as_dict().cloned()).unwrap(); + assert!(embedded.get(b"Names").is_some()); + stream(&doc, EMBEDDED).decoded().unwrap() +} + +#[test] +fn a_document_encrypting_only_its_embedded_files_opens_without_a_password() { + let bytes = fixture(Algorithm::Aes256, "EFOpen"); + let doc = Document::open(Arc::new(bytes.clone())).unwrap(); + assert!(doc.embedded_files_locked()); + // Not a repair: the specification has it open so. + assert!(doc.repair_log().is_empty(), "{:?}", doc.repair_log()); + assert!(doc.trailer().get(b"Encrypt").is_some()); + // The page reads as stored; the attachment keeps its encrypted bytes and says so. + let content = stream(&doc, CONTENT); + assert_eq!(content.decoded().unwrap(), PAGE.as_bytes()); + assert!(!doc.reads_as_stored(&content)); + let embedded = stream(&doc, EMBEDDED); + assert!(doc.reads_as_stored(&embedded)); + assert_ne!(embedded.decoded().unwrap(), ATTACHED); + // The password still opens it, and reads the attachment. + assert_eq!(attachment_with_password(bytes.clone()), ATTACHED); + // A wrong password is still refused. + assert!(matches!(Document::open_with_password(Arc::new(bytes), Some("nope")), Err(CosError::WrongPassword))); +} + +#[test] +fn a_crypt_filter_authorised_when_the_document_opens_still_needs_the_password() { + let bytes = fixture(Algorithm::Aes256, "DocOpen"); + assert!(matches!(Document::open(Arc::new(bytes.clone())), Err(CosError::NeedsPassword))); + assert_eq!(attachment_with_password(bytes), ATTACHED); +} + +#[test] +fn saves_keep_the_encrypted_embedded_files_as_stored() { + for algorithm in [Algorithm::Aes256, Algorithm::Aes128] { + let mut doc = Document::open(Arc::new(fixture(algorithm, "EFOpen"))).unwrap(); + assert!(doc.embedded_files_locked(), "{algorithm:?}"); + let root = doc.root().unwrap(); + doc.update_dict(root, |c| c.set(b"PageMode".to_vec(), Object::name("UseAttachments"))).unwrap(); + // An incremental save keeps /Encrypt, and the attachment decrypts with the password. + let saved = write_incremental(&doc, &SaveOptions::default()).unwrap(); + assert_eq!(attachment_with_password(saved), ATTACHED, "{algorithm:?}"); + // A full save renumbers: AES-256 keys don't depend on the number, AES-128 keys do. + match algorithm { + Algorithm::Aes256 => { + let full = write_full(&doc, &SaveOptions::default()).unwrap(); + let reopened = Document::open(Arc::new(full.clone())).unwrap(); + assert!(reopened.embedded_files_locked()); + let doc = Document::open_with_password(Arc::new(full), Some(PASSWORD)).unwrap(); + let names = doc.root().and_then(|r| doc.get(r).as_dict().cloned()).unwrap(); + let tree = doc.resolve(names.get(b"Names").unwrap()).as_dict().cloned().unwrap(); + let files = doc.resolve(tree.get(b"EmbeddedFiles").unwrap()).as_dict().cloned().unwrap(); + let list = doc.resolve(files.get(b"Names").unwrap()).as_array().cloned().unwrap(); + let spec = doc.resolve(&list[1]).as_dict().cloned().unwrap(); + let ef = doc.resolve(spec.get(b"EF").unwrap()).as_dict().cloned().unwrap(); + let file = match &*doc.resolve(ef.get(b"F").unwrap()) { + Object::Stream(s) => s.decoded().unwrap(), + other => panic!("{other:?}"), + }; + assert_eq!(file, ATTACHED); + } + _ => assert!(matches!(write_full(&doc, &SaveOptions::default()), Err(CosError::Security(_))), "{algorithm:?}"), + } + } +} + +#[test] +fn security_cannot_change_without_the_embedded_files_password() { + let mut doc = Document::open(Arc::new(fixture(Algorithm::Aes256, "EFOpen"))).unwrap(); + doc.remove_encryption(); + assert!(matches!(write_full(&doc, &SaveOptions::default()), Err(CosError::Security(_)))); + assert!(matches!(write_incremental(&doc, &SaveOptions::default()), Err(CosError::Security(_)))); +} + +/// From the review of the EFOpen change: the empty password opens it too, and the credentials of a +/// document opened without the key open no other. +#[test] +fn the_empty_password_opens_it_and_keyless_credentials_open_nothing_more() { + let bytes = fixture(Algorithm::Aes256, "EFOpen"); + let doc = Document::open_with_password(Arc::new(bytes.clone()), Some("")).unwrap(); + assert!(doc.embedded_files_locked()); + assert!(doc.open_with_same_credentials(Arc::new(bytes)).unwrap().embedded_files_locked()); + // The same /Encrypt values and /ID, authorised when the document opens. + let doc_open = fixture(Algorithm::Aes256, "DocOpen"); + assert!(matches!(doc.open_with_same_credentials(Arc::new(doc_open.clone())), Err(CosError::NeedsPassword))); + assert!(matches!(Document::open_with_password(Arc::new(doc_open), Some("")), Err(CosError::WrongPassword))); +} + +/// An embedded file that names an EFOpen crypt filter of its own, with no `/EFF`, opens the same way. +#[test] +fn an_embedded_file_naming_its_own_crypt_filter_opens_without_a_password() { + let bytes = fixture_with(Algorithm::Aes256, "EFOpen", true); + let doc = Document::open(Arc::new(bytes.clone())).unwrap(); + assert!(doc.embedded_files_locked()); + assert!(doc.reads_as_stored(&stream(&doc, EMBEDDED))); + assert_eq!(stream(&doc, CONTENT).decoded().unwrap(), PAGE.as_bytes()); + assert_eq!(attachment_with_password(bytes), ATTACHED); +} + +/// A save can't encrypt without the key: removing `/Encrypt` through the trailer, or putting in an +/// embedded file, is refused rather than written in the clear. +#[test] +fn saves_refuse_what_would_need_the_key() { + let open = || Document::open(Arc::new(fixture(Algorithm::Aes256, "EFOpen"))).unwrap(); + let mut doc = open(); + doc.trailer_mut().remove(b"Encrypt"); + assert!(matches!(write_incremental(&doc, &SaveOptions::default()), Err(CosError::Security(_)))); + assert!(matches!(write_full(&doc, &SaveOptions::default()), Err(CosError::Security(_)))); + let mut doc = open(); + let mut replaced = stream(&doc, EMBEDDED); + replaced.raw = b"new words".to_vec().into(); + doc.set(EMBEDDED, Object::Stream(replaced)); + assert!(matches!(write_incremental(&doc, &SaveOptions::default()), Err(CosError::Security(_)))); +} + +/// The password given after opening unlocks the embedded files; a wrong one doesn't. +#[test] +fn the_password_given_later_unlocks_the_embedded_files() { + let mut doc = Document::open(Arc::new(fixture(Algorithm::Aes256, "EFOpen"))).unwrap(); + assert!(matches!(doc.unlock("nope"), Err(CosError::WrongPassword))); + assert!(doc.embedded_files_locked()); + doc.unlock(PASSWORD).unwrap(); + assert!(!doc.embedded_files_locked()); + assert_eq!(stream(&doc, EMBEDDED).decoded().unwrap(), ATTACHED); + assert_eq!(stream(&doc, CONTENT).decoded().unwrap(), PAGE.as_bytes()); + // Not while an embedded file read as stored has been edited. + let mut doc = Document::open(Arc::new(fixture(Algorithm::Aes256, "EFOpen"))).unwrap(); + let edited = stream(&doc, EMBEDDED); + doc.set(EMBEDDED, Object::Stream(edited)); + assert!(matches!(doc.unlock(PASSWORD), Err(CosError::Security(_)))); +} diff --git a/crates/cos/tests/it/main.rs b/crates/cos/tests/it/main.rs index f8e75ba804..0f4d9d7c2b 100644 --- a/crates/cos/tests/it/main.rs +++ b/crates/cos/tests/it/main.rs @@ -2,6 +2,7 @@ mod bounded_writer; mod corpus; +mod efopen; mod encryption; mod generations; mod proptests; diff --git a/crates/cos/tests/it/writer.rs b/crates/cos/tests/it/writer.rs index 2716b9242e..1dc6275b5f 100644 --- a/crates/cos/tests/it/writer.rs +++ b/crates/cos/tests/it/writer.rs @@ -544,3 +544,74 @@ fn incremental_saves_of_a_file_whose_offsets_count_from_its_first_byte_keep_coun // No qpdf check here: qpdf counts every offset from the header in a file with bytes before // it, so it already reconstructs this input before anything is appended. } + +/// One page whose catalog carries a PDF 2.0 document part tree (`/DPartRoot`, ISO 32000-2 +/// 14.12) and whose structure tree root lists structure namespaces (`/Namespaces`, 14.7.4), +/// both through indirect objects that nothing else references. +fn pdf2_fixture() -> Vec { + Pdf::new() + .version("2.0") + .obj("<< /Type /Catalog /Pages 2 0 R /DPartRoot 6 0 R /StructTreeRoot 8 0 R >>") + .obj("<< /Type /Pages /Kids [4 0 R] /Count 1 /MediaBox [0 0 200 300] >>") + .obj("<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>") + .obj("<< /Type /Page /Parent 2 0 R /Contents 5 0 R /Resources << /Font << /F1 3 0 R >> >> >>") + .stream("", "BT /F1 24 Tf 20 150 Td (Part) Tj ET") + .obj("<< /Type /DPartRoot /DPartRootNode 7 0 R /NodeNameList [/Chapter] >>") + .obj("<< /Type /DPart /Parent 6 0 R /Start 4 0 R /End 4 0 R /DPM << /Kind (cover) >> >>") + .obj("<< /Type /StructTreeRoot /K [] /Namespaces [9 0 R] >>") + .obj("<< /Type /Namespace /NS (http://iso.org/pdf2/ssn) /Schema << /Type /Filespec /F (ssn.rnc) >> >>") + .build() +} + +/// What `pdf2_fixture` declares, read back through the catalog. +fn assert_dpart_and_namespaces(doc: &Document) { + let catalog = doc.get(doc.root().unwrap()); + let catalog = catalog.as_dict().unwrap(); + let root = doc.resolve(catalog.get(b"DPartRoot").expect("catalog keeps /DPartRoot")); + let root = root.as_dict().unwrap(); + let node = doc.resolve(root.get(b"DPartRootNode").expect("/DPartRootNode kept")); + let node = node.as_dict().unwrap(); + assert_eq!(node.name(b"Type"), Some(&b"DPart"[..])); + // Both ends of the part are still the one page, found through /Pages (references are + // renumbered by a full save, so they are compared inside this document). + let pages = doc.resolve(catalog.get(b"Pages").unwrap()); + let kids = pages.as_dict().unwrap().get(b"Kids").unwrap().as_array().unwrap(); + let page = kids.first().and_then(|k| k.as_ref()).expect("the page is still the first kid"); + assert_eq!(node.reference(b"Start"), Some(page), "the part still starts at its page"); + assert_eq!(node.reference(b"End"), Some(page), "the part still ends at its page"); + assert!(doc.resolve(node.get(b"DPM").unwrap()).as_dict().unwrap().get(b"Kind").is_some()); + let tree = doc.resolve(catalog.get(b"StructTreeRoot").unwrap()); + let namespaces = doc.resolve(tree.as_dict().unwrap().get(b"Namespaces").expect("/Namespaces kept")); + let namespace = doc.resolve(&namespaces.as_array().unwrap()[0]); + let namespace = namespace.as_dict().unwrap(); + assert_eq!(namespace.get(b"NS").unwrap().as_string().unwrap().bytes, b"http://iso.org/pdf2/ssn"); + assert!(namespace.get(b"Schema").is_some(), "/Schema is retained"); +} + +#[test] +fn document_parts_and_structure_namespaces_survive_full_and_incremental_saves() { + let original = Arc::new(pdf2_fixture()); + let doc = Document::open(original.clone()).unwrap(); + assert_dpart_and_namespaces(&doc); + + for object_streams in [true, false] { + let opts = SaveOptions { object_streams, ..SaveOptions::default() }; + let full = write_full(&doc, &opts).unwrap(); + let back = Document::open(Arc::new(full.clone())).unwrap(); + assert!(back.repair_log().is_empty(), "{:?}", back.repair_log()); + assert_dpart_and_namespaces(&back); + // A second generation, rewritten from the rewritten file. + let second = Document::open(Arc::new(write_full(&back, &opts).unwrap())).unwrap(); + assert!(second.repair_log().is_empty(), "{:?}", second.repair_log()); + assert_dpart_and_namespaces(&second); + } + + // An incremental save of an unrelated edit leaves the file's bytes, and so both, alone. + let mut edited = Document::open(original.clone()).unwrap(); + edited.add(Object::Int(1)); + let updated = write_incremental(&edited, &SaveOptions::default()).unwrap(); + assert!(updated.starts_with(&original)); + let reopened = Document::open(Arc::new(updated)).unwrap(); + assert!(reopened.repair_log().is_empty(), "{:?}", reopened.repair_log()); + assert_dpart_and_namespaces(&reopened); +} diff --git a/crates/create/README.md b/crates/create/README.md index f72f70aa26..95d47d27a9 100644 --- a/crates/create/README.md +++ b/crates/create/README.md @@ -19,10 +19,12 @@ Layer L4. Create a PDF (Acrobat's Create a PDF tool), execution plan M10.2: for one point per pixel, or `ImageResolution::Embedded` for the same behavior as `from_images`. The largest page side is still capped at 14,400 points; - `from_text(text, …)`: plain text set in Helvetica, wrapped and paginated. Text Helvetica - can't draw is refused, naming the line and the character, never set as `?`. + can't draw is set in the bundled Inter face instead (Latin, Greek and Cyrillic letters), with + the same page, margins, wrapping and form feeds; text Inter can't draw either is refused, + naming the line and the character, never set as `?`. - `from_text_escaped(text, …)`: the same for reports PdfCraft generates from documents - (comment summaries, compare reports): such characters are written as `[U+XXXX]` and the first - line says so (`ESCAPED_NOTE`). + (comment summaries, compare reports): text Inter draws is set in it; otherwise the characters + Helvetica can't draw are written as `[U+XXXX]` and the first line says so (`ESCAPED_NOTE`). Every function returns a `pdfcraft_cos::Document`; the caller writes it. diff --git a/crates/create/src/extract.rs b/crates/create/src/extract.rs index 863ad2da0a..46b203a071 100644 --- a/crates/create/src/extract.rs +++ b/crates/create/src/extract.rs @@ -302,9 +302,14 @@ thread_local! { } /// The size a JPEG's header gives, which is what decoding it goes by, whatever its dictionary -/// says: an image of one pixel can hold a header of 500 megapixels. +/// says: an image of one pixel can hold a header of 500 megapixels. `None`, as for a JPEG that +/// can't be read, when its metadata would take the decoder quadratic time +/// ([`decodable_metadata`](crate::upright_image::decodable_metadata)). fn jpeg_size(data: &[u8]) -> Option<(usize, usize)> { use zune_jpeg::zune_core::{bytestream::ZCursor, options::DecoderOptions}; + if !crate::upright_image::decodable_metadata(data) { + return None; + } let options = DecoderOptions::default().set_strict_mode(false).set_max_width(usize::from(u16::MAX)).set_max_height(usize::from(u16::MAX)); let mut dec = zune_jpeg::JpegDecoder::new_with_options(ZCursor::new(data), options); dec.decode_headers().ok()?; @@ -329,6 +334,15 @@ fn jpeg_samples(data: &[u8], space: &Space, w: usize, h: usize) -> Option<(Vec return cmyk_jpeg_samples(data, w, h).map(|px| (px, Space::Cmyk)), _ => {} } + // Past the decoder's metadata bound, drop the Extended XMP chunks rather than the mask: + // decoded as it stands, an over-chunked JPEG is written as is, without its /SMask. + let stripped; + let data = if crate::upright_image::decodable_metadata(data) { + data + } else { + stripped = crate::upright_image::without_extended_xmp(data); + &stripped[..] + }; let img = jpeg_of_size(data, w, h)?; Some(match space { Space::Gray => (img.into_luma8().into_raw(), Space::Gray), @@ -340,6 +354,15 @@ fn jpeg_samples(data: &[u8], space: &Space, w: usize, h: usize) -> Option<(Vec Option> { use zune_jpeg::zune_core::{bytestream::ZCursor, colorspace::ColorSpace, options::DecoderOptions}; + // Past the decoder's metadata bound, decode the image without its Extended XMP chunks + // rather than not at all: written out as stored, a CMYK JPEG shows inverted colours. + let stripped; + let data = if crate::upright_image::decodable_metadata(data) { + data + } else { + stripped = crate::upright_image::without_extended_xmp(data); + &stripped[..] + }; let options = DecoderOptions::default().set_strict_mode(false).set_max_width(usize::from(u16::MAX)).set_max_height(usize::from(u16::MAX)); let mut dec = zune_jpeg::JpegDecoder::new_with_options(ZCursor::new(data), options); dec.decode_headers().ok()?; diff --git a/crates/create/src/lib.rs b/crates/create/src/lib.rs index 149bd99d97..389173d848 100644 --- a/crates/create/src/lib.rs +++ b/crates/create/src/lib.rs @@ -5,9 +5,11 @@ use pdfcraft_cos::{Dict, Document, ObjRef, Object, PdfString, Stream, serialize}; mod extract; +mod text_font; mod upright_image; pub use extract::{ExtractedImage, ImageExport, extract_images, image_file, image_file_within}; -use pdfcraft_fonts::{literal, win_ansi, wrap}; +use pdfcraft_fonts::wrap_with; +use text_font::TextFont; #[derive(Debug, thiserror::Error, PartialEq)] pub enum CreateError { @@ -281,7 +283,8 @@ fn jp2_boxes(b: &[u8]) -> Vec<([u8; 4], &[u8])> { }, n => (8, n), }; - let end = i.saturating_add(len as usize).min(b.len()); + // `len as usize` would truncate on 32-bit targets and put the end before the payload. + let end = i.saturating_add(usize::try_from(len).unwrap_or(usize::MAX)).min(b.len()); if len < head as u64 || end <= i { break; } @@ -428,7 +431,13 @@ fn png_image(name: &str, bytes: &[u8]) -> Result { // The decoded size comes from the header alone, so a file of a few bytes can claim any // size: keep the ceiling, and ask for the memory fallibly (a failed allocation would abort // the whole app, and no `catch_unwind` stops that). - let size = reader.output_buffer_size().filter(|n| *n <= MAX_DECODED_BYTES).ok_or_else(|| bad("image too large".into()))?; + // Flate expands at most 1032:1 and decoding at most 32:1 more (1-bit palette to RGBA), so a + // file cannot honestly hold more than that: refuse a header the bytes cannot back before + // zero-filling a buffer of its size. The slack keeps tiny real images out of the check. + let size = reader + .output_buffer_size() + .filter(|n| *n <= MAX_DECODED_BYTES && *n <= bytes.len().saturating_mul(PNG_MAX_EXPANSION).saturating_add(PNG_SLACK)) + .ok_or_else(|| bad("image too large".into()))?; let mut buf = Vec::new(); buf.try_reserve_exact(size).map_err(|_| bad("not enough memory for this image".into()))?; buf.resize(size, 0); @@ -646,6 +655,11 @@ const MAX_GIF_PIXELS: u64 = 1 << 24; /// twice over. const MAX_DECODED_BYTES: usize = 2 << 30; +/// Most a PNG's decoded size can exceed its file size: Flate's 1032:1 times the 32:1 of +/// expanding a 1-bit palette to RGBA, plus a fixed allowance for small files. +const PNG_MAX_EXPANSION: usize = 1032 * 32; +const PNG_SLACK: usize = 1 << 20; + /// Most decoded bytes all the pages of one TIFF may take together: its pages stay in memory until /// they are embedded, and many small directories can share one compressed strip. Saturates to the /// whole address space on 32-bit targets (wasm32), where 4 GiB doesn't fit in `usize`. @@ -900,14 +914,20 @@ fn turkish_code_page(bytes: &[u8]) -> bool { } /// Plain text set in Helvetica on pages of `page` size with 1-inch margins. Text Helvetica can't -/// draw (outside Western European) is refused with [`CreateError::Invalid`] naming the line and -/// the character; text PdfCraft generates goes through [`from_text_escaped`] instead. +/// draw (outside Western European) is set in the bundled Inter face instead, whole, when Inter +/// draws all of it (Latin, Greek and Cyrillic letters); anything else is refused with +/// [`CreateError::Invalid`] naming the line and the first character neither face draws. Text +/// PdfCraft generates goes through [`from_text_escaped`] instead. pub fn from_text(title: &str, text: &str, page: (f64, f64), font_size: f64) -> Result { - // Helvetica in WinAnsiEncoding: text it can't draw is refused with where it is, not set as `?`. - if let Some((line, c)) = first_undrawable_line(text) { - return Err(CreateError::Invalid(pdfcraft_fonts::undrawable_message(&format!("the text (line {line})"), c))); + // Helvetica in WinAnsiEncoding draws it: set as it always was. + if first_undrawable_line(text).is_none() { + return set_text(title, text, page, font_size, false); } - set_text(title, text, page, font_size) + // Otherwise the bundled face, or a refusal with where it is, never `?`. + if let Some((line, c)) = text_font::first_missing(text) { + return Err(CreateError::Invalid(text_font::undrawable_message(line, c))); + } + set_text(title, &text.chars().map(text_font::drawn_as).collect::(), page, font_size, true) } /// The line (1-based) and character of the first one Helvetica can't draw. Line breaks, tabs and @@ -921,10 +941,15 @@ fn first_undrawable_line(text: &str) -> Option<(usize, char)> { pub const ESCAPED_NOTE: &str = "Characters Helvetica can't draw are written as [U+XXXX] code points."; /// [`from_text`] for text PdfCraft generates from documents (a comment summary, a compare -/// report), which may quote any language. Each character Helvetica can't draw is written as -/// `[U+XXXX]` instead of being refused or set as `?`, so nothing is lost, and the first line says -/// so ([`ESCAPED_NOTE`]). Returns the document and how many characters were written that way. +/// report), which may quote any language. Text the bundled Inter face draws all of is set as +/// [`from_text`] sets it, nothing escaped. Otherwise each character Helvetica can't draw is +/// written as `[U+XXXX]` instead of being refused or set as `?`, so nothing is lost, and the first +/// line says so ([`ESCAPED_NOTE`]). Returns the document and how many characters were written +/// that way. pub fn from_text_escaped(title: &str, text: &str, page: (f64, f64), font_size: f64) -> Result<(Document, usize), CreateError> { + if first_undrawable_line(text).is_some() && text_font::first_missing(text).is_none() { + return Ok((set_text(title, &text.chars().map(text_font::drawn_as).collect::(), page, font_size, true)?, 0)); + } let mut escaped = 0usize; let mut out = String::with_capacity(text.len()); for c in text.chars() { @@ -939,29 +964,25 @@ pub fn from_text_escaped(title: &str, text: &str, page: (f64, f64), font_size: f out = format!("{ESCAPED_NOTE}\n\n{out}"); } // The title is a text string in /Info, which holds any character. - Ok((set_text(title, &out, page, font_size)?, escaped)) + Ok((set_text(title, &out, page, font_size, false)?, escaped)) } -fn set_text(title: &str, text: &str, page: (f64, f64), font_size: f64) -> Result { +/// `text` set on pages of `page` size, in Helvetica or, with `inter`, the bundled Inter face. +fn set_text(title: &str, text: &str, page: (f64, f64), font_size: f64, inter: bool) -> Result { let size = if font_size.is_finite() && font_size > 0.0 { font_size.clamp(4.0, 72.0) } else { 11.0 }; let (w, h) = page; let margin = 72.0; let line_h = size * 1.25; let width = (w - 2.0 * margin).max(36.0); + let mut doc = Document::new_empty(); + let font = if inter { TextFont::inter(&mut doc)? } else { TextFont::helvetica(&mut doc) }; let lines: Vec = text .replace("\r\n", "\n") .replace('\t', " ") .split('\u{c}') - .flat_map(|p| wrap(p, size, width).into_iter().chain(std::iter::once("\u{c}".to_string()))) + .flat_map(|p| wrap_with(p, size, width, |s, size| font.width(s, size)).into_iter().chain(std::iter::once("\u{c}".to_string()))) .collect(); let per_page = (((h - 2.0 * margin) / line_h).floor() as usize).max(1); - let mut doc = Document::new_empty(); - let mut font = Dict::new(); - font.set(b"Type".to_vec(), Object::name("Font")); - font.set(b"Subtype".to_vec(), Object::name("Type1")); - font.set(b"BaseFont".to_vec(), Object::name("Helvetica")); - font.set(b"Encoding".to_vec(), Object::name("WinAnsiEncoding")); - let fr = doc.add(font); let mut tree = Pages::of(&doc)?; let mut page_lines: Vec> = vec![Vec::new()]; for l in lines.iter().take(lines.len().saturating_sub(1)) { @@ -978,12 +999,12 @@ fn set_text(title: &str, text: &str, page: (f64, f64), font_size: f64) -> Result for lines in page_lines { let mut c = format!("BT /F1 {size} Tf {line_h:.3} TL {margin} {:.3} Td\n", h - margin - size).into_bytes(); for l in lines { - c.extend(literal(&win_ansi(&l))); + c.extend(font.show(&l)?); c.extend_from_slice(b" Tj T*\n"); } c.extend_from_slice(b"ET\n"); let mut fonts = Dict::new(); - fonts.set(b"F1".to_vec(), Object::Ref(fr)); + fonts.set(b"F1".to_vec(), Object::Ref(font.font())); let mut res = Dict::new(); res.set(b"Font".to_vec(), Object::Dict(fonts)); tree.add(&mut doc, w, h, res, Some(c)); diff --git a/crates/create/src/tests.rs b/crates/create/src/tests.rs index 88a6e05ac9..d2a8c143e7 100644 --- a/crates/create/src/tests.rs +++ b/crates/create/src/tests.rs @@ -163,7 +163,7 @@ fn text_keeps_leading_spaces_and_tabs() { fn text_helvetica_cannot_draw_is_refused_or_escaped() { let err = from_text("notes", "Café —\tok\r\n\u{c}第二页 notes", LETTER, 11.0).unwrap_err(); let CreateError::Invalid(msg) = &err else { panic!("{err:?}") }; - assert!(msg.contains("(line 2)") && msg.contains("\"第\" (U+7B2C)") && msg.contains("Western European"), "{msg}"); + assert!(msg.contains("(line 2)") && msg.contains("\"第\" (U+7B2C)") && msg.contains("Cyrillic letters"), "{msg}"); // Western European text, tabs, CRLF and form feeds included, still converts. from_text("notes", "Café —\tok\r\n\u{c}Seite zwei: Grüße", LETTER, 11.0).unwrap(); let (doc, escaped) = from_text_escaped("Summary of Comments on 报告.pdf", "Comment: 签名 ok\nPage 2", LETTER, 11.0).unwrap(); @@ -183,6 +183,99 @@ fn text_helvetica_cannot_draw_is_refused_or_escaped() { assert!(!String::from_utf8_lossy(&s.decoded().unwrap()).contains("U+XXXX")); } +/// The decoded content of each page. +fn page_contents(doc: &Document) -> Vec { + pages(doc) + .iter() + .map(|p| { + let Object::Stream(s) = &*doc.resolve(p.get(b"Contents").unwrap()) else { panic!("no content stream") }; + String::from_utf8_lossy(&s.decoded().unwrap()).into_owned() + }) + .collect() +} + +/// The text each page shows, read back through its font's `ToUnicode`, one line per `Tj`. +fn page_text(doc: &Document, page: &Dict) -> Vec { + let res = doc.resolve(page.get(b"Resources").unwrap()).as_dict().cloned().unwrap(); + let fonts = doc.resolve(res.get(b"Font").unwrap()).as_dict().cloned().unwrap(); + let font = doc.resolve(fonts.get(b"F1").unwrap()).as_dict().cloned().unwrap(); + let metrics = pdfcraft_fonts::pdf::Metrics::from_dict(doc, &font); + let Object::Stream(s) = &*doc.resolve(page.get(b"Contents").unwrap()) else { panic!("no content stream") }; + let content = String::from_utf8_lossy(&s.decoded().unwrap()).into_owned(); + content + .split("> Tj") + .filter_map(|part| part.rsplit_once('<').map(|(_, hex)| hex)) + .map(|hex| { + let bytes: Vec = (0..hex.len()).step_by(2).map(|i| u8::from_str_radix(&hex[i..i + 2], 16).unwrap()).collect(); + metrics.decode(&bytes) + }) + .collect() +} + +const PANGRAM: &str = "Жълтата дюля беше щастлива, че пухът, който цъфна, замръзна като гьон."; + +/// Cyrillic text was refused (Helvetica can't draw it). Text with letters Helvetica lacks is set +/// in the bundled Inter face when that draws all of it, its Latin lines too, with Helvetica's +/// page, margins, line height, wrapping, tabs and form feeds; it reads back as typed. +#[test] +fn text_with_cyrillic_is_set_in_the_bundled_face() { + let long: String = (0..120).map(|i| format!("Ред {i}: {PANGRAM}\n")).collect(); + let text = format!("{long}\tТаб, „кавички“, № 5 и Ελληνικά\u{c}Нова страница"); + let doc = reopen(&from_text("бележки", &text, LETTER, 11.0).unwrap()); + let fonts: Vec<_> = doc.scan_objects().filter_map(|(_, o)| o.as_dict().and_then(|d| d.name(b"BaseFont").map(<[u8]>::to_vec))).collect(); + assert!(fonts.iter().any(|f| f == b"Inter") && !fonts.iter().any(|f| f == b"Helvetica"), "{fonts:?}"); + let programs = doc.scan_objects().filter(|(_, o)| matches!(o.as_ref(), Object::Stream(s) if s.dict.get(b"PCFullFont").is_some())).count(); + assert_eq!(programs, 1, "one copy of the face for every page"); + let p = pages(&doc); + let contents = page_contents(&doc); + assert!(p.len() >= 4, "{} pages", p.len()); + assert_eq!(media(&p[0]), [0.0, 0.0, 612.0, 792.0]); + // The same page set-up as Helvetica's: 11 pt, 1.25 line height, 1-inch margins. + assert!(contents[0].starts_with("BT /F1 11 Tf 13.750 TL 72 709.000 Td\n<"), "{}", contents[0]); + assert!(contents.iter().all(|c| !c.contains('?') && !c.contains("[U+"))); + // Each line fits between the margins in Inter's widths, and long lines wrap. + let lines: Vec = p.iter().flat_map(|page| page_text(&doc, page)).collect(); + assert!(lines.len() > 120, "{} lines", lines.len()); + assert!(lines.iter().all(|l| pdfcraft_fonts::inter_width(l, 11.0) <= 612.0 - 144.0 + 1e-6), "{lines:?}"); + let joined = lines.join(" "); + assert!(joined.contains("Ред 0: Жълтата дюля") && joined.contains("Ред 119:"), "{joined}"); + assert!(joined.contains(" Таб, „кавички“, № 5 и Ελληνικά"), "a tab is four spaces: {joined}"); + assert_eq!(page_text(&doc, p.last().unwrap()), ["Нова страница"], "a form feed starts a page"); + // As many lines on a page as in Helvetica. + let latin: String = (0..120).map(|i| format!("Row {i}\n")).collect(); + let cyrillic: String = (0..120).map(|i| format!("Ред {i}\n")).collect(); + let per_page = + |text: &str| page_contents(&from_text("t", text, LETTER, 11.0).unwrap()).iter().map(|c| c.matches("Tj T*").count()).collect::>(); + assert_eq!(per_page(&cyrillic), per_page(&latin)); +} + +/// Text the bundled face can't draw all of keeps the Helvetica way: refused, naming the first +/// character neither face draws, or, for text PdfCraft generates, escaped as before. Text it draws +/// all of is set in it with nothing escaped, and Western European text stays in Helvetica. +#[test] +fn text_the_bundled_face_cannot_draw_is_refused_or_escaped_as_before() { + let err = from_text("t", "Бележка\nПодпис: 签名", LETTER, 11.0).unwrap_err(); + let CreateError::Invalid(msg) = &err else { panic!("{err:?}") }; + assert!(msg.contains("(line 2)") && msg.contains("\"签\" (U+7B7E)"), "{msg}"); + let (doc, escaped) = from_text_escaped("r", "Бележка: 签名", LETTER, 11.0).unwrap(); + assert_eq!(escaped, 9, "every character Helvetica can't draw, as before"); + let shown = page_contents(&reopen(&doc)).join("\n"); + assert!(shown.contains(&format!("({ESCAPED_NOTE}) Tj")) && shown.contains("([U+0411][U+0435]"), "{shown}"); + let (doc, escaped) = from_text_escaped("Обобщение на коментарите", &format!("Коментар: {PANGRAM}\nPage 2"), LETTER, 11.0).unwrap(); + assert_eq!(escaped, 0); + let doc = reopen(&doc); + let shown = page_contents(&doc).join("\n"); + assert!(!shown.contains("[U+") && !shown.contains(ESCAPED_NOTE), "{shown}"); + assert_eq!(page_text(&doc, &pages(&doc)[0]), [format!("Коментар: {PANGRAM}"), "Page 2".to_string()]); + // A soft hyphen, which Inter lacks, is drawn as the hyphen Helvetica draws for it. + let doc = reopen(&from_text("t", "при\u{ad}мер", LETTER, 11.0).unwrap()); + assert_eq!(page_text(&doc, &pages(&doc)[0]), ["при-мер"]); + // Western European text is still set in Helvetica, exactly as before. + let doc = from_text("t", "Café 5€ ™", LETTER, 11.0).unwrap(); + let fonts: Vec<_> = doc.scan_objects().filter_map(|(_, o)| o.as_dict().and_then(|d| d.name(b"BaseFont").map(<[u8]>::to_vec))).collect(); + assert_eq!(fonts, [b"Helvetica".to_vec()]); +} + fn image_of(doc: &Document, page: usize) -> Dict { let p = &pages(doc)[page]; let res = doc.resolve(p.get(b"Resources").unwrap()).as_dict().cloned().unwrap(); @@ -316,6 +409,13 @@ fn a_tiny_png_claiming_huge_dimensions_is_refused_before_allocating() { } } +#[test] +fn a_small_png_claiming_a_large_frame_is_refused_before_zero_filling() { + // 100 MP RGBA = 400 MB is under the 2 GiB ceiling, but 68 bytes cannot hold it. + let r = from_images(&[("big.png".into(), png_claiming(10_000, 10_000))]); + assert!(matches!(&r, Err(CreateError::Image(_, m)) if m.contains("too large"))); +} + /// Likewise a PNG: the png crate's limits don't cover the frame buffer, which its header sizes /// (60000x60000 RGBA is 14.4 GB). Refused before anything is allocated, not after. #[test] @@ -941,6 +1041,43 @@ fn damaged_masks_and_sizes_never_panic() { } } +#[test] +fn a_jpeg_with_thousands_of_extended_xmp_chunks_keeps_its_mask() { + // Export decodes a JPEG with a soft mask, or a JPEG soft mask, to make a PNG with alpha. A JPEG + // whose Extended XMP chunks would take the decoder quadratic time is decoded without them, + // which is linear in what is left, so the mask survives; a JPEG mask past the same bound is + // still left out, without reaching the decoder. + let flood = crate::upright_image::tests::with_extended_xmp_chunks(&real_jpeg(), 5_000); + let mut doc = from_images(&[("flat.png".into(), png_bytes(false))]).unwrap(); + with_image(&mut doc, |doc| { + let mask = doc.add(Object::Stream(Stream::from_raw(image_dict(4, 2, "DeviceGray", 8), vec![0; 8]))); + let mut d = image_dict(4, 2, "DeviceGray", 8); + d.set(b"Filter".to_vec(), Object::name("DCTDecode")); + d.set(b"SMask".to_vec(), Object::Ref(mask)); + Stream::from_raw(d, flood.clone()) + }); + crate::extract::JPEGS_DECODED.with(|n| n.set(0)); + let out = extract_images(&doc, &[0], 0); + assert_eq!(out.images[0].extension, "png"); + assert_eq!(crate::extract::JPEGS_DECODED.with(std::cell::Cell::get), 1); + // The chunks themselves are gone, so the decoder never saw them. + assert!(!out.images[0].data.windows(5).any(|w| w == b"http:")); + + crate::extract::JPEGS_DECODED.with(|n| n.set(0)); + + let mut doc = from_images(&[("flat.png".into(), png_bytes(false))]).unwrap(); + with_image(&mut doc, |doc| { + let mut md = image_dict(4, 2, "DeviceGray", 8); + md.set(b"Filter".to_vec(), Object::name("DCTDecode")); + let mask = doc.add(Object::Stream(Stream::from_raw(md, flood.clone()))); + let mut d = image_dict(4, 2, "DeviceRGB", 8); + d.set(b"SMask".to_vec(), Object::Ref(mask)); + Stream::from_raw(d, vec![10; 24]) + }); + assert_eq!(alphas(&exported_rgba(&doc).1), [255; 8]); + assert_eq!(crate::extract::JPEGS_DECODED.with(std::cell::Cell::get), 0); +} + #[test] fn indexed_image_export_clamps_samples_to_hival() { for palette in [vec![255, 0, 0, 0, 0, 255], vec![255, 0, 0, 0, 0, 255, 0, 255, 0, 255, 255, 255]] { @@ -2788,3 +2925,30 @@ fn proportions_up_to_the_limit_scale_as_they_did() { let mask = noise(20, 10); assert_eq!(crate::extract::scaled(&mask, 20, 10), mask); } + +/// A CMYK JPEG past the decoder's Extended XMP bound is still decoded (its chunks left out) and +/// exported with the colours the page shows, not written as stored, which inverts them. +#[test] +fn a_cmyk_jpeg_flooded_with_extended_xmp_is_exported_in_its_colours() { + let jpeg = crate::upright_image::tests::with_extended_xmp_chunks(&flat_four_component_jpeg([255, 0, 0, 0], 0), 5_000); + let mut doc = from_images(&[("flat.png".into(), png_bytes(false))]).unwrap(); + with_image(&mut doc, |doc| { + let mut d = image_dict(8, 8, "DeviceCMYK", 8); + d.set(b"Filter".to_vec(), Object::name("DCTDecode")); + let m = doc.add(Object::Stream(Stream::from_raw(image_dict(8, 8, "DeviceGray", 8), vec![255; 64]))); + d.set(b"SMask".to_vec(), Object::Ref(m)); + Stream::from_raw(d, jpeg.clone()) + }); + let (_, rgba) = exported_rgba(&doc); + assert!(rgba.as_chunks::<4>().0.iter().all(|p| p[0] <= 6 && p[1] >= 249 && p[2] >= 249), "{:?}", &rgba[..4]); +} + +/// Stripping leaves everything but the Extended XMP chunks, byte for byte. +#[test] +fn stripping_extended_xmp_leaves_the_rest_of_the_jpeg() { + let plain = flat_four_component_jpeg([255, 0, 0, 0], 0); + let flooded = crate::upright_image::tests::with_extended_xmp_chunks(&plain, 3); + assert_ne!(flooded, plain); + // The helper puts a stray byte before the chunks, which the decoder skips: it stays. + assert_eq!(crate::upright_image::without_extended_xmp(&flooded), [&plain[..2], b"\0".as_slice(), &plain[2..]].concat()); +} diff --git a/crates/create/src/text_font.rs b/crates/create/src/text_font.rs new file mode 100644 index 0000000000..555ed3be73 --- /dev/null +++ b/crates/create/src/text_font.rs @@ -0,0 +1,108 @@ +//! The font a text document is set in: standard Helvetica, or, for text Helvetica can't draw, the +//! bundled Inter face (`pdfcraft_fonts::document_latin_fonts`), which every build has and which +//! carries the Latin, Greek and Cyrillic letters WinAnsiEncoding lacks. Text Helvetica draws is +//! set in Helvetica exactly as before. + +use pdfcraft_cos::{Dict, Document, ObjRef, Object}; +use pdfcraft_fonts::pdf::Metrics; +use pdfcraft_fonts::{helvetica_width, literal, win_ansi}; + +use crate::CreateError; + +/// The bundled face, Regular. +fn face() -> Option<&'static pdfcraft_fonts::CraftFont> { + pdfcraft_fonts::document_latin_fonts(false).into_iter().next() +} + +/// Line breaks, tabs and form feeds lay the text out; they are not drawn. +fn layout(c: char) -> bool { + matches!(c, '\n' | '\r' | '\t' | '\u{c}') +} + +/// The character the bundled face draws for `c`: a soft hyphen as a hyphen, as Helvetica draws it +/// (Inter has no glyph for it). +pub(crate) fn drawn_as(c: char) -> char { + if c == '\u{ad}' { '-' } else { c } +} + +/// The line (1-based, lines split at `\n`) and character of the first one the bundled face can't +/// draw, or `None` when it draws all of `text`. +pub(crate) fn first_missing(text: &str) -> Option<(usize, char)> { + let face = face(); + let draws = |s: &str| !s.chars().any(char::is_control) && face.is_some_and(|f| pdfcraft_fonts::embedding::covers(f.bytes, s)); + let mut buf = [0u8; 4]; + text.split('\n').enumerate().find_map(|(i, line)| { + // A whole line at once; the character only for a line the face can't draw. + let drawn: String = line.chars().filter(|c| !layout(*c)).map(drawn_as).collect(); + if draws(&drawn) { + return None; + } + line.chars().find(|&c| !layout(c) && !draws(drawn_as(c).encode_utf8(&mut buf))).map(|c| (i + 1, c)) + }) +} + +/// The refusal for text with `c` on `line`, which neither Helvetica nor the bundled face can draw. +pub(crate) fn undrawable_message(line: usize, c: char) -> String { + format!( + "neither Helvetica nor the bundled Inter face can draw \"{}\" (U+{:04X}) in the text (line {line}); only Latin, Greek and Cyrillic letters can be written there for now", + c.escape_debug(), + u32::from(c) + ) +} + +/// The font a text document is set in. +pub(crate) struct TextFont { + font: ObjRef, + /// The bundled face's metrics; `None` for Helvetica. + inter: Option, +} + +impl TextFont { + /// Standard Helvetica in WinAnsiEncoding, added to `doc`. + pub(crate) fn helvetica(doc: &mut Document) -> Self { + let mut font = Dict::new(); + font.set(b"Type".to_vec(), Object::name("Font")); + font.set(b"Subtype".to_vec(), Object::name("Type1")); + font.set(b"BaseFont".to_vec(), Object::name("Helvetica")); + font.set(b"Encoding".to_vec(), Object::name("WinAnsiEncoding")); + Self { font: doc.add(font), inter: None } + } + + /// The bundled face, embedded in `doc` once for every page. + pub(crate) fn inter(doc: &mut Document) -> Result { + let face = face().ok_or_else(|| CreateError::Invalid("this build has no bundled Latin, Greek and Cyrillic face".into()))?; + let mut fonts = Dict::new(); + let name = pdfcraft_fonts::embedding::embed(doc, &mut fonts, face.bytes, face.family).map_err(CreateError::Invalid)?; + let font = fonts.reference(name.as_bytes()).ok_or_else(|| CreateError::Invalid("the bundled face could not be added".into()))?; + let dict = doc.dict(&Object::Ref(font)).ok_or_else(|| CreateError::Invalid("the bundled face could not be read".into()))?; + Ok(Self { font, inter: Some(Metrics::from_dict(doc, &dict)) }) + } + + /// The font object every page names. + pub(crate) fn font(&self) -> ObjRef { + self.font + } + + /// The advance of `s` at `size`; characters the face lacks count as nothing ([`Self::show`] + /// refuses them). + pub(crate) fn width(&self, s: &str, size: f64) -> f64 { + match &self.inter { + None => helvetica_width(s, size), + Some(m) => m.encode(s).map_or(0.0, |codes| m.codes(&codes).iter().map(|(code, _)| m.width(*code)).sum::() * size), + } + } + + /// The string operand that shows `line`. Helvetica writes what WinAnsiEncoding can't hold as + /// `?`, so its callers check first; the bundled face refuses a character it lacks. + pub(crate) fn show(&self, line: &str) -> Result, CreateError> { + let Some(m) = &self.inter else { return Ok(literal(&win_ansi(line))) }; + let codes = m.encode(line).ok_or_else(|| CreateError::Invalid(format!("the bundled Inter face can't draw \"{}\"", line.escape_debug())))?; + let mut out = Vec::with_capacity(codes.len().saturating_mul(2).saturating_add(2)); + out.push(b'<'); + for b in codes { + out.extend_from_slice(format!("{b:02X}").as_bytes()); + } + out.push(b'>'); + Ok(out) + } +} diff --git a/crates/create/src/upright_image.rs b/crates/create/src/upright_image.rs index b89c517190..d193a38bfb 100644 --- a/crates/create/src/upright_image.rs +++ b/crates/create/src/upright_image.rs @@ -30,7 +30,7 @@ pub(crate) fn upright(bytes: &[u8]) -> Cow<'_, [u8]> { } fn turned(bytes: &[u8]) -> Option> { - if !bytes.starts_with(&[0xFF, 0xD8]) || extended_xmp_chunks(bytes) > MAX_EXTENDED_XMP_CHUNKS { + if !bytes.starts_with(&[0xFF, 0xD8]) || !decodable_metadata(bytes) { return None; } #[cfg(test)] @@ -65,10 +65,41 @@ fn turned(bytes: &[u8]) -> Option> { /// decoder re-sorts and rescans every chunk so far after each marker, quadratic in their number. const MAX_EXTENDED_XMP_CHUNKS: usize = 1024; +/// Whether JPEG `bytes` may be handed to a decoder (`image`'s or `zune-jpeg`'s, which `image` uses), +/// headers included: not when it carries more than [`MAX_EXTENDED_XMP_CHUNKS`]. Every path that +/// decodes a JPEG from a file checks this first: image import here, and image export. +pub(crate) fn decodable_metadata(bytes: &[u8]) -> bool { + extended_xmp_chunks(bytes) <= MAX_EXTENDED_XMP_CHUNKS +} + /// How many Extended XMP chunks (APP1 segments) come before the image data. fn extended_xmp_chunks(bytes: &[u8]) -> usize { + let mut chunks = 0usize; + each_extended_xmp(bytes, |_| chunks += 1); + chunks +} + +/// JPEG `bytes` without their Extended XMP chunks, which nothing here reads: what is left decodes +/// in time linear in its size however many chunks the file carried. +pub(crate) fn without_extended_xmp(bytes: &[u8]) -> Vec { + // Only the bytes that are kept: a CMYK JPEG can be mostly Extended XMP, and reserving the + // whole file for the little that remains aborts the export under a tight address-space limit. + let mut removed = 0usize; + each_extended_xmp(bytes, |r| removed = removed.saturating_add(r.len())); + let mut out = Vec::with_capacity(bytes.len().saturating_sub(removed)); + let mut at = 0; + each_extended_xmp(bytes, |r| { + out.extend_from_slice(bytes.get(at..r.start).unwrap_or_default()); + at = r.end; + }); + out.extend_from_slice(bytes.get(at..).unwrap_or_default()); + out +} + +/// Calls `f` with the byte range of each Extended XMP chunk (APP1 segment) before the image data. +fn each_extended_xmp(bytes: &[u8], mut f: impl FnMut(std::ops::Range)) { const PREFIX: &[u8] = b"http://ns.adobe.com/xmp/extension/\0"; - let (mut i, mut chunks) = (2usize, 0usize); + let mut i = 2usize; while let Some(&byte) = bytes.get(i) { // Stray bytes between segments, which the decoder skips too. if byte != 0xFF { @@ -86,14 +117,16 @@ fn extended_xmp_chunks(bytes: &[u8]) -> usize { _ => { let Some(&[hi, lo]) = bytes.get(i.saturating_add(2)..i.saturating_add(4)) else { break }; let len = usize::from(u16::from_be_bytes([hi, lo])); - if marker == 0xE1 && bytes.get(i.saturating_add(4)..).is_some_and(|seg| seg.starts_with(PREFIX)) { - chunks += 1; + let end = i.saturating_add(2).saturating_add(len.max(2)); + // The prefix must start the APP1 segment's own payload: a zero-length header + // followed by prefix bytes is not a chunk. + if marker == 0xE1 && bytes.get(i.saturating_add(4)..end.min(bytes.len())).is_some_and(|seg| seg.starts_with(PREFIX)) { + f(i..end.min(bytes.len())); } - i = i.saturating_add(2).saturating_add(len.max(2)); + i = end; } } } - chunks } /// The resolution in a JFIF header, if the file has one in inches or centimetres. @@ -120,7 +153,7 @@ fn jfif_density(bytes: &[u8]) -> Option { } #[cfg(test)] -mod tests { +pub(crate) mod tests { use super::*; use crate::tests::phone_jpeg; @@ -138,22 +171,24 @@ mod tests { assert!(matches!(upright(b"\xff\xd8 not a jpeg"), Cow::Borrowed(_))); } + /// `jpeg` with `n` empty chunks of one never-complete Extended XMP packet after its SOI. + pub(crate) fn with_extended_xmp_chunks(jpeg: &[u8], n: usize) -> Vec { + let mut seg = b"http://ns.adobe.com/xmp/extension/\0".to_vec(); + seg.extend_from_slice(&[b'0'; 32]); // the packet's GUID + seg.extend_from_slice(&1000u32.to_be_bytes()); // its full length + seg.extend_from_slice(&0u32.to_be_bytes()); // this chunk's offset + let mut chunk = vec![0xFF, 0xE1]; + chunk.extend_from_slice(&u16::try_from(seg.len() + 2).unwrap().to_be_bytes()); + chunk.extend_from_slice(&seg); + [&jpeg[..2], b"\0".as_slice(), &chunk.repeat(n), &jpeg[2..]].concat() + } + #[test] fn a_jpeg_with_thousands_of_extended_xmp_chunks_is_not_decoded() { // Empty chunks of one never-complete extended packet: the decoder re-sorts and rescans every // chunk so far after each marker, quadratic in their number. - let chunk = |n: usize| { - let mut seg = b"http://ns.adobe.com/xmp/extension/\0".to_vec(); - seg.extend_from_slice(&[b'0'; 32]); // the packet's GUID - seg.extend_from_slice(&1000u32.to_be_bytes()); // its full length - seg.extend_from_slice(&0u32.to_be_bytes()); // this chunk's offset - let mut out = vec![0xFF, 0xE1]; - out.extend_from_slice(&u16::try_from(seg.len() + 2).unwrap().to_be_bytes()); - out.extend_from_slice(&seg); - out.repeat(n) - }; let photo = phone_jpeg(6, 144); - let with = |n: usize| [&photo[..2], b"\0".as_slice(), &chunk(n), &photo[2..]].concat(); + let with = |n: usize| with_extended_xmp_chunks(&photo, n); DECODES.with(|c| c.set(0)); assert!(matches!(upright(&with(4)), Cow::Owned(_)), "a few chunks are an ordinary photo"); assert!(matches!(upright(&with(5_000)), Cow::Borrowed(_))); diff --git a/crates/crypt/src/lib.rs b/crates/crypt/src/lib.rs index e697bc8553..9478e74130 100644 --- a/crates/crypt/src/lib.rs +++ b/crates/crypt/src/lib.rs @@ -234,6 +234,42 @@ impl SecurityHandler { Ok(Self::new(dict, key, auth, recovered_user)) } + /// A handler with no file key, for a document opened without a password whose strings and + /// streams aren't encrypted and whose encrypted crypt filters are all ones the caller found + /// authorised only when an embedded file is opened (`/AuthEvent /EFOpen`, ISO 32000-2 §7.6.6, + /// Table 25). Data such a filter encrypts is left as stored, read and written alike, so it + /// keeps its encrypted bytes; [`SecurityHandler::decrypts`] tells which. `None` when strings or + /// streams are encrypted, which needs the key, or the handler is older than crypt filters (V4). + pub fn without_key(dict: EncryptDict) -> Option { + if dict.v < 4 { + return None; + } + let methods = Methods::new(&dict); + if methods.string != Method::Identity || methods.stream != Method::Identity { + return None; + } + Some(Self { dict, key: Vec::new(), auth: Auth::User, recovered_user: None, methods }) + } + + /// Whether the handler has the file key: false for [`SecurityHandler::without_key`]. + pub fn has_key(&self) -> bool { + !self.key.is_empty() + } + + /// Whether data of `kind` is encrypted, so that reading it needs the file key. + pub fn decrypts(&self, kind: StreamKind) -> bool { + self.method(Some(kind)) != Method::Identity + } + + /// Whether data left as stored by a handler without the key still decrypts after its object + /// is renumbered: only AES-256 doesn't mix the object number into the key (Algorithm 1.A). + pub fn survives_renumbering(&self) -> bool { + [self.methods.string, self.methods.stream, self.methods.embedded_file] + .into_iter() + .chain(self.methods.named.values().copied()) + .all(|m| matches!(m, Method::Identity | Method::Aes256)) + } + /// Upgrade to owner access with the owner (permissions) password, as when a restricted /// document is unlocked after opening. The user password, or any other, is `WrongPassword`. /// Both passwords unlock the same file key, so what was decrypted so far stays valid; a @@ -267,7 +303,7 @@ impl SecurityHandler { /// `/P` as `/Perms` holds it: AES-256 ECB of one block (= CBC with a zero IV), marked `adb`. fn perms_p(&self) -> Option { - if !matches!(self.dict.r, 5 | 6) { + if !matches!(self.dict.r, 5 | 6) || !self.has_key() { return None; } let plain = aes_cbc_decrypt(&self.key, &[0; 16], self.dict.perms.first_chunk::<16>()?, false); @@ -302,23 +338,40 @@ impl SecurityHandler { } } - /// Algorithm 1 (§7.6.3.3): the key for one object. AES-256 uses the file key directly. + /// Algorithm 1 (§7.6.3.2): the key for one object. AES-256 uses the file key directly + /// (Algorithm 1.A, §7.6.3.3). + /// + /// For an RC4 crypt filter under `/V 4`, a file key shorter than 16 bytes is first extended + /// to 16 bytes with zeros. The standard doesn't say so (Algorithm 1 takes the n-byte key as + /// it is), but that is how Acrobat writes and reads such files: a `/V 4` RC4 crypt filter + /// with a 40- or 48-bit key (pdf.js issue 19484). The renderer's `hayro-syntax` reads them + /// the same way, so a page it shows is one this crate can read and save. AES-128 keys are + /// left as they are: `AESV2` takes a 128-bit key (Table 25), and Acrobat's extension is only + /// seen with RC4. Every key PdfCraft writes under `/V 4` is 16 bytes already. fn object_key(&self, method: Method, num: u32, generation: u16) -> Vec { if method == Method::Aes256 { return self.key.clone(); } + let mut key = self.key.clone(); + if method == Method::Rc4 && self.dict.v == 4 && key.len() < 16 { + key.resize(16, 0); + } let mut h = Md5::new(); - h.update(&self.key); + h.update(&key); h.update(&num.to_le_bytes()[..3]); h.update(generation.to_le_bytes()); if method == Method::Aes128 { h.update(b"sAlT"); } let digest = h.finalize(); - digest[..(self.key.len() + 5).min(16)].to_vec() + digest[..(key.len() + 5).min(16)].to_vec() } fn apply(&self, method: Method, num: u32, generation: u16, data: &[u8], encrypt: bool) -> Vec { + // Without the key, encrypted data stays as stored (see `without_key`). + if !self.has_key() { + return data.to_vec(); + } match method { Method::Identity => data.to_vec(), Method::Rc4 => rc4(&self.object_key(method, num, generation), data), diff --git a/crates/crypt/src/tests.rs b/crates/crypt/src/tests.rs index 09a88ed129..a9745d7777 100644 --- a/crates/crypt/src/tests.rs +++ b/crates/crypt/src/tests.rs @@ -280,6 +280,69 @@ proptest::proptest! { } } +/// A `/V 4` RC4 crypt filter with a 40- or 48-bit key, as Acrobat writes it (pdf.js issue 19484): +/// the file key is extended to 16 bytes with zeros before Algorithm 1, so each object's key is +/// all 16 bytes of MD5(key, zeros, object, generation). Everything else keeps Algorithm 1 as the +/// standard has it, n + 5 bytes (at most 16) of MD5(key, object, generation[, "sAlT"]): a `/V 2` +/// file with the same short key, a full 128-bit `/V 4` RC4 key, and AESV2, short (88 bits) or +/// full. Each object key is computed here by hand; data written with it must decrypt. +#[test] +fn a_v4_rc4_key_shorter_than_16_bytes_is_extended_with_zeros() { + let text = b"Page tree in an object stream".to_vec(); + let cases = + [(4, 40, Method::Rc4), (4, 48, Method::Rc4), (2, 40, Method::Rc4), (4, 128, Method::Rc4), (4, 88, Method::Aes128), (4, 128, Method::Aes128)]; + for (v, bits, method) in cases { + let mut d = EncryptDict { + filter: b"Standard".to_vec(), + v, + r: if v == 4 { 4 } else { 3 }, + length_bits: bits, + o: vec![7; 32], + p: -3392, + encrypt_metadata: v != 4, + ..Default::default() + }; + if v == 4 { + d.crypt_filters = vec![(b"StdCF".to_vec(), method)]; + d.stm_f = b"StdCF".to_vec(); + d.str_f = b"StdCF".to_vec(); + } + let key = file_key_r4(&d, ID, b""); + assert_eq!(key.len(), (bits / 8) as usize); + d.u = u_value_r4(&d, ID, &key); + let h = SecurityHandler::open(d, ID, None).unwrap(); + let (num, generation) = (13u32, 0u16); + let extended = v == 4 && method == Method::Rc4 && key.len() < 16; + let mut md5 = Md5::new(); + md5.update(&key); + if extended { + md5.update(vec![0u8; 16 - key.len()]); + } + md5.update(&num.to_le_bytes()[..3]); + md5.update(generation.to_le_bytes()); + if method == Method::Aes128 { + md5.update(b"sAlT"); + } + let digest = md5.finalize(); + let n = if extended { 16 } else { key.len() }; + let object_key = digest[..(n + 5).min(16)].to_vec(); + let case = format!("V{v}, {bits}-bit key, {method:?}"); + let written = if method == Method::Rc4 { + rc4(&object_key, &text) + } else { + let iv = [3u8; 16]; + [iv.to_vec(), aes_cbc_encrypt(&object_key, &iv, &text, true)].concat() + }; + assert_eq!(h.decrypt_string(num, generation, &written), text, "{case}"); + assert_eq!(h.decrypt_stream(num, generation, &written, StreamKind::Normal), text, "{case}"); + let saved = h.encrypt_stream(num, generation, &text, StreamKind::Normal); + if method == Method::Rc4 { + assert_eq!(saved, written, "saved as it was read: {case}"); + } + assert_eq!(h.decrypt_stream(num, generation, &saved, StreamKind::Normal), text, "{case}"); + } +} + #[test] fn owner_authentication_recovers_the_user_password() { for alg in [Algorithm::Rc4_40, Algorithm::Rc4_128, Algorithm::Aes128] { diff --git a/crates/edit/Cargo.toml b/crates/edit/Cargo.toml index 2cd33a275d..c5eae81e40 100644 --- a/crates/edit/Cargo.toml +++ b/crates/edit/Cargo.toml @@ -21,6 +21,9 @@ kurbo = { workspace = true } # Display order of right-to-left lines in added text, and reading and writing order in Edit text; # already in the tree via the interface. unicode-bidi = "0.3" +# Thai word boundaries for rewrapping and Add text (dictionary data compiled in, no LSTM model); +# the ICU4X 2.3 crates it builds on are already in the tree. +icu_segmenter = { version = "2.3", default-features = false, features = ["compiled_data"] } [dev-dependencies] hayro-syntax = { workspace = true } diff --git a/crates/edit/src/added.rs b/crates/edit/src/added.rs index 71a70334c8..98b428ed92 100644 --- a/crates/edit/src/added.rs +++ b/crates/edit/src/added.rs @@ -8,12 +8,8 @@ //! after `/UserUnit` and `/Rotate`), so items stay upright on rotated pages; an item records the //! `/UserUnit` it was written for. -use std::collections::{HashMap, HashSet}; - use pdfcraft_cos::{Dict, Document, ObjRef, Object, PdfString, Stream}; -use pdfcraft_fonts::{ - GlyphError, GlyphOutline, ShapedCluster, ShapedScript, Std14, literal, shape_arabic, shape_hebrew, shape_thai, shaped_glyph, win_ansi, -}; +use pdfcraft_fonts::{GlyphError, GlyphOutline, ShapedScript, Std14, literal, shape_arabic, shape_hebrew, shape_thai, shaped_glyph, win_ansi}; use unicode_bidi::{Level, ParagraphBidiInfo}; use crate::{EditError, MarkKind, check, contents, matrix_number, n, page_list, place_tagged}; @@ -30,7 +26,11 @@ use draw::{GLYPHS_PER_FONT, LaidLine, draw_laid_shaped}; use layout::Piece; #[cfg(test)] pub(crate) use layout::WRAP_CALLS; +#[cfg(test)] +pub(crate) use layout::shaped_width; pub use layout::{PASTE_STEP, first_undrawable, lines, paste_place, paste_rect, text_rect, text_width}; +#[cfg(test)] +use pdfcraft_fonts::ShapedCluster; pub use store::{add_content, delete_content, isolate_content, list_added, scan_markers, update_content}; pub use types::{Added, AddedImage, AddedText, Align, CHAR_SPACING_LIMIT, Family, Ink, MarkerScan}; diff --git a/crates/edit/src/added/draw.rs b/crates/edit/src/added/draw.rs index 90c9e6877b..8e0ec75791 100644 --- a/crates/edit/src/added/draw.rs +++ b/crates/edit/src/added/draw.rs @@ -3,17 +3,17 @@ use super::layout::{ }; use super::types::{AddedText, Align, Family, Ink}; use super::{ - Content, Dict, Document, EditError, GlyphError, GlyphOutline, HashMap, HashSet, ObjRef, Object, PdfString, ShapedCluster, ShapedScript, Stream, - align_code, literal, matrix_number, n, shaped_glyph, win_ansi, + Content, Dict, Document, EditError, GlyphError, GlyphOutline, Object, PdfString, ShapedScript, align_code, literal, matrix_number, n, + shaped_glyph, win_ansi, }; +use pdfcraft_fonts::shaped_text::{self, ShapedFonts}; fn font_name(base: &str) -> String { format!("PCF{}", base.replace('-', "")) } -/// Most glyphs in one Type 3 font: its codes are single bytes, 1–240. An item gets one such font -/// per 240 distinct glyphs it needs. -pub(super) const GLYPHS_PER_FONT: usize = 240; +#[cfg(test)] +pub(super) use pdfcraft_fonts::shaped_text::GLYPHS_PER_FONT; /// A line of a shaped-text item after layout: its text, whether its paragraph runs right to left, /// its paragraph, its pieces in drawing order and its width at the item's size. @@ -61,50 +61,15 @@ pub(super) fn draw_laid_shaped( ) -> Result<(), EditError> { let r = text_rect(t); let latin = font_name(t.family.base_font(t.bold, t.italic)); - // Every distinct cluster (a letter with its dots and the text it stands for) is one glyph of - // one of the fonts, so copy and search get each character exactly once. - let mut keys: Vec<&ShapedCluster> = Vec::new(); - let mut index: HashMap = HashMap::new(); - for c in laid.iter().flat_map(|(_, _, _, pieces, _)| pieces).flat_map(|p| match p { - Piece::Shaped(clusters) => clusters.as_slice(), - Piece::Latin(_) => &[], - }) { - index.entry(cluster_key(c)).or_insert_with(|| { - keys.push(c); - keys.len() - 1 - }); - } - - // Every outline is read before anything is written, so a failure leaves the document as it was. - let mut outlines = HashMap::new(); - for c in &keys { - for (id, _) in &c.glyphs { - if let std::collections::hash_map::Entry::Vacant(entry) = outlines.entry((c.script, *id)) { - entry.insert(outline(c.script, *id).map_err(|e| match c.script { - ShapedScript::Arabic => arabic_error(e, &c.text), - ShapedScript::Hebrew => hebrew_error(e, &c.text), - ShapedScript::Thai => thai_error(e, &c.text), - })?); - } - } - } - // The names the page and this item already use: a new font never takes one of them. - let mut taken_names: HashSet> = taken.iter().chain(fonts.iter()).map(|(name, _)| name.clone()).collect(); - let mut names = Vec::new(); - for chunk in keys.chunks(GLYPHS_PER_FONT) { - let font = type3_shaped(doc, chunk, &outlines); - // A name no font on the page has, so items never replace each other's fonts (object - // numbers alone don't do: a full save renumbers objects but keeps resource names). - let mut name = format!("PCAr{}", font.num); - let mut suffix = 0usize; - while taken_names.contains(name.as_bytes()) { - suffix = suffix.saturating_add(1); - name = format!("PCAr{}_{suffix}", font.num); - } - taken_names.insert(name.clone().into_bytes()); - fonts.set(name.clone().into_bytes(), Object::Ref(font)); - names.push(name); - } + let shaped = + ShapedFonts::new(shaped_text::clusters(laid.iter().map(|(_, _, _, pieces, _)| pieces.as_slice())), outline, taken, fonts, "PCAr", &mut |o| { + Object::Ref(doc.add(o)) + }) + .map_err(|(e, c)| match c.script { + ShapedScript::Arabic => arabic_error(e, &c.text), + ShapedScript::Hebrew => hebrew_error(e, &c.text), + ShapedScript::Thai => thai_error(e, &c.text), + })?; out.extend(format!("BT {}\n", t.fill_op()).bytes()); if t.char_spacing != 0.0 { out.extend(format!("{} Tc\n", n(t.char_spacing)).bytes()); @@ -119,32 +84,17 @@ pub(super) fn draw_laid_shaped( let (x, tw) = placed(t, r, ¶graphs, i, *w, spaces, *rtl); let y = r[3] - (i as f64 * 1.2 + 0.95) * t.size; let mut pen = x; - let mut current: Option<&str> = None; + let mut current = None; for piece in pieces { match piece { Piece::Latin(s) => { out.extend(format!("/{latin} {} Tf {} Tw 1 0 0 1 {} {} Tm ", n(t.size), n(tw), n(pen), n(y)).bytes()); out.extend(literal(&win_ansi(s))); out.extend_from_slice(b" Tj\n"); - current = Some(latin.as_str()); + current = None; pen += latin_width(t, s) + tw * s.matches(' ').count() as f64; } - Piece::Shaped(clusters) => { - for c in clusters { - let Some(&k) = index.get(&cluster_key(c)) else { continue }; - let Some(name) = names.get(k / GLYPHS_PER_FONT) else { continue }; - // Codes start at 1; k % 240 + 1 is at most 240. - let code = u8::try_from(k % GLYPHS_PER_FONT + 1).unwrap_or(1); - if current != Some(name.as_str()) { - out.extend(format!("/{name} {} Tf ", n(t.size)).bytes()); - current = Some(name.as_str()); - } - out.extend(format!("1 0 0 1 {} {} Tm ", n(pen), n(y)).bytes()); - out.extend(literal(&[code])); - out.extend_from_slice(b" Tj\n"); - pen += c.advance * t.size + t.char_spacing + if c.text == " " { tw } else { 0.0 }; - } - } + Piece::Shaped(clusters) => pen = shaped.show(out, clusters, pen, y, t.size, t.char_spacing, tw, &mut current), } } out.extend_from_slice(b"EMC\n"); @@ -153,104 +103,6 @@ pub(super) fn draw_laid_shaped( Ok(()) } -/// What makes two clusters the same Type 3 glyph: text, glyphs, offsets and advance (bit for bit; -/// equal clusters come from the same font units). -type ClusterKey<'a> = (ShapedScript, &'a str, Vec<(u32, u64, u64)>, u64); - -fn cluster_key(c: &ShapedCluster) -> ClusterKey<'_> { - (c.script, c.text.as_str(), c.glyphs.iter().map(|(id, [x, y])| (*id, x.to_bits(), y.to_bits())).collect(), c.advance.to_bits()) -} - -/// A Type 3 font with `clusters` as codes 1…, each drawn from its script face's glyph `outlines`. -fn type3_shaped(doc: &mut Document, clusters: &[&ShapedCluster], outlines: &HashMap<(ShapedScript, u32), GlyphOutline>) -> ObjRef { - let mut charprocs = Dict::new(); - let mut differences = vec![Object::Int(1)]; - let mut widths = Vec::with_capacity(clusters.len()); - let mut bbox = [0.0f64; 4]; - let mut to_unicode = Vec::new(); - for (i, c) in clusters.iter().enumerate() { - let code = i + 1; - // Glyph space is 1000 units per em. - let mut paths = Vec::new(); - let mut ink: Option<[f64; 4]> = None; - for (id, [dx, dy]) in &c.glyphs { - let Some(g) = outlines.get(&(c.script, *id)) else { continue }; - for contour in &g.contours { - for (j, p) in contour.iter().enumerate() { - paths.extend(format!("{} {} {}\n", n((p[0] + dx) * 1000.0), n((p[1] + dy) * 1000.0), if j == 0 { "m" } else { "l" }).bytes()); - } - paths.extend_from_slice(b"h\n"); - let b = [g.bbox[0] + dx, g.bbox[1] + dy, g.bbox[2] + dx, g.bbox[3] + dy]; - ink = Some(ink.map_or(b, |a| [a[0].min(b[0]), a[1].min(b[1]), a[2].max(b[2]), a[3].max(b[3])])); - } - } - // Rounded outwards so no ink is clipped. - let [x0, y0, x1, y1] = - ink.map_or([0.0; 4], |b| [(b[0] * 1000.0).floor(), (b[1] * 1000.0).floor(), (b[2] * 1000.0).ceil(), (b[3] * 1000.0).ceil()]); - bbox = [bbox[0].min(x0), bbox[1].min(y0), bbox[2].max(x1), bbox[3].max(y1)]; - let mut proc = format!("{} 0 {} {} {} {} d1\n", n(c.advance * 1000.0), n(x0), n(y0), n(x1), n(y1)).into_bytes(); - if !paths.is_empty() { - proc.extend(paths); - proc.extend_from_slice(b"f\n"); - } - let glyph = format!("g{code:02X}"); - charprocs.set(glyph.clone().into_bytes(), Object::Ref(doc.add(Object::Stream(Stream::flate(Dict::new(), &proc))))); - differences.push(Object::name(&glyph)); - widths.push(Object::Real((c.advance * 1000.0).round())); - if !c.text.is_empty() { - let hex: String = c.text.encode_utf16().map(|u| format!("{u:04X}")).collect(); - to_unicode.push(format!("<{code:02X}> <{hex}>\n")); - } - } - let mut cmap = String::from( - "/CIDInit /ProcSet findresource begin\n12 dict begin\nbegincmap\n/CMapType 2 def\n1 begincodespacerange\n<01> \nendcodespacerange\n", - ); - // A CMap block holds at most 100 entries. - for block in to_unicode.chunks(100) { - cmap.push_str(&format!("{} beginbfchar\n{}endbfchar\n", block.len(), block.concat())); - } - cmap.push_str("endcmap\nCMapName currentdict /CMap defineresource pop\nend\nend\n"); - let cmap = doc.add(Object::Stream(Stream::flate(Dict::new(), cmap.as_bytes()))); - // The face's family when every glyph comes from one script; without the face, the script's - // name (as Arabic items were always named). - let family = match clusters.first().map(|c| c.script).filter(|s| clusters.iter().all(|c| c.script == *s)) { - Some(script) => script.font().map_or( - match script { - ShapedScript::Arabic => "Arabic", - ShapedScript::Hebrew => "Hebrew", - ShapedScript::Thai => "Thai", - }, - |f| f.family, - ), - None => "PdfCraft Shaped Text", - }; - let mut descriptor = Dict::new(); - descriptor.set(b"Type".to_vec(), Object::name("FontDescriptor")); - descriptor.set(b"FontName".to_vec(), Object::name(&family.replace(' ', ""))); - descriptor.set(b"FontFamily".to_vec(), Object::String(PdfString::literal(family.as_bytes().to_vec()))); - descriptor.set(b"Flags".to_vec(), Object::Int(4)); - descriptor.set(b"ItalicAngle".to_vec(), Object::Int(0)); - let mut encoding = Dict::new(); - encoding.set(b"Type".to_vec(), Object::name("Encoding")); - encoding.set(b"Differences".to_vec(), Object::Array(differences)); - let mut font = Dict::new(); - font.set(b"Type".to_vec(), Object::name("Font")); - font.set(b"Subtype".to_vec(), Object::name("Type3")); - font.set(b"FontDescriptor".to_vec(), Object::Ref(doc.add(Object::Dict(descriptor)))); - font.set(b"FontBBox".to_vec(), Object::Array(bbox.iter().map(|v| Object::Real(*v)).collect())); - font.set( - b"FontMatrix".to_vec(), - Object::Array(vec![Object::Real(0.001), Object::Int(0), Object::Int(0), Object::Real(0.001), Object::Int(0), Object::Int(0)]), - ); - font.set(b"FirstChar".to_vec(), Object::Int(1)); - font.set(b"LastChar".to_vec(), Object::Int(clusters.len() as i64)); - font.set(b"Widths".to_vec(), Object::Array(widths)); - font.set(b"Encoding".to_vec(), Object::Dict(encoding)); - font.set(b"CharProcs".to_vec(), Object::Dict(charprocs)); - font.set(b"ToUnicode".to_vec(), Object::Ref(cmap)); - doc.add(Object::Dict(font)) -} - /// Content and resources for an item; `view` maps display space to user space. `taken` holds the /// page's font names. `existing`: the item is being replaced, so a build without the Arabic or /// Hebrew face keeps drawing that text as `?`, as before it was supported, rather than refusing to @@ -401,9 +253,9 @@ pub(crate) fn thai_glyph_resources( if shaped.glyphs.len() > 65_534 { return Err(EditError::Invalid("shaped row exceeds its geometry budget".into())); } - let mut outlines = HashMap::new(); + let mut outlines = std::collections::HashMap::new(); let mut clusters = Vec::new(); - let mut index = HashMap::new(); + let mut index = std::collections::HashMap::new(); let mut indices = Vec::new(); for glyph in &shaped.glyphs { let id = glyph.id.to_u32(); @@ -414,7 +266,12 @@ pub(crate) fn thai_glyph_resources( let at = *index.entry((id, glyph.text.as_str())).or_insert_with(|| { let at = clusters.len(); let width = outlines.get(&key).map_or(0.0, |g| g.width); - clusters.push(ShapedCluster { script: ShapedScript::Thai, glyphs: vec![(id, [0.0, 0.0])], advance: width, text: glyph.text.clone() }); + clusters.push(pdfcraft_fonts::ShapedCluster { + script: ShapedScript::Thai, + glyphs: vec![(id, [0.0, 0.0])], + advance: width, + text: glyph.text.clone(), + }); at }); indices.push(at); @@ -422,9 +279,9 @@ pub(crate) fn thai_glyph_resources( // Validate all outlines before adding resources. Type 3 character codes are single bytes; // shard long rows and return the appropriate resource/code for every positioned glyph. let mut names = Vec::new(); - for chunk in clusters.chunks(GLYPHS_PER_FONT) { + for chunk in clusters.chunks(shaped_text::GLYPHS_PER_FONT) { let refs: Vec<_> = chunk.iter().collect(); - let font = type3_shaped(doc, &refs, &outlines); + let font = shaped_text::add_type3_font(doc, &refs, &outlines); doc.update_dict(font, |f| f.set(b"Name".to_vec(), Object::name("PCThai")))?; let mut name = format!("PCThai{}", font.num); while fonts.contains(name.as_bytes()) { @@ -436,8 +293,8 @@ pub(crate) fn thai_glyph_resources( indices .into_iter() .map(|at| { - let name = names.get(at / GLYPHS_PER_FONT).ok_or_else(|| EditError::Invalid("Thai font resource is missing".into()))?; - let code = u8::try_from(at % GLYPHS_PER_FONT + 1).map_err(|_| EditError::Invalid("Thai glyph limit".into()))?; + let name = names.get(at / shaped_text::GLYPHS_PER_FONT).ok_or_else(|| EditError::Invalid("Thai font resource is missing".into()))?; + let code = u8::try_from(at % shaped_text::GLYPHS_PER_FONT + 1).map_err(|_| EditError::Invalid("Thai glyph limit".into()))?; Ok((name.clone(), vec![code])) }) .collect() diff --git a/crates/edit/src/added/layout.rs b/crates/edit/src/added/layout.rs index 31a4dc409f..a6a55224d9 100644 --- a/crates/edit/src/added/layout.rs +++ b/crates/edit/src/added/layout.rs @@ -1,7 +1,6 @@ use super::types::{AddedImage, AddedText}; use super::{ - Content, Document, EditError, GlyphError, Level, ObjRef, Object, ParagraphBidiInfo, ShapedCluster, ShapedScript, shape_arabic, shape_hebrew, - shape_thai, + Content, Document, EditError, GlyphError, Level, ObjRef, Object, ParagraphBidiInfo, ShapedScript, shape_arabic, shape_hebrew, shape_thai, }; pub(super) fn nums(doc: &Document, o: Option<&Object>) -> Vec { @@ -55,16 +54,21 @@ pub(super) fn wrapped(t: &AddedText) -> Vec<(String, bool, usize)> { let measure = |s: &str| if rtl_script || thai { shaped_width(t, s, rtl) } else { latin_width(t, s) }; let space = measure(" "); let (mut line, mut line_w) = (String::new(), 0.0); + // Widths add up (the standard-14 widths have no kerning), so each word is measured once + // and appended in place: a line that never fills (negative character spacing) stays + // linear instead of being copied and measured again for every word. for word in para.split(' ') { - let candidate = if line.is_empty() { word.to_string() } else { format!("{line} {word}") }; - let candidate_w = - if rtl_script || thai { if line.is_empty() { measure(word) } else { line_w + space + measure(word) } } else { measure(&candidate) }; + let word_w = measure(word); + let candidate_w = if line.is_empty() { word_w } else { line_w + space + word_w }; if !line.is_empty() && candidate_w > width { out.push((std::mem::take(&mut line), rtl, index)); - line = word.to_string(); - line_w = measure(word); + line.push_str(word); + line_w = word_w; } else { - line = candidate; + if !line.is_empty() { + line.push(' '); + } + line.push_str(word); line_w = candidate_w; } } @@ -73,8 +77,11 @@ pub(super) fn wrapped(t: &AddedText) -> Vec<(String, bool, usize)> { out } -/// Thai wrapping at shaped cluster boundaries never detaches a tone/vowel mark from its base. -/// This is glyph-safe wrapping, not dictionary-based Thai word segmentation. +/// Thai wrapping: a row ends before a word that would overflow it, the words found by the +/// dictionary ([`crate::linebreak`]) inside runs without spaces and at the spaces between them. The +/// spaces where a row ends are dropped, as in the Latin wrapper. Only a word wider than the row is +/// broken inside, at shaped cluster boundaries, which never detach a tone or vowel mark from its +/// base. fn thai_lines(t: &AddedText, para: &str, width: f64) -> Result, EditError> { let (pieces, _) = shaped_layout(t, para, false)?; let mut atoms = Vec::new(); @@ -88,24 +95,88 @@ fn thai_lines(t: &AddedText, para: &str, width: f64) -> Result, Edit })), } } - let mut lines = Vec::new(); - let (mut line, mut used) = (String::new(), 0.0); - for (text, advance) in atoms { - if !line.is_empty() && used + advance > width { - lines.push(std::mem::take(&mut line)); - used = 0.0; - } - line.push_str(&text); - used += advance; - } - lines.push(line); - Ok(lines) + Ok(wrap_words(thai_words(atoms), width)) } -pub(super) fn is_thai(c: char) -> bool { - matches!(u32::from(c), 0x0E00..=0x0E7F) +/// `words` (each a list of text and advance, with the spaces that follow it) laid into rows no +/// wider than `width`: a word that doesn't fit what is left of a row starts the next one, and only +/// a word wider than a whole row is split, between its clusters. +fn wrap_words(words: Vec>, width: f64) -> Vec { + let blank = |s: &str| s.chars().all(char::is_whitespace); + let mut lines: Vec = Vec::new(); + let (mut line, mut used) = (String::new(), 0.0); + let end_row = |lines: &mut Vec, line: &mut String| { + let done = std::mem::take(line); + lines.push(done.trim_end().to_string()); + }; + for word in words { + // The word's own width: the spaces after it may hang past the row's end. + let ink: f64 = word.iter().filter(|(s, _)| !blank(s)).map(|(_, w)| w).sum(); + // A row holding only the spaces that open the paragraph has nothing to end yet. + if !blank(&line) && used + ink > width { + end_row(&mut lines, &mut line); + used = 0.0; + } + // Too wide for what is left of the row, counting the spaces that open the paragraph. + let too_wide = blank(&line) && used + ink > width; + for (text, advance) in word { + if too_wide && !blank(&line) && !blank(&text) && used + advance > width { + end_row(&mut lines, &mut line); + used = 0.0; + } + // A row after a wrap does not begin with the spaces where the last one ended. + if line.is_empty() && !lines.is_empty() && blank(&text) { + continue; + } + line.push_str(&text); + used += advance; + } + } + lines.push(line); + lines } +/// The shaped clusters `atoms` (text and advance) grouped into words: a word starts after spaces, +/// or where [`crate::linebreak::breaks`] lets a row end inside a run without spaces, and keeps the +/// spaces that follow it. A break that falls inside a cluster is not one. +fn thai_words(atoms: Vec<(String, f64)>) -> Vec> { + let text: String = atoms.iter().map(|(s, _)| s.as_str()).collect(); + let mut cuts = Vec::new(); + let mut run_start: Option = None; + for (at, c) in text.char_indices().chain(std::iter::once((text.len(), ' '))) { + match (c.is_whitespace(), run_start) { + (false, None) => { + cuts.push(at); + run_start = Some(at); + } + (true, Some(start)) => { + let run = text.get(start..at).unwrap_or_default(); + cuts.extend(crate::linebreak::breaks(run).into_iter().map(|b| start.saturating_add(b))); + run_start = None; + } + _ => {} + } + } + // `cuts` is in order: each run's start, then the breaks inside it, then the next run's start. + let mut words: Vec> = Vec::new(); + let mut cuts = cuts.into_iter().peekable(); + let mut offset = 0usize; + for atom in atoms { + while cuts.peek().is_some_and(|&c| c < offset) { + cuts.next(); + } + let starts_word = cuts.peek() == Some(&offset); + offset = offset.saturating_add(atom.0.len()); + match words.last_mut() { + Some(word) if !starts_word => word.push(atom), + _ => words.push(vec![atom]), + } + } + words +} + +pub(super) use pdfcraft_fonts::shaped_text::{Piece, is_thai}; + /// Arabic letters, marks, digits and punctuation: an item with any of them is shaped with the /// craft-fonts Arabic face. pub(super) fn is_arabic(c: char) -> bool { @@ -184,13 +255,6 @@ fn mirrored(c: char) -> char { } } -/// A piece of a line with shaped text, in drawing order. -pub(super) enum Piece { - /// Shown in the item's own standard font. - Latin(String), - Shaped(Vec), -} - /// A line of a shaped-text item in drawing order (the Unicode bidirectional algorithm, UAX #9), and /// its width at the item's size. `rtl` is its paragraph's direction. pub(super) fn shaped_layout(t: &AddedText, line: &str, rtl: bool) -> Result<(Vec, f64), EditError> { @@ -244,20 +308,13 @@ pub(super) fn shaped_layout(t: &AddedText, line: &str, rtl: bool) -> Result<(Vec }); } } - let width = pieces - .iter() - .map(|p| match p { - Piece::Latin(s) => latin_width(t, s), - // Each cluster is shown on its own, so the spacing follows each once. - Piece::Shaped(clusters) => clusters.iter().map(|c| c.advance * t.size + t.char_spacing).sum::(), - }) - .sum(); + let width = pdfcraft_fonts::shaped_text::pieces_width(&pieces, t.size, t.char_spacing, |s, _| latin_width(t, s)); Ok((pieces, width)) } /// The width of `s` in a shaped-text item; the item's font's estimate if it can't be shaped (drawing /// then reports why). -fn shaped_width(t: &AddedText, s: &str, rtl: bool) -> f64 { +pub(crate) fn shaped_width(t: &AddedText, s: &str, rtl: bool) -> f64 { shaped_layout(t, s, rtl).map(|(_, w)| w).unwrap_or_else(|_| latin_width(t, s)) } @@ -282,13 +339,7 @@ pub(super) fn hebrew_error(e: GlyphError, text: &str) -> EditError { } pub(super) fn thai_error(e: GlyphError, text: &str) -> EditError { - EditError::Invalid(match e { - GlyphError::NoFont => { - "Thai text needs a Thai font in this build; rebuild with CRAFT_FONTS_DIR pointing to craft-fonts with a Thai face".into() - } - GlyphError::Missing => format!("the Thai font can't show \"{text}\""), - GlyphError::TooComplex => "the Thai glyphs are too complex; use a shorter text box".into(), - }) + EditError::Invalid(pdfcraft_fonts::shaped_text::thai_message(e, text)) } /// The box a text item occupies (height from its lines). @@ -357,3 +408,40 @@ fn copy_key(c: &Content) -> Content { pub fn text_width(t: &AddedText) -> f64 { t.text.split('\n').map(|line| shaped_width(t, line, unicode_bidi::get_base_direction(line) == unicode_bidi::Direction::Rtl)).fold(0.0, f64::max) } + +#[cfg(test)] +mod thai_wrap_tests { + use super::{thai_words, wrap_words}; + + /// One atom per character, each `advance` wide: what shaping hands over, without a Thai face. + fn atoms(text: &str, advance: f64) -> Vec<(String, f64)> { + text.chars().map(|c| (c.to_string(), advance)).collect() + } + + fn texts(words: &[Vec<(String, f64)>]) -> Vec { + words.iter().map(|w| w.iter().map(|(s, _)| s.as_str()).collect()).collect() + } + + #[test] + fn thai_atoms_group_into_dictionary_words_with_their_spaces() { + assert_eq!(texts(&thai_words(atoms("น้ำประปาสะอาด", 1.0))), ["น้ำ", "ประปา", "สะอาด"]); + assert_eq!(texts(&thai_words(atoms("สวัสดี ครับ", 1.0))), ["สวัสดี ", "ครับ"]); + // A break the dictionary finds inside one shaped cluster is not a place to end a row. + assert_eq!(texts(&thai_words(vec![("น้ำประปา".into(), 8.0)])), ["น้ำประปา"]); + assert!(thai_words(Vec::new()).is_empty()); + } + + #[test] + fn words_wrap_whole_and_only_an_overlong_word_is_split() { + let words = || vec![vec![("ab".into(), 2.0), (" ".into(), 1.0)], vec![("cd".into(), 2.0)]]; + assert_eq!(wrap_words(words(), 10.0), ["ab cd"]); + // The space that ends a row hangs past it, and the next row doesn't begin with it. + assert_eq!(wrap_words(words(), 3.0), ["ab", "cd"]); + let long = vec![vec![("a".into(), 2.0), ("b".into(), 2.0), ("c".into(), 2.0)]]; + assert_eq!(wrap_words(long, 3.0), ["a", "b", "c"]); + // Spaces that open a paragraph stay, and a word too wide for the rest of their row is split. + let opening = vec![vec![(" ".into(), 1.0)], vec![("a".into(), 2.0), ("b".into(), 2.0)]]; + assert_eq!(wrap_words(opening, 3.0), [" a", "b"]); + assert_eq!(wrap_words(Vec::new(), 3.0), [""]); + } +} diff --git a/crates/edit/src/added/store.rs b/crates/edit/src/added/store.rs index 3555910f04..1ee4c9c4cd 100644 --- a/crates/edit/src/added/store.rs +++ b/crates/edit/src/added/store.rs @@ -119,6 +119,8 @@ pub fn scan_markers(doc: &Document) -> MarkerScan { Some(MarkKind::Watermark) } else if tag == Some(&b"Background"[..]) { Some(MarkKind::Background) + } else if tag == Some(&b"PrinterMarks"[..]) { + Some(MarkKind::PrinterMarks) } else { None }; @@ -328,8 +330,15 @@ pub fn isolate_content(doc: &mut Document, page: usize, index: usize) -> Result< /// names them so), so they go when the item is replaced or deleted. fn own_fonts(doc: &Document, obj: ObjRef) -> Vec> { let Object::Stream(s) = &*doc.get(obj) else { return Vec::new() }; - let Ok(data) = s.decoded() else { return Vec::new() }; - data.split(u8::is_ascii_whitespace).filter_map(|w| w.strip_prefix(b"/")).filter(|w| w.starts_with(b"PCAr")).map(<[u8]>::to_vec).collect() + // The stream is one this writer put there, so its decode is bounded by the stream itself: the + // page limit alone would refuse an item larger than a page's content, and its fonts would stay + // on the page's /Font, which `write` and `delete_content` are supposed to take them off. + let limit = pdfcraft_cos::MAX_PAGE_DECODED.max(s.raw.len().saturating_mul(16)); + let Ok(data) = s.decoded_within(limit) else { return Vec::new() }; + // Each name once: a stream that repeats a name is not copied out once per use. + let names: std::collections::BTreeSet<&[u8]> = + data.split(u8::is_ascii_whitespace).filter_map(|w| w.strip_prefix(b"/")).filter(|w| w.starts_with(b"PCAr")).collect(); + names.into_iter().map(<[u8]>::to_vec).collect() } fn drop_fonts(doc: &Document, res: &mut Dict, names: &[Vec]) { diff --git a/crates/edit/src/apply.rs b/crates/edit/src/apply.rs index 0d0213904b..8b52c81c76 100644 --- a/crates/edit/src/apply.rs +++ b/crates/edit/src/apply.rs @@ -1,9 +1,12 @@ +use pdfcraft_fonts::shaped_text::{is_thai, thai_refusal, thai_width}; + use super::marks::{ - Background, Context, HeaderFooter, MarkKind, Watermark, add_picture_resource, bates_start, expand, matrix_number, n, picture, rgb, shrink_matrix, + Background, Context, HeaderFooter, MarkKind, MarkText, Watermark, add_picture_resource, bates_start, expand, matrix_number, n, picture, rgb, + shrink_matrix, }; use super::place::{ END, Isolation, add_page_resources, add_resources, begin, check, check_on_top, contents, free_font_name, isolation, page_fonts, page_list, place, - place_inside_wrapper, stream_tag, text_op, undrawable, wrapper_role, + place_inside_wrapper, stream_tag, undrawable, wrapper_role, }; use super::{Document, EditError, Object, added, backdrop, helvetica_width}; @@ -31,6 +34,9 @@ fn add_header_footer_in_place(doc: &mut Document, pages: &[usize], hf: &HeaderFo if hf.text.iter().all(|t| t.trim().is_empty()) { return Err(EditError::Invalid("type the header or footer text first".into())); } + if let Some(why) = hf.text.iter().find_map(|t| thai_refusal(t)) { + return Err(EditError::Invalid(why)); + } if let Some(c) = hf.first_undrawable() { return Err(undrawable(c)); } @@ -73,12 +79,23 @@ fn add_header_footer_in_place(doc: &mut Document, pages: &[usize], hf: &HeaderFo } let all = page_list(doc); let count = all.len(); + let number_of = |k: usize| { + u32::try_from(u64::from(hf.start_number) + k as u64).map_err(|_| EditError::Invalid("the header page numbers are too large".into())) + }; + let bates_of = |k: usize| bates0.checked_add(k as u64).ok_or_else(|| EditError::Invalid("the Bates numbers are too large".into())); + // Every Thai line drawn on any page (a Bates prefix may join a template's Thai), for its fonts. + let mut thai_lines = std::collections::BTreeSet::new(); + if hf.text.iter().any(|t| t.chars().any(is_thai)) { + for k in 0..pages.len() { + let (number, bates) = (number_of(k)?, bates_of(k)?); + thai_lines.extend(hf.text.iter().flat_map(|t| expand(t, number, count, bates, cx).lines().map(str::to_owned).collect::>())); + } + } + let thai = MarkText::new(doc, &all, pages, thai_lines.iter().map(String::as_str))?; for (k, &i) in pages.iter().enumerate() { let page = &all[i]; let (w, h) = page.display_size(doc); - let number = - u32::try_from(u64::from(hf.start_number) + k as u64).map_err(|_| EditError::Invalid("the header page numbers are too large".into()))?; - let bates = bates0.checked_add(k as u64).ok_or_else(|| EditError::Invalid("the Bates numbers are too large".into()))?; + let (number, bates) = (number_of(k)?, bates_of(k)?); let size = hf.font_size; let [top, bottom, left, right] = hf.margins; // The page resource the face is added under: the usual name, unless the page already @@ -91,6 +108,7 @@ fn add_header_footer_in_place(doc: &mut Document, pages: &[usize], hf: &HeaderFo let base = hf.font.base_font(false, false); let resource = free_font_name(doc, &page_fonts(doc, page), preferred, base); let mut body: Vec = format!("BT\n/{resource} {} Tf\n{}\n", n(size), rgb(hf.color)).into_bytes(); + let mut current = None; let mut underlines = String::new(); for (slot, template) in hf.text.iter().enumerate() { if template.trim().is_empty() { @@ -101,7 +119,8 @@ fn add_header_footer_in_place(doc: &mut Document, pages: &[usize], hf: &HeaderFo let line_h = size * 1.2; for (li, line) in lines.iter().enumerate() { // Centred and right-aligned text is placed by its width in the chosen face. - let tw = hf.font.width(line, size, false, false); + let face = |s: &str, size: f64| hf.font.width(s, size, false, false); + let tw = thai_width(line, size, face); let x = match slot % 3 { 0 => left, 1 => (w - tw) / 2.0, @@ -109,7 +128,7 @@ fn add_header_footer_in_place(doc: &mut Document, pages: &[usize], hf: &HeaderFo }; // Headers hang below the top margin; footers sit on the bottom margin. let y = if header { h - top - size * 0.8 - li as f64 * line_h } else { bottom + (lines.len() - 1 - li) as f64 * line_h }; - body.extend(text_op(x, y, line)); + thai.show(&mut body, line, x, y, size, &resource, |s| face(s, size), &mut current); if hf.underline { underlines.push_str(&format!("{} {} {} {} re f\n", n(x), n(y - size * 0.15), n(tw), n((size * 0.06).max(0.4)))); } @@ -126,6 +145,7 @@ fn add_header_footer_in_place(doc: &mut Document, pages: &[usize], hf: &HeaderFo if !page_fonts(doc, page).contains(resource.as_bytes()) { add_page_resources(doc, page, None, &[(resource.into_bytes(), base)])?; } + thai.add_to(doc, &page_list(doc)[i])?; let page = &page_list(doc)[i]; place(doc, page, MarkKind::HeaderFooter, content, false)?; if let Some(Some(m)) = shrinks.get(k) { @@ -150,6 +170,9 @@ fn add_watermark_in_place(doc: &mut Document, pages: &[usize], wm: &Watermark, r if text.is_empty() && wm.source.is_none() { return Err(EditError::Invalid("type the watermark text first".into())); } + if let Some(why) = wm.source.is_none().then(|| thai_refusal(text)).flatten() { + return Err(EditError::Invalid(why)); + } if let Some(c) = wm.first_undrawable() { return Err(undrawable(c)); } @@ -171,6 +194,7 @@ fn add_watermark_in_place(doc: &mut Document, pages: &[usize], wm: &Watermark, r } let opacity = wm.opacity.clamp(0.0, 1.0); let lines: Vec<&str> = text.lines().collect(); + let thai = if wm.source.is_none() { MarkText::new(doc, &page_list(doc), pages, lines.iter().copied())? } else { MarkText::default() }; for &i in pages { let page = page_list(doc)[i].clone(); let (w, h) = page.display_size(doc); @@ -185,7 +209,7 @@ fn add_watermark_in_place(doc: &mut Document, pages: &[usize], wm: &Watermark, r place(doc, page, MarkKind::Watermark, content.into_bytes(), wm.behind)?; continue; } - let widest = lines.iter().map(|l| helvetica_width(l, 1.0)).fold(0.0, f64::max).max(0.01); + let widest = lines.iter().map(|l| thai_width(l, 1.0, helvetica_width)).fold(0.0, f64::max).max(0.01); let size = if wm.font_size > 0.0 { wm.font_size } else { ((w * w + h * h).sqrt() * 0.5 / widest).clamp(6.0, 300.0) }; let (s, c) = wm.rotation.to_radians().sin_cos(); let (cx, cy) = (w / 2.0 + wm.offset[0], h / 2.0 + wm.offset[1]); @@ -207,13 +231,24 @@ fn add_watermark_in_place(doc: &mut Document, pages: &[usize], wm: &Watermark, r ); let line_h = size * 1.15; let total = line_h * (lines.len() as f64 - 1.0); + let mut current = None; for (li, line) in lines.iter().enumerate() { - let tw = helvetica_width(line, size); - content.extend(text_op(-tw / 2.0, total / 2.0 - li as f64 * line_h - size * 0.35, line)); + let tw = thai_width(line, size, helvetica_width); + thai.show( + &mut content, + line, + -tw / 2.0, + total / 2.0 - li as f64 * line_h - size * 0.35, + size, + "PCHelv", + |s| helvetica_width(s, size), + &mut current, + ); } content.extend_from_slice(b"ET\n"); content.extend_from_slice(END.as_bytes()); add_resources(doc, &page, Some(opacity), None)?; + thai.add_to(doc, &page_list(doc)[i])?; let page = &page_list(doc)[i]; place(doc, page, MarkKind::Watermark, content, wm.behind)?; } diff --git a/crates/edit/src/colour.rs b/crates/edit/src/colour.rs new file mode 100644 index 0000000000..7eaab55384 --- /dev/null +++ b/crates/edit/src/colour.rs @@ -0,0 +1,230 @@ +//! Colours as sRGB through the colour space in force, for Convert colours: `g`/`rg`/`k`, and +//! `cs` + `sc`/`scn` through the space named. Device spaces, ICCBased by its component count (or +//! its Alternate), CalRGB and CalGray as their device kin, Lab through CIE XYZ with the +//! dictionary's white point, Indexed through its table, Separation and DeviceN through a Type 2 +//! (exponential) tint transform into the alternate space; a sampled or PostScript tint function, +//! or a pattern, falls back to a plain darkness so nothing reads as the wrong hue. + +use pdfcraft_cos::{Dict, Document, Object}; + +/// A colour space as far as a fill colour needs it. +#[derive(Clone, Debug, PartialEq)] +pub(crate) enum Space { + Gray, + Rgb, + Cmyk, + Lab { + white: [f64; 3], + range: [f64; 4], + }, + Indexed { + base: Box, + hival: usize, + table: Vec, + }, + /// Separation and DeviceN: the number of inputs, the alternate space, and the tint + /// transform when it is a Type 2 function (C0, C1, N). + Tint { + inputs: usize, + alt: Box, + exp: Option<(Vec, Vec, f64)>, + }, + Pattern, +} + +impl Space { + /// How many numbers set a colour in this space. + fn components(&self) -> usize { + match self { + Space::Gray => 1, + Space::Rgb | Space::Lab { .. } => 3, + Space::Cmyk => 4, + Space::Indexed { .. } => 1, + Space::Tint { inputs, .. } => *inputs, + Space::Pattern => 0, + } + } +} + +fn nums(doc: &Document, o: &Object) -> Vec { + doc.resolve(o).as_array().map(|a| a.iter().filter_map(|v| doc.resolve(v).as_f64()).collect()).unwrap_or_default() +} + +/// The colour space an operand of `cs` names: a device space, or a name in the resources' +/// `/ColorSpace` dictionary, or (inside arrays) a family. +pub(crate) fn space_of(doc: &Document, operand: &Object, cs_res: &Dict, depth: u8) -> Space { + if depth > 4 { + return Space::Gray; + } + let resolved = doc.resolve(operand); + match &*resolved { + Object::Name(n) => match n.as_slice() { + b"DeviceGray" | b"G" | b"CalGray" => Space::Gray, + b"DeviceRGB" | b"RGB" | b"CalRGB" => Space::Rgb, + b"DeviceCMYK" | b"CMYK" => Space::Cmyk, + b"Pattern" => Space::Pattern, + other => match cs_res.get(other) { + Some(o) => space_of(doc, o, &Dict::default(), depth + 1), + None => Space::Gray, + }, + }, + Object::Array(a) => { + let family = a.first().map(|f| doc.resolve(f)).and_then(|f| f.as_name().map(|n| n.to_vec())).unwrap_or_default(); + match family.as_slice() { + b"ICCBased" => { + let stream = a.get(1).map(|s| doc.resolve(s)); + let dict = stream.as_ref().and_then(|s| match &**s { + Object::Stream(st) => Some(st.dict.clone()), + _ => None, + }); + if let Some(alt) = dict.as_ref().and_then(|d| d.get(b"Alternate").cloned()) { + return space_of(doc, &alt, cs_res, depth + 1); + } + match dict.and_then(|d| d.int(b"N")) { + Some(1) => Space::Gray, + Some(4) => Space::Cmyk, + _ => Space::Rgb, + } + } + b"CalRGB" => Space::Rgb, + b"CalGray" => Space::Gray, + b"Lab" => { + let d = a.get(1).map(|d| doc.resolve(d)).and_then(|d| d.as_dict().cloned()).unwrap_or_default(); + let wp = d.get(b"WhitePoint").map(|w| nums(doc, w)).unwrap_or_default(); + let white = if wp.len() == 3 { [wp[0], wp[1], wp[2]] } else { [0.9505, 1.0, 1.089] }; + let r = d.get(b"Range").map(|w| nums(doc, w)).unwrap_or_default(); + let range = if r.len() == 4 { [r[0], r[1], r[2], r[3]] } else { [-100.0, 100.0, -100.0, 100.0] }; + Space::Lab { white, range } + } + b"Indexed" | b"I" => { + let base = a.get(1).map(|b| space_of(doc, b, cs_res, depth + 1)).unwrap_or(Space::Rgb); + let hival = a.get(2).and_then(|h| doc.resolve(h).as_f64()).unwrap_or(0.0).clamp(0.0, 255.0) as usize; + let table = match a.get(3).map(|t| doc.resolve(t)) { + Some(t) => match &*t { + Object::String(s) => s.bytes.clone(), + Object::Stream(st) => st.decoded().unwrap_or_default(), + _ => Vec::new(), + }, + None => Vec::new(), + }; + Space::Indexed { base: Box::new(base), hival, table } + } + b"Separation" | b"DeviceN" => { + // The `/None` colorant paints nothing (ISO 32000-2, 8.6.6.4): given a colour + // of its own it would appear, so it stays what it is, as a pattern does. + let is_none = |o: &Object| doc.resolve(o).as_name() == Some(b"None"); + let none = a.get(1).is_some_and(|names| { + is_none(names) || doc.resolve(names).as_array().is_some_and(|names| !names.is_empty() && names.iter().all(is_none)) + }); + if none { + return Space::Pattern; + } + let inputs = if family == b"Separation" { + 1 + } else { + a.get(1).map(|n| doc.resolve(n)).and_then(|n| n.as_array().map(Vec::len)).unwrap_or(1).max(1) + }; + let alt = a.get(2).map(|b| space_of(doc, b, cs_res, depth + 1)).unwrap_or(Space::Gray); + let exp = a.get(3).map(|f| doc.resolve(f)).and_then(|f| { + let d = match &*f { + Object::Dict(d) => d.clone(), + Object::Stream(s) => s.dict.clone(), + _ => return None, + }; + if d.int(b"FunctionType") != Some(2) { + return None; + } + let c0 = d.get(b"C0").map(|v| nums(doc, v)).unwrap_or_else(|| vec![0.0]); + let c1 = d.get(b"C1").map(|v| nums(doc, v)).unwrap_or_else(|| vec![1.0]); + let n = d.get(b"N").and_then(|v| doc.resolve(v).as_f64()).unwrap_or(1.0); + Some((c0, c1, n)) + }); + Space::Tint { inputs, alt: Box::new(alt), exp } + } + b"Pattern" => Space::Pattern, + b"DeviceGray" | b"G" => Space::Gray, + b"DeviceRGB" | b"RGB" => Space::Rgb, + b"DeviceCMYK" | b"CMYK" => Space::Cmyk, + _ => Space::Gray, + } + } + _ => Space::Gray, + } +} + +fn clamp01(v: f64) -> f64 { + if v.is_finite() { v.clamp(0.0, 1.0) } else { 0.0 } +} + +/// CMYK to sRGB, the usual multiplicative approximation (no ICC profile). +pub(crate) fn cmyk(c: f64, m: f64, y: f64, k: f64) -> [f64; 3] { + let (c, m, y, k) = (clamp01(c), clamp01(m), clamp01(y), clamp01(k)); + [(1.0 - c) * (1.0 - k), (1.0 - m) * (1.0 - k), (1.0 - y) * (1.0 - k)] +} + +/// CIE L*a*b* (with the dictionary's white point) to sRGB, through XYZ and the D65 sRGB matrix. +fn lab(white: [f64; 3], l: f64, a: f64, b: f64) -> [f64; 3] { + let m = (l + 16.0) / 116.0; + let (ll, mm, nn) = (m + a / 500.0, m, m - b / 200.0); + let g = |x: f64| if x >= 6.0 / 29.0 { x * x * x } else { 108.0 / 841.0 * (x - 4.0 / 29.0) }; + let (xw, yw, zw) = (white[0].max(1e-6), white[1].max(1e-6), white[2].max(1e-6)); + let (x, y, z) = (xw * g(ll), yw * g(mm), zw * g(nn)); + // Scale the white point to D65 before the sRGB matrix (a plain von Kries over XYZ). + let (x, y, z) = (x * 0.9505 / xw, y / yw, z * 1.089 / zw); + let lin = [3.2406 * x - 1.5372 * y - 0.4986 * z, -0.9689 * x + 1.8758 * y + 0.0415 * z, 0.0557 * x - 0.2040 * y + 1.0570 * z]; + lin.map(|c| { + let c = clamp01(c); + if c <= 0.0031308 { 12.92 * c } else { 1.055 * c.powf(1.0 / 2.4) - 0.055 } + }) +} + +/// The sRGB of `values` in `space` (missing values read as 0; a pattern reads as black). +pub(crate) fn rgb(space: &Space, values: &[f64]) -> [f64; 3] { + let v = |i: usize| values.get(i).copied().unwrap_or(0.0); + match space { + Space::Gray => { + let g = clamp01(v(0)); + [g, g, g] + } + Space::Rgb => [clamp01(v(0)), clamp01(v(1)), clamp01(v(2))], + Space::Cmyk => cmyk(v(0), v(1), v(2), v(3)), + Space::Lab { white, range } => { + let l = v(0).clamp(0.0, 100.0); + let a = v(1).clamp(range[0].min(range[1]), range[1].max(range[0])); + let b = v(2).clamp(range[2].min(range[3]), range[3].max(range[2])); + lab(*white, l, a, b) + } + Space::Indexed { base, hival, table } => { + let n = base.components().max(1); + let i = (v(0).max(0.0).round() as usize).min(*hival); + let vals: Vec = (0..n).map(|k| table.get(i * n + k).map(|b| *b as f64 / 255.0).unwrap_or(0.0)).collect(); + // Lab tables hold the L*, a*, b* range in bytes; the others are 0..1 already. + match &**base { + Space::Lab { range, .. } => { + let scaled = [ + vals[0] * 100.0, + range[0] + vals.get(1).copied().unwrap_or(0.0) * (range[1] - range[0]), + range[2] + vals.get(2).copied().unwrap_or(0.0) * (range[3] - range[2]), + ]; + rgb(base, &scaled) + } + _ => rgb(base, &vals), + } + } + Space::Tint { inputs, alt, exp } => match exp { + Some((c0, c1, n)) => { + let t = clamp01(v(0)); + let k = if *n == 1.0 { t } else { t.powf(*n) }; + let out: Vec = c0.iter().zip(c1.iter()).map(|(a, b)| a + k * (b - a)).collect(); + rgb(alt, &out) + } + // No usable transform: the darkness of the strongest tint, in the alternate space's ink. + None => { + let t = (0..*inputs).map(v).fold(0.0, f64::max); + let g = 1.0 - clamp01(t); + [g, g, g] + } + }, + Space::Pattern => [0.0, 0.0, 0.0], + } +} diff --git a/crates/edit/src/convert_colours.rs b/crates/edit/src/convert_colours.rs new file mode 100644 index 0000000000..7134e537d8 --- /dev/null +++ b/crates/edit/src/convert_colours.rs @@ -0,0 +1,492 @@ +//! Convert colours (Acrobat's Print production ▸ Convert Colors): every colour +//! operator in the page content and the artwork it draws is rewritten in a target space - +//! grayscale, RGB or CMYK - by the standard device formulas (no ICC profile is applied), black +//! preserved on request; pictures are re-encoded through a helper the caller supplies. +//! +//! This is not a prepress separation. No ICC profile, rendering intent or output intent is +//! used, so the numbers are not calibrated for any press, and the catalog's `/OutputIntents` +//! stay as they were. A PDF/A or PDF/X file may no longer conform afterwards, and still says it +//! does: nothing here changes what it declares (the `doc_convert_colours` tool lists what it +//! declared; `standards_remove_id` takes that off). +//! +//! Spot colours are converted, not kept: Separation and DeviceN through a Type 2 tint transform +//! into their alternate space, any other tint transform as a gray of the strongest tint's +//! darkness (the hue is lost). `/All` converts like any other; `/None` paints nothing and stays. +//! Patterns, shadings, inline images and transparency group spaces stay as they are. +//! +//! Soft masks are not touched: a picture keeps the one it has. The helper leaves a picture alone +//! when it cannot re-encode it (indexed, CMYK, not 8-bit, an explicit mask, a matted soft mask, +//! JPEG 2000 ...); those that are not in the target already are counted in +//! [`Converted::images_left`]. + +use pdfcraft_content::{Op, parse, serialize_ops}; +use pdfcraft_cos::{Dict, Document, ObjRef, Object, Stream}; +use std::collections::{HashMap, HashSet}; + +use crate::colour::{Space, rgb, space_of}; +use crate::page_streams::PageStreams; +use crate::{EditError, check, page_list}; + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum ColourTarget { + Gray, + Rgb, + Cmyk, +} + +#[derive(Clone, Debug, PartialEq)] +pub struct ConvertColours { + pub target: ColourTarget, + /// Pure black stays 100% K (or gray 0) rather than a mix. + pub preserve_black: bool, + /// Re-encode pictures too (through the caller's helper). + pub images: bool, + /// The appearance streams of the pages' comments and form fields too. + pub annotations: bool, +} + +impl Default for ConvertColours { + fn default() -> Self { + Self { target: ColourTarget::Gray, preserve_black: true, images: true, annotations: false } + } +} + +#[derive(Clone, Debug, Default, PartialEq, Eq)] +pub struct Converted { + /// Colour operators rewritten. + pub colours: usize, + /// Pictures re-encoded. + pub images: usize, + /// Pictures not in the target space that could not be re-encoded and are left as they were. + pub images_left: usize, + /// Pages on which something changed. + pub pages: usize, +} + +/// Re-encode one image stream in the target space, if it can be: the caller's (the optimizer +/// knows how to decode and encode pictures; the edit crate does not). +pub type ImageRecolour<'a> = dyn FnMut(&Document, &Stream, ColourTarget) -> Option + 'a; + +fn luma(c: [f64; 3]) -> f64 { + (0.299 * c[0] + 0.587 * c[1] + 0.114 * c[2]).clamp(0.0, 1.0) +} + +/// sRGB → the target's components. +pub fn components(target: ColourTarget, c: [f64; 3], preserve_black: bool) -> Vec { + let black = c.iter().all(|v| *v < 0.02); + match target { + ColourTarget::Gray => vec![if black && preserve_black { 0.0 } else { luma(c) }], + ColourTarget::Rgb => c.to_vec(), + ColourTarget::Cmyk => { + if black && preserve_black { + return vec![0.0, 0.0, 0.0, 1.0]; + } + let k = 1.0 - c.iter().cloned().fold(0.0, f64::max); + if k >= 0.999 { + return vec![0.0, 0.0, 0.0, 1.0]; + } + let f = |v: f64| ((1.0 - v - k) / (1.0 - k)).clamp(0.0, 1.0); + vec![f(c[0]), f(c[1]), f(c[2]), k] + } + } +} + +fn op_for(target: ColourTarget, stroke: bool) -> &'static str { + match (target, stroke) { + (ColourTarget::Gray, false) => "g", + (ColourTarget::Gray, true) => "G", + (ColourTarget::Rgb, false) => "rg", + (ColourTarget::Rgb, true) => "RG", + (ColourTarget::Cmyk, false) => "k", + (ColourTarget::Cmyk, true) => "K", + } +} + +/// Whether a `cs` operand names DeviceCMYK itself (directly or through a `/ColorSpace` +/// resource), whose initial colour is black, unlike an ICC CMYK space's all-zero white. +fn device_cmyk(doc: &Document, operand: &Object, cs_res: &Dict) -> bool { + let is_device = |o: &Object| match &*doc.resolve(o) { + Object::Name(n) => matches!(n.as_slice(), b"DeviceCMYK" | b"CMYK"), + Object::Array(a) => a.len() == 1 && a.first().and_then(Object::as_name).is_some_and(|n| matches!(n, b"DeviceCMYK" | b"CMYK")), + _ => false, + }; + match operand.as_name() { + Some(n) => is_device(operand) || cs_res.get(n).is_some_and(is_device), + None => is_device(operand), + } +} + +fn r3(v: f64) -> Object { + Object::Real((v * 1000.0).round() / 1000.0) +} + +/// The current fill and stroke spaces (None = a device space set by g/rg/k, which the +/// converted operator sets again), with the q/Q stack: what one content stream leaves for the +/// next on the same page. +#[derive(Clone, Debug, Default)] +struct State { + fill: Option, + stroke: Option, + stack: Vec<(Option, Option)>, +} + +/// Rewrite one content stream's colour operators, starting from `state` and leaving in it the +/// state at the stream's end. `cs_res` is its `/ColorSpace` resource dictionary. Returns the +/// new bytes when anything changed, and how many operators. +fn rewrite(data: &[u8], state: &mut State, doc: &Document, cs_res: &Dict, settings: &ConvertColours) -> (Option>, usize) { + let mut ops: Vec = parse(data).ops; + let mut count = 0usize; + let mut out: Vec = Vec::with_capacity(ops.len()); + let State { fill, stroke, stack } = state; + let numbers = |op: &Op| -> Vec { op.operands.iter().filter_map(Object::as_f64).collect() }; + let convert = |c: [f64; 3], is_stroke: bool| -> Op { + Op::new(op_for(settings.target, is_stroke), components(settings.target, c, settings.preserve_black).into_iter().map(r3).collect()) + }; + for op in ops.drain(..) { + match op.op.as_slice() { + b"q" => { + stack.push((fill.clone(), stroke.clone())); + if stack.len() > 512 { + stack.remove(0); + } + out.push(op); + } + b"Q" => { + if let Some((f, s)) = stack.pop() { + *fill = f; + *stroke = s; + } + out.push(op); + } + b"cs" | b"CS" => { + let is_stroke = op.op.as_slice() == b"CS"; + let space = op.operands.first().map(|o| space_of(doc, o, cs_res, 0)).unwrap_or(Space::Gray); + let pattern = space == Space::Pattern; + if is_stroke { + *stroke = Some(space); + } else { + *fill = Some(space); + } + if pattern { + // Patterns stay what they are (their own content is not converted here). + out.push(op); + } else { + // The space's initial colour, now in the target: black (a tint at full, + // DeviceCMYK's 0 0 0 1, ISO 32000-1 8.6.5.5). + let space = if is_stroke { stroke.clone() } else { fill.clone() }.unwrap_or(Space::Gray); + let initial = match space { + Space::Tint { .. } => rgb(&space, &[1.0; 8]), + Space::Cmyk if op.operands.first().is_some_and(|o| device_cmyk(doc, o, cs_res)) => rgb(&space, &[0.0, 0.0, 0.0, 1.0]), + _ => rgb(&space, &[]), + }; + out.push(convert(initial, is_stroke)); + count += 1; + } + } + b"g" | b"rg" | b"k" | b"G" | b"RG" | b"K" => { + let is_stroke = op.op[0].is_ascii_uppercase(); + let space = match op.op.to_ascii_lowercase().as_slice() { + b"g" => Space::Gray, + b"rg" => Space::Rgb, + _ => Space::Cmyk, + }; + let already = matches!( + (settings.target, &space), + (ColourTarget::Gray, Space::Gray) | (ColourTarget::Rgb, Space::Rgb) | (ColourTarget::Cmyk, Space::Cmyk) + ); + if is_stroke { + *stroke = None; + } else { + *fill = None; + } + if already { + out.push(op); + } else { + let c = rgb(&space, &numbers(&op)); + out.push(convert(c, is_stroke)); + count += 1; + } + } + b"sc" | b"scn" | b"SC" | b"SCN" => { + let is_stroke = op.op[0].is_ascii_uppercase(); + let space = if is_stroke { stroke.clone() } else { fill.clone() }; + match space { + Some(Space::Pattern) | None if op.operands.iter().any(|o| matches!(o, Object::Name(_))) => out.push(op), + None => { + // sc after g/rg/k: the device space those set; read the numbers by count. + let n = numbers(&op); + let space = match n.len() { + 1 => Space::Gray, + 4 => Space::Cmyk, + _ => Space::Rgb, + }; + out.push(convert(rgb(&space, &n), is_stroke)); + count += 1; + } + Some(Space::Pattern) => out.push(op), + Some(space) => { + out.push(convert(rgb(&space, &numbers(&op)), is_stroke)); + count += 1; + } + } + } + _ => out.push(op), + } + } + ((count > 0).then(|| serialize_ops(&out)), count) +} + +/// Whether `space`'s colours are `target`'s (an indexed space's are its base's). +fn in_target(space: &Space, target: ColourTarget) -> bool { + match (space, target) { + (Space::Gray, ColourTarget::Gray) | (Space::Rgb, ColourTarget::Rgb) | (Space::Cmyk, ColourTarget::Cmyk) => true, + (Space::Indexed { base, .. }, _) => in_target(base, target), + _ => false, + } +} + +/// Whether the picture `s` is outside `target`: a stencil mask has no colours of its own (it +/// paints with the fill colour, which is converted). +fn needs_converting(doc: &Document, s: &Stream, target: ColourTarget) -> bool { + let stencil = s.dict.get(b"ImageMask").is_some_and(|m| matches!(&*doc.resolve(m), Object::Bool(true))); + !stencil && !s.dict.get(b"ColorSpace").is_some_and(|cs| in_target(&space_of(doc, cs, &Dict::default(), 0), target)) +} + +fn stream_of(doc: &Document, r: ObjRef) -> Option { + match &*doc.get(r) { + Object::Stream(s) => Some(s.clone()), + _ => None, + } +} + +fn cs_resources(doc: &Document, resources: Option<&Dict>) -> Dict { + resources.and_then(|r| r.get(b"ColorSpace")).map(|c| doc.resolve(c)).and_then(|c| c.as_dict().cloned()).unwrap_or_default() +} + +/// What a run keeps across pages: each XObject already handled, and the object that now +/// stands for it (itself, or the converted copy made because a page not being converted draws +/// it too); and the XObjects those other pages draw. +struct Run { + done: HashMap, + shared: HashSet, +} + +/// The XObjects (forms and images, and those the forms draw), and appearance streams, that +/// the pages not in `pages` draw: converting one of those in place would change those pages. +fn drawn_elsewhere(doc: &Document, pages: &[usize]) -> HashSet { + let mut seen: HashSet = HashSet::new(); + let mut todo: Vec = Vec::new(); + let xobjects_of = |res: Option<&Object>, todo: &mut Vec| { + if let Some(xo) = res + .map(|r| doc.resolve(r)) + .and_then(|r| r.as_dict().and_then(|d| d.get(b"XObject")).map(|x| doc.resolve(x))) + .and_then(|x| x.as_dict().cloned()) + { + todo.extend(xo.iter().map(|(_, o)| o.clone())); + } + }; + for (i, page) in page_list(doc).into_iter().enumerate() { + if pages.contains(&i) { + continue; + } + xobjects_of(page.dict.get(b"Resources"), &mut todo); + let annots = page.dict.get(b"Annots").map(|a| doc.resolve(a)).and_then(|a| a.as_array().cloned()).unwrap_or_default(); + for a in annots { + let Some(ap) = doc.resolve(&a).as_dict().and_then(|d| d.get(b"AP").cloned()) else { continue }; + let Some(ap) = doc.resolve(&ap).as_dict().cloned() else { continue }; + for (_, v) in ap.iter() { + match &*doc.resolve(v) { + Object::Dict(states) => todo.extend(states.iter().map(|(_, s)| s.clone())), + _ => todo.push(v.clone()), + } + } + } + } + while let Some(o) = todo.pop() { + let Some(r) = o.as_ref() else { continue }; + if !seen.insert(r) { + continue; + } + let res = match &*doc.get(r) { + Object::Stream(s) => s.dict.get(b"Resources").cloned(), + _ => None, + }; + xobjects_of(res.as_ref(), &mut todo); + } + seen +} + +/// Store a converted XObject: in place, or, when another page draws `r` too, as a new object +/// the caller points to instead. Returns the object that now stands for `r`. +fn put(doc: &mut Document, run: &mut Run, r: ObjRef, obj: Object) -> ObjRef { + let to = if run.shared.contains(&r) { + doc.add(obj) + } else { + doc.set(r, obj); + r + }; + run.done.insert(r, to); + to +} + +/// Convert the XObjects of `resources`: forms rewritten (and their own XObjects after them), +/// images through `recolour`. Each object is converted once; one that a page not being +/// converted draws is converted as a copy, and the returned `/XObject` dictionary, when there +/// is one, points at the copies (the caller puts it in `resources`' place). When `may_copy` is +/// false the caller cannot do that, so such objects are left alone. +fn convert_xobjects( + doc: &mut Document, + resources: Option<&Dict>, + settings: &ConvertColours, + recolour: &mut ImageRecolour<'_>, + run: &mut Run, + result: &mut Converted, + may_copy: bool, +) -> Option { + let xo = resources.and_then(|r| r.get(b"XObject")).map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned())?; + let mut repointed = xo.clone(); + let mut changed = false; + for (name, o) in xo.iter() { + let Some(r) = o.as_ref() else { continue }; + if let Some(&to) = run.done.get(&r) { + if to != r { + repointed.set(name.clone(), Object::Ref(to)); + changed = true; + } + continue; + } + if !may_copy && run.shared.contains(&r) { + continue; + } + // Marked before converting, so a form that draws itself ends. + run.done.insert(r, r); + let Some(s) = stream_of(doc, r) else { continue }; + let stored = match s.dict.name(b"Subtype") { + Some(b"Form") => { + let res = s.dict.get(b"Resources").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()); + let cs = cs_resources(doc, res.as_ref()); + let (new, n) = match s.decoded() { + Ok(data) => rewrite(&data, &mut State::default(), doc, &cs, settings), + Err(_) => (None, 0), + }; + let inner = convert_xobjects(doc, res.as_ref(), settings, recolour, run, result, true); + if new.is_none() && inner.is_none() { + None + } else { + let mut form = s.clone(); + if let Some(xo) = inner { + let mut res = res.unwrap_or_default(); + res.set(b"XObject".to_vec(), Object::Dict(xo)); + form.dict.set(b"Resources".to_vec(), Object::Dict(res)); + } + if let Some(bytes) = new { + let mut dict = form.dict; + for k in [&b"Length"[..], b"Filter", b"DecodeParms"] { + dict.remove(k); + } + form = Stream::flate(dict, &bytes); + result.colours += n; + } + Some(put(doc, run, r, Object::Stream(form))) + } + } + Some(b"Image") if settings.images => match recolour(doc, &s, settings.target) { + Some(new) => { + result.images += 1; + Some(put(doc, run, r, Object::Stream(new))) + } + None => { + result.images_left += usize::from(needs_converting(doc, &s, settings.target)); + None + } + }, + _ => None, + }; + if let Some(to) = stored + && to != r + { + repointed.set(name.clone(), Object::Ref(to)); + changed = true; + } + } + changed.then_some(repointed) +} + +/// Convert the colours on `pages` (0-based). An object that pages not in `pages` also draw is +/// converted as a copy for these pages, so the others keep their colours. +pub fn convert_colours( + doc: &mut Document, + pages: &[usize], + settings: &ConvertColours, + recolour: &mut ImageRecolour<'_>, +) -> Result { + let all = page_list(doc); + check(pages, all.len())?; + let mut result = Converted::default(); + let mut run = Run { done: HashMap::new(), shared: drawn_elsewhere(doc, pages) }; + for &i in pages { + let Some(page) = page_list(doc).get(i).cloned() else { continue }; + let resources = page.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()); + let cs = cs_resources(doc, resources.as_ref()); + let before = (result.colours, result.images); + // Each content stream on its own (state carries across them); a page with a stream + // that cannot be read keeps its content as it is. + if let Some(streams) = PageStreams::read(doc, &page.dict) { + let mut state = State::default(); + let mut new: Vec>> = Vec::with_capacity(streams.data.len()); + for data in &streams.data { + let (bytes, n) = rewrite(data, &mut state, doc, &cs, settings); + result.colours += n; + new.push(bytes); + } + streams.write(doc, page.obj, new)?; + } + if let Some(xo) = convert_xobjects(doc, resources.as_ref(), settings, recolour, &mut run, &mut result, true) { + // This page's own resources point at the copies; a resource dictionary other pages + // share is not changed. + let mut res = resources.clone().unwrap_or_default(); + res.set(b"XObject".to_vec(), Object::Dict(xo)); + doc.update_dict(page.obj, |d| d.set(b"Resources".to_vec(), Object::Dict(res)))?; + } + if settings.annotations { + let annots: Vec = page + .dict + .get(b"Annots") + .map(|a| doc.resolve(a)) + .and_then(|a| a.as_array().cloned()) + .unwrap_or_default() + .iter() + .filter_map(|a| a.as_ref()) + .collect(); + for a in annots { + let Some(ap) = + doc.get(a).as_dict().and_then(|d| d.get(b"AP").cloned()).map(|ap| doc.resolve(&ap)).and_then(|ap| ap.as_dict().cloned()) + else { + continue; + }; + // Every appearance stream (and the states of a /N dictionary) as a form. + let mut streams: Vec = Vec::new(); + for (_, v) in ap.iter() { + match &*doc.resolve(v) { + Object::Stream(_) => streams.extend(v.as_ref()), + Object::Dict(states) => streams.extend(states.iter().filter_map(|(_, s)| s.as_ref())), + _ => {} + } + } + let mut fake = Dict::new(); + let mut xo = Dict::new(); + for (k, r) in streams.iter().enumerate() { + xo.set(format!("A{k}").into_bytes(), Object::Ref(*r)); + } + fake.set(b"XObject".to_vec(), Object::Dict(xo)); + // An appearance stream other pages draw is left alone (there is no resource + // dictionary here to point at a copy). + convert_xobjects(doc, Some(&fake), settings, recolour, &mut run, &mut result, false); + } + } + if (result.colours, result.images) != before { + result.pages += 1; + } + } + Ok(result) +} diff --git a/crates/edit/src/flatten.rs b/crates/edit/src/flatten.rs index ef86d6c67a..03ed272f3c 100644 --- a/crates/edit/src/flatten.rs +++ b/crates/edit/src/flatten.rs @@ -4,7 +4,8 @@ //! Each visible annotation's normal appearance (for check boxes and radio buttons, the one its //! `/AS` selects) is drawn into the page with the matrix of ISO 32000-2 §12.5.5 (Algorithm 8.1: //! the appearance's bounding box, transformed by its `/Matrix`, mapped onto `/Rect`). Hidden -//! annotations are left alone; links are kept (they have no ink and stay clickable). Flattened +//! annotations and replies are left alone (a reply is not shown on the page: [`is_reply`]); +//! links are kept (they have no ink and stay clickable). Flattened //! comments take their pop-ups and replies with them; flattened widgets are removed from the //! form's field tree. An annotation whose appearance cannot be drawn is not flattened: it stays //! as it is, so no comment or field value is lost with nothing in its place. @@ -19,6 +20,21 @@ const HIDDEN: i64 = 2; const NO_ROTATE: i64 = 16; const NO_VIEW: i64 = 32; +/// A reply of type R: a markup annotation with an `/IRT` to the annotation it answers and an +/// `/RT` other than `/Group` (R is the default, so a missing, unknown or malformed `/RT` reads as +/// R, as the comment list reads it). It is shown in its comment's thread, never on the page +/// ("Interactive PDF processors shall not display replies to an annotation individually", ISO +/// 32000-2 Table 172), so flattening neither draws it nor builds it an appearance; it goes with +/// its comment. A `/RT /Group` member is part of its group's look and is flattened as any other +/// annotation. `/IRT` is a markup annotation's entry: a widget, link or pop-up with one is no +/// reply, and is flattened (or kept) as it would be without. Keep in step with hayro-interpret's +/// PdfCraft patch (81). +fn is_reply(doc: &Document, d: &Dict) -> bool { + !matches!(d.name(b"Subtype"), Some(b"Widget" | b"Link" | b"Popup")) + && d.get(b"IRT").is_some_and(|o| doc.resolve(o).as_dict().is_some()) + && d.get(b"RT").map(|o| doc.resolve(o)).as_deref().and_then(Object::as_name) != Some(b"Group") +} + fn nums(doc: &Document, o: Option<&Object>) -> Option> { let o = doc.resolve(o?); o.as_array()?.iter().map(|x| doc.resolve(x).as_f64()).collect() @@ -136,6 +152,160 @@ fn xobject_name(existing: &Dict, pending: &Dict, n: &mut u32) -> String { } } +/// Copy usable property resources, including inherited resources supplied by the page model. +/// Invalid data must stay intact rather than being replaced while transferring membership. +fn properties(doc: &Document, page: &Dict) -> Option { + let Some(resources) = page.get(b"Resources") else { return Some(Dict::new()) }; + let resources = doc.resolve(resources); + let Object::Dict(resources) = &*resources else { return None }; + let Some(properties) = resources.get(b"Properties") else { return Some(Dict::new()) }; + match &*doc.resolve(properties) { + Object::Dict(properties) => Some(properties.clone()), + _ => None, + } +} + +/// Validate an ISO 32000-1 §8.11.2.2 `/VE` visibility expression within recursion depth 16 +/// and a shared node-visit `budget`. +fn valid_ve(doc: &Document, expr: &Object, depth: usize, budget: &mut usize) -> bool { + if depth > 16 || *budget == 0 { + return false; + } + *budget -= 1; + let resolved = doc.resolve(expr); + if let Some(arr) = resolved.as_array() { + let [op, rest @ ..] = arr.as_slice() else { return false }; + let op_obj = doc.resolve(op); + let Some(name) = op_obj.as_name() else { return false }; + match (name, rest) { + (b"Not", [operand]) => valid_ve(doc, operand, depth + 1, budget), + (b"And" | b"Or", [_, ..]) => rest.iter().all(|o| valid_ve(doc, o, depth + 1, budget)), + _ => false, + } + } else if depth > 0 { + expr.as_ref().is_some_and(|r| matches!(&*doc.get(r), Object::Dict(d) if d.name(b"Type") == Some(b"OCG"))) + } else { + false + } +} + +/// Validate an `/OCMD` dictionary's `/P` policy and `/VE` visibility expression, if present. +fn valid_ocmd(doc: &Document, d: &Dict) -> bool { + if let Some(p) = d.get(b"P") + && !matches!(doc.resolve(p).as_name(), Some(b"AllOn" | b"AnyOn" | b"AnyOff" | b"AllOff")) + { + return false; + } + if let Some(ve) = d.get(b"VE") + && !valid_ve(doc, ve, 0, &mut 4096) + { + return false; + } + true +} + +/// Preserve the stored value, not a reconstructed membership dictionary. A direct inline +/// dictionary lacks an indirect reference (ObjRef) to identify it in page resources, and an +/// OCMD with a malformed or overly deep `/VE` expression stays on the annotation. +fn transferable(doc: &Document, annotation: &Dict, usable_properties: bool) -> bool { + let Some(membership) = annotation.get(b"OC") else { return true }; + if !usable_properties { + return false; + } + match &*doc.resolve(membership) { + Object::Dict(d) => match d.name(b"Type") { + Some(b"OCG") => membership.as_ref().is_some(), + Some(b"OCMD") => valid_ocmd(doc, d), + _ => false, + }, + _ => false, + } +} + +#[derive(Default)] +struct Retained { + refs: HashSet, + inline: HashSet, +} + +impl Retained { + fn contains(&self, index: usize, entry: &Object) -> bool { + self.inline.contains(&index) || entry.as_ref().is_some_and(|r| self.refs.contains(&r)) + } + + fn insert(&mut self, index: usize, entry: &Object) { + if let Some(r) = entry.as_ref() { + self.refs.insert(r); + } else { + self.inline.insert(index); + } + } +} + +/// Retained annotations keep their replies and pop-ups, including transitive reply chains. +/// +/// Retention is strictly downward (parent -> child/reply/popup). A non-transferable child does not +/// keep its transferable parent, matching the engine invariant for undrawn children (#1794). +fn retained_annotations(doc: &Document, list: &[Object], usable_properties: bool) -> Retained { + let mut children: HashMap> = HashMap::new(); + let mut inline_children: HashMap> = HashMap::new(); + let mut pending = Vec::new(); + let mut retained = Retained::default(); + for (k, entry) in list.iter().enumerate() { + let object = doc.resolve(entry); + let Some(annotation) = object.as_dict() else { continue }; + let not_transferable = !transferable(doc, annotation, usable_properties); + if let Some(reference) = entry.as_ref() { + if not_transferable { + pending.push(reference); + } + if let Some(parent) = annotation.reference(b"IRT") { + children.entry(parent).or_default().push(reference); + } + if annotation.name(b"Subtype") == Some(b"Popup") + && let Some(parent) = annotation.reference(b"Parent") + { + children.entry(parent).or_default().push(reference); + } + } else { + if not_transferable { + retained.inline.insert(k); + } + if let Some(parent) = annotation.reference(b"IRT") { + inline_children.entry(parent).or_default().push(k); + } + if annotation.name(b"Subtype") == Some(b"Popup") + && let Some(parent) = annotation.reference(b"Parent") + { + inline_children.entry(parent).or_default().push(k); + } + } + } + while let Some(current) = pending.pop() { + if !retained.refs.insert(current) { + continue; + } + if let Some(kids) = children.get(¤t) { + pending.extend(kids.iter().copied()); + } + if let Some(inline_kids) = inline_children.get(¤t) { + retained.inline.extend(inline_kids.iter().copied()); + } + } + retained +} + +/// A Properties resource name that is not already on the page or in `pending`. +fn property_name(existing: &Dict, pending: &Dict, n: &mut u64) -> Option { + loop { + let name = format!("PCFlOC{n}"); + *n = n.checked_add(1)?; + if !existing.contains(name.as_bytes()) && !pending.contains(name.as_bytes()) { + return Some(name); + } + } +} + /// Flatten comments and/or form fields on `pages`. Returns how many annotations were merged /// into page content. pub fn flatten(doc: &mut Document, pages: &[usize], comments: bool, fields: bool) -> Result { @@ -157,10 +327,16 @@ pub fn flatten_fill_sign(doc: &mut Document, pages: &[usize]) -> Result Option<(usize, bool, char)> { for &i in pages { let list = all.get(i)?.dict.get(b"Annots").map(|a| doc.resolve(a)).and_then(|a| a.as_array().cloned()).unwrap_or_default(); - for entry in &list { + let usable_properties = properties(doc, &all[i].dict).is_some(); + let retained = retained_annotations(doc, &list, usable_properties); + for (k, entry) in list.iter().enumerate() { + if retained.contains(k, entry) { + continue; + } let Some(d) = doc.resolve(entry).as_dict().cloned() else { continue }; let flags = d.get(b"F").and_then(|f| doc.resolve(f).as_int()).unwrap_or(0); - if !selected(doc, &d, which) || flags & (HIDDEN | NO_VIEW) != 0 { + // A reply is never drawn into the page (`is_reply`), so its text can't bake in a `?`. + if !selected(doc, &d, which) || flags & (HIDDEN | NO_VIEW) != 0 || !transferable(doc, &d, usable_properties) || is_reply(doc, &d) { continue; } let widget = d.name(b"Subtype") == Some(b"Widget"); @@ -174,6 +350,73 @@ fn undrawn_on(doc: &Document, all: &[pdfcraft_model::Page], pages: &[usize], whi None } +/// Ensure comments without stored appearances have appearances generated before flattening. +fn prepare_appearances(doc: &mut Document, list: &[Object], which: Which, retained: &HashSet, usable_properties: bool) { + for entry in list { + let Some(r) = entry.as_ref() else { continue }; + if retained.contains(&r) { + continue; + } + let d = doc.resolve(entry).as_dict().cloned().unwrap_or_default(); + let subtype = d.name(b"Subtype").unwrap_or_default(); + if subtype == b"Widget" || !selected(doc, &d, which) || !transferable(doc, &d, usable_properties) { + continue; + } + let flags = d.get(b"F").and_then(|f| doc.resolve(f).as_int()).unwrap_or(0); + // A reply is not shown on the page (`is_reply`), so it gets no appearance either. + if flags & (HIDDEN | NO_VIEW) != 0 || is_reply(doc, &d) || appearance(doc, &d).is_some() { + continue; + } + // A subtype this build cannot draw fails here and is kept below, so the comment is not + // deleted with nothing drawn in its place. (Text the standard fonts can't draw was + // refused before anything changed: `undrawn_on`.) + let _ = pdfcraft_annot::set_appearance(doc, r); + } +} + +/// Adds to `gone` and `gone_inline` the pop-ups and replies of what is in them, at any depth of +/// the thread and in any order of `/Annots`, passing by the entries `stays` keeps. What points at +/// each annotation (a pop-up by `/Parent`, a reply by `/IRT`) is gathered once, then followed +/// from what went. `/Annots` should hold references (ISO 32000-2 Table 31), but a pop-up or +/// reply written inline still goes with what it points at; nothing can point at it in turn. +fn take_threads( + doc: &Document, + list: &[Object], + gone: &mut HashSet, + gone_inline: &mut HashSet, + stays: impl Fn(usize, &Object) -> bool, +) { + let mut pointing: HashMap> = HashMap::new(); + let mut pointing_inline: Vec<(ObjRef, usize)> = Vec::new(); + for (k, entry) in list.iter().enumerate() { + if stays(k, entry) { + continue; + } + let r = entry.as_ref(); + let obj = doc.resolve(entry); + let Some(d) = obj.as_dict() else { continue }; + let popup_of = d.reference(b"Parent").filter(|_| d.name(b"Subtype") == Some(b"Popup")); + // Only a reply goes with what it answers: a `/RT /Group` member was drawn (and went) or + // stays, hidden or undrawable, as any annotation. + let reply_to = d.reference(b"IRT").filter(|_| is_reply(doc, d)); + for p in popup_of.into_iter().chain(reply_to) { + match r { + Some(r) => pointing.entry(p).or_default().push(r), + None => pointing_inline.push((p, k)), + } + } + } + let mut follow: Vec = gone.iter().copied().collect(); + while let Some(p) = follow.pop() { + for &r in pointing.get(&p).into_iter().flatten() { + if gone.insert(r) { + follow.push(r); + } + } + } + gone_inline.extend(pointing_inline.into_iter().filter(|(p, _)| gone.contains(p)).map(|(_, k)| k)); +} + /// Runs [`flatten_in`] on a copy of `doc` (unchanged objects are shared) and keeps it only on /// success, so a page refused part way leaves the pages before it as they were. fn flatten_which(doc: &mut Document, pages: &[usize], which: Which) -> Result { @@ -183,6 +426,19 @@ fn flatten_which(doc: &mut Document, pages: &[usize], which: Which) -> Result pdfcraft_model::Page { + let mut dict = page.dict.clone(); + if let Some(own) = doc.get(page.obj).as_dict() { + for (k, v) in own.iter() { + dict.set(k.clone(), v.clone()); + } + } + pdfcraft_model::Page { obj: page.obj, dict } +} + fn flatten_in(doc: &mut Document, pages: &[usize], which: Which) -> Result { let all = page_list(doc); check(pages, all.len())?; @@ -205,6 +461,10 @@ fn flatten_in(doc: &mut Document, pages: &[usize], which: Which) -> Result Result Result Result = list .iter() .enumerate() @@ -321,11 +575,18 @@ fn flatten_in(doc: &mut Document, pages: &[usize], which: Which) -> Result doc.set(r, Object::Array(kept)), None => doc.update_dict(page.obj, |d| { diff --git a/crates/edit/src/hairlines.rs b/crates/edit/src/hairlines.rs new file mode 100644 index 0000000000..f70fd50d76 --- /dev/null +++ b/crates/edit/src/hairlines.rs @@ -0,0 +1,638 @@ +//! Fix hairlines (Acrobat's Print production ▸ Fix Hairlines; Fix hairlines): strokes thinner +//! than a printer holds - zero-width lines, and lines whose width on the page, after every +//! transformation, is below a threshold - are given a chosen width. Page content and the Form +//! XObjects it draws are rewritten; Type 3 glyphs and tiling patterns on request. +//! +//! A form is rewritten for the scale and line width it is drawn under. One drawn under several +//! (or drawn by a page that is not being fixed) is rewritten as a copy for each use that needs +//! one, and the resources of what draws it point at the copy, so no other use changes. +//! +//! Type 3 glyphs and tiling patterns are each a content stream of their own, reached from the +//! resources of the page and of the forms it draws. They are rewritten in place, so a page +//! outside the run that uses one changes too; [`Fixed::elsewhere`] counts those. + +use pdfcraft_content::{Op, parse, serialize_ops}; +use pdfcraft_cos::{Dict, Document, Name, ObjRef, Object, Stream}; +use std::collections::{HashMap, HashSet}; + +use crate::page_streams::PageStreams; +use crate::{EditError, check, n, page_list}; + +/// The stream `r` holds, if it is one. +fn stream_of(doc: &Document, r: ObjRef) -> Option { + match &*doc.get(r) { + Object::Stream(s) => Some(s.clone()), + _ => None, + } +} + +/// What counts as a hairline and what it becomes, in points on the page. +#[derive(Clone, Debug, PartialEq)] +pub struct Hairlines { + /// Lines narrower than this (points, after the page's transformations) are replaced; + /// zero-width lines always are. Acrobat's default is 0.003 in. + pub narrower_than: f64, + /// The width they are given (points on the page). Acrobat's default is 0.01 in. + pub width: f64, + /// Also the strokes inside Type 3 font glyphs. + pub type3: bool, + /// Also the strokes inside tiling patterns. + pub patterns: bool, +} + +impl Default for Hairlines { + fn default() -> Self { + Self { narrower_than: 0.003 * 72.0, width: 0.01 * 72.0, type3: false, patterns: false } + } +} + +impl Hairlines { + /// The widest either width may be, in points: past this it is no hairline. + pub const MAX_POINTS: f64 = 10.0; + + /// Whether both widths are numbers a run accepts: a width above 0, a threshold of 0 or + /// more, neither past [`Self::MAX_POINTS`]. + pub fn in_range(&self) -> bool { + (0.0..=Self::MAX_POINTS).contains(&self.narrower_than) && self.width > 0.0 && self.width <= Self::MAX_POINTS + } +} + +/// What a run changed. +#[derive(Clone, Debug, Default, PartialEq, Eq)] +pub struct Fixed { + /// Strokes given a new width. + pub lines: usize, + /// Pages on which something changed. + pub pages: usize, + /// Type 3 glyphs and tiling patterns changed that a page outside the run, or an + /// annotation's appearance, also uses: what those draw changed too. + pub elsewhere: usize, +} + +/// The uniform scale a matrix applies to lengths (the geometric mean of its two axes). +fn scale_of(m: [f64; 6]) -> f64 { + (m[0] * m[3] - m[1] * m[2]).abs().sqrt() +} + +fn mul(a: [f64; 6], b: [f64; 6]) -> [f64; 6] { + // a then b. + [ + a[0] * b[0] + a[1] * b[2], + a[0] * b[1] + a[1] * b[3], + a[2] * b[0] + a[3] * b[2], + a[2] * b[1] + a[3] * b[3], + a[4] * b[0] + a[5] * b[2] + b[4], + a[4] * b[1] + a[5] * b[3] + b[5], + ] +} + +/// The graphics state a stream leaves for the next one on the same page: CTM and line width, +/// with the q/Q stack, and whether a path is still being built. +#[derive(Clone, Debug)] +struct State { + ctm: [f64; 6], + width: f64, + stack: Vec<([f64; 6], f64)>, + in_path: bool, +} + +impl State { + /// A stream drawn under `base` with line width `width` in effect. + fn new(base: [f64; 6], width: f64) -> Self { + Self { ctm: base, width, stack: Vec::new(), in_path: false } + } +} + +/// All a form's rewrite depends on: the scale it is drawn under and the line width in effect +/// at its `Do` (as bits, so it can be a map key). +type Key = (u64, u64); + +fn key_of(ctm: [f64; 6], width: f64) -> Key { + (scale_of(ctm).to_bits(), width.to_bits()) +} + +/// A Form XObject a stream draws: the resource name it is drawn by, the object, and the CTM +/// and line width at its `Do`. +struct Drawn { + name: Name, + r: ObjRef, + ctm: [f64; 6], + width: f64, +} + +/// The names a stream's `Do`s are given instead, by the name they had and the key they are +/// drawn under. +type Renames = HashMap<(Name, Key), Name>; + +/// How deep forms drawing forms are followed, and how many form rewrites one run makes; past +/// either, a form is drawn as it is. +const MAX_DEPTH: usize = 32; +const MAX_FORMS: usize = 4096; + +/// The width `/name gs` sets, when its ExtGState has an `/LW`. +fn line_width_of(doc: &Document, gstates: Option<&Dict>, name: &[u8]) -> Option { + let gs = doc.resolve(gstates?.get(name)?); + let lw = gs.as_dict()?.get(b"LW")?; + doc.resolve(lw).as_f64().filter(|w| w.is_finite()) +} + +/// Rewrite one content stream drawn under `state` (the CTM when it starts, scale only +/// matters), leaving in it the state at the stream's end. A thin stroke gets a `w` of its own +/// before its path begins (graphics-state operators are not allowed inside a path) and the +/// old width back after it is painted, so nothing else drawn changes width. +/// `forms` collects the Form XObjects it draws, with the CTM and width each is drawn under, +/// for the caller to rewrite in turn; a `Do` listed in `renames` draws the name given there. +/// Returns the new bytes when anything changed, and the count. +fn rewrite( + data: &[u8], + state: &mut State, + h: &Hairlines, + resources: Option<&Dict>, + doc: &Document, + forms: &mut Vec, + renames: &Renames, +) -> (Option>, usize) { + let parsed = parse(data); + let mut ops: Vec = parsed.ops; + let mut fixed = 0usize; + let mut changed = false; + // Where in `out` the path being built began (None when it began in an earlier stream). + let mut path_start: Option = None; + let mut out: Vec = Vec::with_capacity(ops.len() + 8); + let dict_of = |key: &[u8]| resources.and_then(|r| r.get(key)).map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()); + let xobjects = dict_of(&b"XObject"[..]); + let gstates = dict_of(&b"ExtGState"[..]); + for mut op in ops.drain(..) { + match op.op.as_slice() { + b"q" => { + state.stack.push((state.ctm, state.width)); + if state.stack.len() > 512 { + state.stack.remove(0); + } + } + b"Q" => { + if let Some((c, w)) = state.stack.pop() { + state.ctm = c; + state.width = w; + } + } + b"cm" => { + if let Some(m) = op.nums::<6>() { + state.ctm = mul(m, state.ctm); + } + } + b"w" => { + if let Some([w]) = op.nums::<1>() { + state.width = w; + } + } + b"gs" => { + // An ExtGState's /LW sets the width as `w` does (so the width put back after + // a fixed stroke is the right one). + if let Some(Object::Name(name)) = op.operands.first() + && let Some(w) = line_width_of(doc, gstates.as_ref(), name) + { + state.width = w; + } + } + b"m" | b"re" => { + if !state.in_path { + state.in_path = true; + path_start = Some(out.len()); + } + } + b"Do" => { + if let Some(Object::Name(name)) = op.operands.first() + && let Some(x) = xobjects.as_ref().and_then(|x| x.get(name)) + && let Some(r) = x.as_ref() + && matches!(&*doc.get(r), Object::Stream(s) if s.dict.name(b"Subtype") == Some(b"Form")) + { + let name = name.clone(); + if !renames.is_empty() + && let Some(to) = renames.get(&(name.clone(), key_of(state.ctm, state.width))) + && let Some(first) = op.operands.first_mut() + { + *first = Object::Name(to.clone()); + changed = true; + } + forms.push(Drawn { name, r, ctm: state.ctm, width: state.width }); + } + } + b"S" | b"s" | b"B" | b"B*" | b"b" | b"b*" => { + let scale = scale_of(state.ctm); + let on_page = state.width * scale; + let thin = state.width <= 0.0 || (scale > 0.0 && on_page < h.narrower_than); + // A path begun in an earlier stream cannot get its `w` before it here; it is + // left as it is rather than given an operator inside the path. + let at = match path_start { + Some(i) => Some(i), + None if !state.in_path => Some(out.len()), + None => None, + }; + state.in_path = false; + path_start = None; + if thin + && scale > 0.0 + && let Some(at) = at + { + let new_w = h.width / scale; + out.insert(at, Op::new("w", vec![Object::Real(new_w)])); + out.push(op); + out.push(Op::new("w", vec![Object::Real(state.width)])); + fixed += 1; + changed = true; + continue; + } + } + b"f" | b"F" | b"f*" | b"n" => { + state.in_path = false; + path_start = None; + } + _ => {} + } + out.push(op); + } + (changed.then(|| serialize_ops(&out)), fixed) +} + +/// What a run keeps across pages. +struct Run { + /// Each form already rewritten under a key, and the object drawn for it there. + done: HashMap<(ObjRef, Key), ObjRef>, + /// The forms drawn where this run does not rewrite: they are never changed in place. + shared: HashSet, + /// The forms already drawn as themselves: `true` when rewritten in place, `false` when + /// left as they were. Any other use that needs something else gets a copy. + used: HashMap, + /// Each form's stream as it was before the run. + original: HashMap, + /// Form rewrites left. + budget: usize, +} + +/// The XObjects (forms and images, and those the forms draw), tiling patterns and Type 3 +/// glyphs that the pages not in `pages` use, and those any page's annotation appearances use +/// (this run does not rewrite them): rewriting one of those in place would change what they +/// show. +fn drawn_elsewhere(doc: &Document, pages: &[usize]) -> HashSet { + let mut seen: HashSet = HashSet::new(); + let mut todo: Vec = Vec::new(); + let xobjects_of = |res: Option<&Object>, todo: &mut Vec| { + let Some(res) = res.map(|r| doc.resolve(r)) else { return }; + let Some(res) = res.as_dict() else { return }; + let dict_at = |key: &[u8]| res.get(key).map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()); + for dict in [dict_at(b"XObject"), dict_at(b"Pattern")].into_iter().flatten() { + todo.extend(dict.iter().map(|(_, o)| o.clone())); + } + for (_, font) in dict_at(b"Font").iter().flat_map(Dict::iter) { + let font = doc.resolve(font); + if let Some(procs) = font.as_dict().and_then(|f| f.get(b"CharProcs")).map(|p| doc.resolve(p)).and_then(|p| p.as_dict().cloned()) { + todo.extend(procs.iter().map(|(_, g)| g.clone())); + } + } + }; + for (i, page) in page_list(doc).into_iter().enumerate() { + if !pages.contains(&i) { + xobjects_of(page.dict.get(b"Resources"), &mut todo); + } + let annots = page.dict.get(b"Annots").map(|a| doc.resolve(a)).and_then(|a| a.as_array().cloned()).unwrap_or_default(); + for a in annots { + let Some(ap) = doc.resolve(&a).as_dict().and_then(|d| d.get(b"AP").cloned()) else { continue }; + let Some(ap) = doc.resolve(&ap).as_dict().cloned() else { continue }; + for (_, v) in ap.iter() { + match &*doc.resolve(v) { + Object::Dict(states) => todo.extend(states.iter().map(|(_, s)| s.clone())), + _ => todo.push(v.clone()), + } + } + } + } + while let Some(o) = todo.pop() { + let Some(r) = o.as_ref() else { continue }; + if !seen.insert(r) { + continue; + } + let res = match &*doc.get(r) { + Object::Stream(s) => s.dict.get(b"Resources").cloned(), + _ => None, + }; + xobjects_of(res.as_ref(), &mut todo); + } + seen +} + +/// A name for another copy drawn by `name` that `xobjects` does not hold yet. +fn fresh_name(xobjects: &Dict, name: &[u8]) -> Name { + let mut i = 1usize; + loop { + let mut candidate = name.to_vec(); + candidate.extend_from_slice(format!("_hl{i}").as_bytes()); + if !xobjects.contains(&candidate) { + return candidate; + } + i = i.saturating_add(1); + } +} + +/// Rewrite one drawing context - a page's streams, or a form's - that starts in `start`, and +/// the forms it draws. Returns each stream's new bytes (`None` where it did not change), the +/// strokes fixed (in the forms too), and the `/XObject` dictionary for the context's resources +/// when its names must now point at other objects. +fn fix_context( + doc: &mut Document, + run: &mut Run, + h: &Hairlines, + data: &[Vec], + start: &State, + resources: Option<&Dict>, + depth: usize, +) -> (Vec>>, usize, Option) { + let none = Renames::new(); + let mut drawn: Vec = Vec::new(); + let mut state = start.clone(); + let mut new: Vec>> = Vec::with_capacity(data.len()); + let mut fixed = 0usize; + for d in data { + let (bytes, count) = rewrite(d, &mut state, h, resources, doc, &mut drawn, &none); + fixed += count; + new.push(bytes); + } + if drawn.is_empty() { + return (new, fixed, None); + } + let Some(xo) = resources.and_then(|r| r.get(b"XObject")).map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()) else { + return (new, fixed, None); + }; + // The object each (name, key) is drawn as, and each name's objects in order of first use. + let mut chosen: HashMap<(Name, Key), ObjRef> = HashMap::new(); + let mut targets: Vec<(Name, Vec)> = Vec::new(); + for d in drawn { + let k = key_of(d.ctm, d.width); + if chosen.contains_key(&(d.name.clone(), k)) { + continue; + } + let (to, count) = fix_form(doc, run, h, d.r, d.ctm, d.width, depth + 1); + fixed += count; + chosen.insert((d.name.clone(), k), to); + match targets.iter_mut().find(|(name, _)| *name == d.name) { + Some((_, list)) => { + if !list.contains(&to) { + list.push(to); + } + } + None => targets.push((d.name, vec![to])), + } + } + // The first object keeps the name; each other one gets a name of its own, and the `Do`s + // drawing it are renamed. + let mut repointed = xo; + let mut changed = false; + let mut renames = Renames::new(); + for (name, list) in &targets { + for (i, to) in list.iter().enumerate() { + let at = if i == 0 { name.clone() } else { fresh_name(&repointed, name) }; + if repointed.get(&at).and_then(Object::as_ref) != Some(*to) { + repointed.set(at.clone(), Object::Ref(*to)); + changed = true; + } + if i > 0 { + for ((was, k), t) in &chosen { + if was == name && t == to { + renames.insert((was.clone(), *k), at.clone()); + } + } + } + } + } + if !renames.is_empty() { + // The same rewrite again, with the renamed `Do`s. + let mut state = start.clone(); + let mut again: Vec = Vec::new(); + new = data.iter().map(|d| rewrite(d, &mut state, h, resources, &*doc, &mut again, &renames).0).collect(); + } + (new, fixed, changed.then_some(repointed)) +} + +/// The object `r` is drawn as under `ctm` with line width `width` in effect, rewritten for +/// them, and the strokes that rewrite fixed. It is `r` itself, rewritten in place, unless +/// something else draws `r` differently; then it is a copy. +fn fix_form(doc: &mut Document, run: &mut Run, h: &Hairlines, r: ObjRef, ctm: [f64; 6], width: f64, depth: usize) -> (ObjRef, usize) { + let k = key_of(ctm, width); + if let Some(&to) = run.done.get(&(r, k)) { + return (to, 0); + } + if depth > MAX_DEPTH || run.budget == 0 { + return (r, 0); + } + run.budget -= 1; + // Marked before rewriting, so a form that draws itself ends. + run.done.insert((r, k), r); + let s = match run.original.get(&r) { + Some(s) => s.clone(), + None => { + let Some(s) = stream_of(doc, r) else { return (r, 0) }; + run.original.insert(r, s.clone()); + s + } + }; + let Ok(data) = s.decoded() else { return (r, 0) }; + let matrix = s.dict.get(b"Matrix").map(|m| doc.resolve(m)).and_then(|m| m.as_array().cloned()).and_then(|a| { + let v: Vec = a.iter().filter_map(|x| doc.resolve(x).as_f64()).collect(); + <[f64; 6]>::try_from(v).ok() + }); + let base = match matrix { + Some(m) => mul(m, ctm), + None => ctm, + }; + let res = s.dict.get(b"Resources").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()); + let (new, fixed, xo) = fix_context(doc, run, h, std::slice::from_ref(&data), &State::new(base, width), res.as_ref(), depth); + let bytes = new.into_iter().next().flatten(); + let to = if bytes.is_none() && xo.is_none() { + if run.used.get(&r) == Some(&true) { + // `r` was rewritten in place for another use; this one draws it as it was. + doc.add(Object::Stream(s)) + } else { + run.used.insert(r, false); + r + } + } else { + let mut form = s; + if let Some(xo) = xo { + let mut res = res.unwrap_or_default(); + res.set(b"XObject".to_vec(), Object::Dict(xo)); + form.dict.set(b"Resources".to_vec(), Object::Dict(res)); + } + if let Some(bytes) = bytes { + let mut dict = form.dict; + for key in [&b"Length"[..], b"Filter", b"DecodeParms"] { + dict.remove(key); + } + form = Stream::flate(dict, &bytes); + } + if run.shared.contains(&r) || run.used.contains_key(&r) { + doc.add(Object::Stream(form)) + } else { + doc.set(r, Object::Stream(form)); + run.used.insert(r, true); + r + } + }; + run.done.insert((r, k), to); + (to, fixed) +} + +/// Fix the hairlines on `pages` (0-based). Each page's content, and each Form XObject it draws +/// (for the scale and line width it is drawn under), is rewritten where a stroke is thinner +/// than `h.narrower_than` points on the page. A form drawn under more than one, or drawn by a +/// page not in `pages` or by an annotation's appearance, is rewritten as a copy for the uses +/// that need one, and the page's (or drawing form's) resources point at the copy; a resource +/// dictionary other pages share is not changed. Type 3 glyphs and tiling patterns are rewritten +/// in place, once, at a nominal scale (see `fix_in_resources`), and those other uses of them are +/// counted in [`Fixed::elsewhere`]. +pub fn fix_hairlines(doc: &mut Document, pages: &[usize], h: &Hairlines) -> Result { + let all = page_list(doc); + check(pages, all.len())?; + if !h.in_range() { + return Err(EditError::Invalid(format!("the hairline widths must be numbers of points, above 0 and at most {}", Hairlines::MAX_POINTS))); + } + let mut result = Fixed::default(); + let mut run = + Run { done: HashMap::new(), shared: drawn_elsewhere(doc, pages), used: HashMap::new(), original: HashMap::new(), budget: MAX_FORMS }; + let mut done_glyphs: HashSet = HashSet::new(); + for &i in pages { + let Some(page) = page_list(doc).get(i).cloned() else { continue }; + let resources = page.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()); + let mut page_fixed = 0usize; + // The page's streams, each rewritten on its own (state carries across them), and the + // forms they draw. A page with a stream that cannot be read is left alone. + if let Some(streams) = PageStreams::read(doc, &page.dict) { + let start = State::new([1.0, 0.0, 0.0, 1.0, 0.0, 0.0], 1.0); + let (new, fixed, xo) = fix_context(doc, &mut run, h, &streams.data, &start, resources.as_ref(), 0); + page_fixed += fixed; + streams.write(doc, page.obj, new)?; + if let Some(xo) = xo { + // This page's own resources point at the copies; a resource dictionary other + // pages share is not changed. + let mut res = resources.clone().unwrap_or_default(); + res.set(b"XObject".to_vec(), Object::Dict(xo)); + doc.update_dict(page.obj, |d| d.set(b"Resources".to_vec(), Object::Dict(res)))?; + } + } + if h.type3 || h.patterns { + for res in resources_below(doc, resources.as_ref()) { + page_fixed += fix_in_resources(doc, Some(&res), h, &mut done_glyphs, &run.shared, &mut result.elsewhere)?; + } + } + if page_fixed > 0 { + result.lines += page_fixed; + result.pages += 1; + } + } + Ok(result) +} + +/// `resources` and the resources of every form they reach (a form that names none uses its +/// parent's), each once. +fn resources_below(doc: &Document, resources: Option<&Dict>) -> Vec { + let mut all: Vec = resources.into_iter().cloned().collect(); + let mut seen: HashSet = HashSet::new(); + let mut at = 0; + while let Some(res) = all.get(at).cloned() { + at += 1; + let Some(xo) = res.get(b"XObject").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()) else { continue }; + for (_, x) in xo.iter() { + let Some(r) = x.as_ref() else { continue }; + if !seen.insert(r) { + continue; + } + let Some(s) = stream_of(doc, r) else { continue }; + if s.dict.name(b"Subtype") == Some(b"Form") + && let Some(inner) = s.dict.get(b"Resources").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()) + { + all.push(inner); + } + } + } + all +} + +/// Strokes inside Type 3 glyph procedures and tiling patterns of `resources`, on request. Their +/// drawing scale is the font matrix or the pattern matrix (the text size and placement are +/// not known here, so the glyph's own scale stands in). They are rewritten in place, once per +/// run: a pattern's space is set by its matrix whatever CTM it is painted under, and a glyph +/// is measured at a nominal size, so there is no per-use scale to make a copy for. The forms +/// they draw are not followed. Each one changed that is in `shared` is added to `elsewhere`. +fn fix_in_resources( + doc: &mut Document, + resources: Option<&Dict>, + h: &Hairlines, + done: &mut HashSet, + shared: &HashSet, + elsewhere: &mut usize, +) -> Result { + let Some(res) = resources else { return Ok(0) }; + let mut fixed = 0usize; + let mut targets: Vec<(ObjRef, [f64; 6])> = Vec::new(); + if h.type3 + && let Some(fonts) = res.get(b"Font").map(|f| doc.resolve(f)).and_then(|f| f.as_dict().cloned()) + { + for (_, f) in fonts.iter() { + let fd = doc.resolve(f); + let Some(fd) = fd.as_dict() else { continue }; + if fd.name(b"Subtype") != Some(b"Type3") { + continue; + } + let fm = fd.get(b"FontMatrix").map(|m| doc.resolve(m)).and_then(|m| m.as_array().cloned()).and_then(|a| { + let v: Vec = a.iter().filter_map(|x| doc.resolve(x).as_f64()).collect(); + <[f64; 6]>::try_from(v).ok() + }); + // Glyph space → text space at a nominal 10 pt. + let base = fm.map(|m| mul(m, [10.0, 0.0, 0.0, 10.0, 0.0, 0.0])).unwrap_or([0.01, 0.0, 0.0, 0.01, 0.0, 0.0]); + if let Some(procs) = fd.get(b"CharProcs").map(|p| doc.resolve(p)).and_then(|p| p.as_dict().cloned()) { + for (_, g) in procs.iter() { + if let Some(r) = g.as_ref() { + targets.push((r, base)); + } + } + } + } + } + if h.patterns + && let Some(pats) = res.get(b"Pattern").map(|p| doc.resolve(p)).and_then(|p| p.as_dict().cloned()) + { + for (_, p) in pats.iter() { + let Some(r) = p.as_ref() else { continue }; + let Some(s) = stream_of(doc, r) else { continue }; + if s.dict.int(b"PatternType") != Some(1) { + continue; + } + let pm = s.dict.get(b"Matrix").map(|m| doc.resolve(m)).and_then(|m| m.as_array().cloned()).and_then(|a| { + let v: Vec = a.iter().filter_map(|x| doc.resolve(x).as_f64()).collect(); + <[f64; 6]>::try_from(v).ok() + }); + targets.push((r, pm.unwrap_or([1.0, 0.0, 0.0, 1.0, 0.0, 0.0]))); + } + } + let none = Renames::new(); + for (r, base) in targets { + if !done.insert(r) { + continue; + } + let Some(s) = stream_of(doc, r) else { continue }; + let Ok(data) = s.decoded() else { continue }; + let res = s.dict.get(b"Resources").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()); + let mut inner = Vec::new(); + let (new, n_fixed) = rewrite(&data, &mut State::new(base, 1.0), h, res.as_ref(), doc, &mut inner, &none); + if let Some(bytes) = new { + let mut dict = s.dict.clone(); + dict.remove(b"Length"); + dict.remove(b"Filter"); + dict.remove(b"DecodeParms"); + doc.set(r, Object::Stream(Stream::flate(dict, &bytes))); + fixed += n_fixed; + *elsewhere += usize::from(n_fixed > 0 && shared.contains(&r)); + } + } + let _ = n; + Ok(fixed) +} diff --git a/crates/edit/src/images.rs b/crates/edit/src/images.rs index 8ac77c9deb..c4a39bf689 100644 --- a/crates/edit/src/images.rs +++ b/crates/edit/src/images.rs @@ -1884,6 +1884,24 @@ fn crop_image_impl( Ok(()) } +/// The `Do` of each image `indexes` (from [`page_images`]) on `page`, as an index among the +/// page's joined operators: what Copy takes of them. +pub(crate) fn image_ops(doc: &Document, page: usize, indexes: &[usize]) -> Result, EditError> { + if indexes.is_empty() { + return Ok(Vec::new()); + } + let images = page_images(doc, page)?; + indexes + .iter() + .map(|i| { + images + .get(*i) + .map(|image| image.op) + .ok_or_else(|| EditError::Invalid("a selected image no longer exists; select the objects again".into())) + }) + .collect() +} + /// Plan all transforms before changing a stream; indexes refer to the original inventory. /// Splice only the chosen Do operators and preserve all other bytes and stream dictionaries. pub(crate) fn translate_images(doc: &mut Document, page: usize, indexes: &[usize], offset: [f64; 2]) -> Result<(), EditError> { diff --git a/crates/edit/src/lib.rs b/crates/edit/src/lib.rs index da51839047..d0f6d2c863 100644 --- a/crates/edit/src/lib.rs +++ b/crates/edit/src/lib.rs @@ -23,7 +23,7 @@ #![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)] use pdfcraft_cos::{Dict, Document, ObjRef, Object, Stream}; -use pdfcraft_fonts::{helvetica_width, literal, win_ansi}; +use pdfcraft_fonts::helvetica_width; #[derive(Debug, thiserror::Error, PartialEq)] pub enum EditError { @@ -46,7 +46,9 @@ pub use background::page_background; mod rules; pub use rules::page_rules; pub mod objects; -pub use objects::{EditableObject, MAX_MOVE_OBJECTS, ObjectKind, ObjectTarget, editable_objects, move_objects}; +pub use objects::{ + CopiedObjects, EditableObject, MAX_MOVE_OBJECTS, ObjectKind, ObjectTarget, copy_objects, editable_objects, move_objects, paste_objects, +}; mod flatten; pub use flatten::{flatten, flatten_fill_sign}; @@ -62,6 +64,14 @@ pub use text::{ BlockKey, BlockStyle, LineEdit, MAX_FORM_DEPTH, MAX_FORM_VISITS, RangeStyle, ReadingBlocks, TextBlock, TextLine, TextSession, block_overlapping, clear_block, glyph_text_lines, reading_blocks, reading_blocks_checked, replace_block, replace_line, rewrite_block, text_blocks, text_lines, }; +pub mod printer_marks; +pub use printer_marks::{PrinterMarks, add_printer_marks}; +mod colour; +pub mod convert_colours; +pub mod hairlines; +mod page_streams; +pub use convert_colours::{ColourTarget, ConvertColours, Converted as ColoursConverted, ImageRecolour, convert_colours}; +pub use hairlines::{Fixed as HairlinesFixed, Hairlines, fix_hairlines}; pub mod added; pub use added::{ Added, AddedImage, AddedText, Align, Content, Family, Ink, MarkerScan, add_content, delete_content, first_undrawable, isolate_content, diff --git a/crates/edit/src/linebreak.rs b/crates/edit/src/linebreak.rs index a4682679f8..cf044d9e5c 100644 --- a/crates/edit/src/linebreak.rs +++ b/crates/edit/src/linebreak.rs @@ -6,8 +6,56 @@ //! and quotes, currency signs before an amount): the line-start and line-end prohibitions of //! JIS X 4051 ("kinsoku"), which UAX #14 gives as classes CL, CP, EX, IS, NS and OP, PR. //! -//! Korean is written with spaces between words and keeps breaking only there; so does Thai, whose -//! words need a dictionary to find. +//! Thai is also written without spaces between words, but a row may not end between any two of its +//! characters: its words need a dictionary to find. They come from ICU4X's dictionary line +//! segmenter (UAX #14 with dictionary breaking for class SA), kept only where a Thai character is on +//! either side. A row never begins with a following vowel, a tone or other mark, Mai Yamok (ๆ) or +//! Paiyannoi (ฯ), nor ends with a leading vowel (เ แ โ ใ ไ), whatever the dictionary says. +//! +//! Korean is written with spaces between words and keeps breaking only there. + +use std::sync::LazyLock; + +use icu_segmenter::options::LineBreakOptions; +use icu_segmenter::{LineSegmenter, LineSegmenterBorrowed}; + +/// The dictionary segmenter, built once: its data is compiled in, so building it cannot fail. +static SEGMENTER: LazyLock> = LazyLock::new(|| LineSegmenter::new_dictionary(LineBreakOptions::default())); + +/// Letters, vowels, marks, digits and signs of the Thai block. +fn thai(c: char) -> bool { + matches!(c, '\u{0e01}'..='\u{0e5b}') +} + +/// Thai that belongs to the character before it: Paiyannoi, the following vowels, Lakkhangyao, +/// Mai Yamok and the tone and other marks. +fn thai_no_start(c: char) -> bool { + matches!(c, '\u{0e2f}' | '\u{0e30}'..='\u{0e3a}' | '\u{0e45}'..='\u{0e4e}') +} + +/// The leading vowels, written before the consonant they follow in speech. +fn thai_no_end(c: char) -> bool { + matches!(c, '\u{0e40}'..='\u{0e44}') +} + +/// Byte offsets where the dictionary lets a row of Thai end inside `word`. +fn thai_breaks(word: &str) -> Vec { + if !word.chars().any(thai) { + return Vec::new(); + } + SEGMENTER + .segment_str(word) + .filter(|&at| { + let before = word.get(..at).and_then(|s| s.chars().next_back()); + let after = word.get(at..).and_then(|s| s.chars().next()); + match (before, after) { + (Some(b), Some(a)) => (thai(b) || thai(a)) && !thai_no_start(a) && !thai_no_end(b) && !no_start(a) && !no_end(b), + // The start and end of the word are not breaks inside it. + _ => false, + } + }) + .collect() +} /// Writing with no spaces between words, where a row may end between two characters (UAX #14 ID). fn ideographic(c: char) -> bool { @@ -77,29 +125,49 @@ fn no_end(c: char) -> bool { ) } -/// Whether a row may end between `before` and `after`, neither of them a space. -pub(crate) fn between(before: char, after: char) -> bool { - (ideographic(before) || ideographic(after)) && !no_start(after) && !no_end(before) +/// Whether a row may end between `before` and `after` in Chinese or Japanese, neither of them a +/// space. Thai needs the whole word ([`breaks`]). +fn between(before: char, after: char) -> bool { + (ideographic(before) || ideographic(after)) && !no_start(after) && !no_end(before) && !thai_no_start(after) && !thai_no_end(before) +} + +/// Byte offsets inside `word` (text without spaces) where a row may end, in order: never 0 or the +/// word's length, always on a character boundary. +pub(crate) fn breaks(word: &str) -> Vec { + let mut out: Vec = Vec::new(); + let mut prev: Option = None; + for (at, c) in word.char_indices() { + if let Some(before) = prev + && between(before, c) + { + out.push(at); + } + prev = Some(c); + } + out.extend(thai_breaks(word)); + out.sort_unstable(); + out.dedup(); + out } /// `word` (text without spaces) cut where a row may end inside it; the pieces join with nothing. pub(crate) fn pieces(word: &str) -> Vec<&str> { let mut out = Vec::new(); let mut start = 0; - let mut prev: Option = None; - for (at, c) in word.char_indices() { - if let Some(before) = prev - && between(before, c) - { - out.extend(word.get(start..at)); - start = at; - } - prev = Some(c); + for at in breaks(word) { + out.extend(word.get(start..at)); + start = at; } out.extend(word.get(start..).filter(|rest| !rest.is_empty())); out } +/// For each character of `word` (text without spaces), whether a row may end just before it. +pub(crate) fn breaks_before_chars(word: &str) -> Vec { + let cuts = breaks(word); + word.char_indices().map(|(at, _)| cuts.binary_search(&at).is_ok()).collect() +} + #[cfg(test)] mod tests { use super::*; @@ -121,4 +189,64 @@ mod tests { assert_eq!(pieces(""), Vec::<&str>::new()); assert!(!between('a', 'b') && between('字', 'a') && !between('字', '。') && !between('「', '字')); } + + #[test] + fn thai_breaks_between_dictionary_words() { + assert_eq!(pieces("ภาษาไทยไม่มีช่องว่างระหว่างคำ"), ["ภาษา", "ไทย", "ไม่มี", "ช่อง", "ว่าง", "ระหว่าง", "คำ"]); + assert_eq!(pieces("น้ำประปาสะอาด"), ["น้ำ", "ประปา", "สะอาด"]); + assert_eq!(pieces("การประปาส่วนภูมิภาค"), ["การ", "ประปา", "ส่วน", "ภูมิภาค"]); + } + + #[test] + fn thai_vowels_marks_and_repetition_stay_with_the_word_before_them() { + // The dictionary alone would start a row with Mai Yamok, with the Sara Aa of a decomposed + // Sara Am, or with a closing bracket. + assert_eq!(pieces("เด็กๆชอบกินขนม"), ["เด็กๆ", "ชอบ", "กิน", "ขนม"]); + assert_eq!(pieces("ท\u{0e4d}\u{0e32}งาน"), ["ท\u{0e4d}\u{0e32}", "งาน"]); + assert_eq!(pieces("กรุงเทพฯเป็นเมืองหลวง"), ["กรุงเทพฯ", "เป็น", "เมือง", "หลวง"]); + assert_eq!(pieces("สวัสดีครับ(ทดสอบ)"), ["สวัสดี", "ครับ", "(ทดสอบ)"]); + } + + #[test] + fn thai_keeps_numbers_and_latin_whole() { + assert_eq!(pieces("ราคา100บาทต่อเดือน"), ["ราคา", "100บาท", "ต่อ", "เดือน"]); + assert_eq!(pieces("พ.ศ.2569"), ["พ.ศ.2569"]); + assert_eq!(pieces("PDF"), ["PDF"]); + assert_eq!(breaks("ไทย"), Vec::::new()); + } + + #[test] + fn breaks_before_chars_marks_the_first_character_of_each_later_piece() { + let flags = breaks_before_chars("น้ำประปา"); + let starts: Vec = flags.iter().enumerate().filter_map(|(i, &b)| b.then_some(i)).collect(); + // น ้ ำ | ป ร ะ ป า + assert_eq!(starts, [3]); + assert_eq!(breaks_before_chars(""), Vec::::new()); + } + + #[test] + fn pieces_of_arbitrary_thai_rejoin_to_the_input_and_never_start_with_a_mark() { + // A deterministic generator over Thai, Latin, digits and CJK, including mark-only runs and + // unusual orders that no dictionary word has. + let alphabet: Vec = ('\u{0e01}'..='\u{0e5b}').chain(['a', '1', '(', ')', '字', 'の', '.']).collect(); + let mut state: u64 = 0x2545_f491_4f6c_dd1d; + for _ in 0..2000 { + let mut word = String::new(); + state = state.wrapping_mul(6_364_136_223_846_793_005).wrapping_add(1); + let len = (state >> 59) as usize + 1; + for _ in 0..len { + state = state.wrapping_mul(6_364_136_223_846_793_005).wrapping_add(1); + let pick = (state >> 33) as usize % alphabet.len(); + word.extend(alphabet.get(pick)); + } + let parts = pieces(&word); + assert_eq!(parts.concat(), word); + assert!(parts.iter().all(|p| !p.is_empty()), "{word:?} gave {parts:?}"); + for part in parts.iter().skip(1) { + let first = part.chars().next().unwrap_or(' '); + assert!(!thai_no_start(first) && !no_start(first), "{word:?} gave {parts:?}"); + } + assert_eq!(breaks_before_chars(&word).len(), word.chars().count()); + } + } } diff --git a/crates/edit/src/mark_transaction_tests.rs b/crates/edit/src/mark_transaction_tests.rs index 2623da88ba..22530c8836 100644 --- a/crates/edit/src/mark_transaction_tests.rs +++ b/crates/edit/src/mark_transaction_tests.rs @@ -121,6 +121,7 @@ fn apply_replacement(doc: &mut Document, kind: MarkKind, source: MarkSource, rep MarkKind::Background => { add_background(doc, SELECTED, &Background { color: [0.1, 0.3, 0.7], replace_backdrop: true, ..Default::default() }, replace) } + MarkKind::PrinterMarks => Ok(()), } } @@ -172,6 +173,7 @@ fn replacements_commit_all_selected_pages_and_preserve_other_marks_and_shared_re MarkKind::HeaderFooter => "(New header)".to_string(), MarkKind::Watermark => format!("/PCPic{} Do", source.xobject.num), MarkKind::Background => "0.1 0.3 0.7 rg".to_string(), + MarkKind::PrinterMarks => String::new(), }; apply_replacement(&mut doc, kind, source, true).unwrap(); let doc = reopen(&doc); diff --git a/crates/edit/src/marks.rs b/crates/edit/src/marks.rs index 344c76250f..d772c48f4b 100644 --- a/crates/edit/src/marks.rs +++ b/crates/edit/src/marks.rs @@ -1,4 +1,75 @@ -use super::{Document, EditError, Object, Rgb, added}; +use pdfcraft_fonts::shaped_text::{Piece, ShapedFonts, clusters, first_undrawable_in_line, is_thai, thai_message, thai_pieces}; + +use super::{Dict, Document, EditError, Object, Rgb, added}; + +/// The Thai of a header, footer or text watermark, drawn from the Thai face as Add text draws it: +/// Type 3 fonts added to the document once and to each page the mark goes on, beside the page +/// font that draws the rest. +#[derive(Default)] +pub(crate) struct MarkText { + shaped: ShapedFonts, + /// The fonts, by the resource names the marked pages give them. + fonts: Dict, +} + +impl MarkText { + /// The fonts for the Thai in `lines` (every line the mark draws, on any page), named so that + /// none of `pages` (of `all`) has the name already. Thai this build can't draw (it has no Thai + /// face, or the face lacks a character) is refused, saying what to do. + pub(crate) fn new<'a>( + doc: &mut Document, + all: &[pdfcraft_model::Page], + pages: &[usize], + lines: impl IntoIterator, + ) -> Result { + let thai = |line: &str| thai_pieces(line).map_err(|e| EditError::Invalid(thai_message(e, line))); + let pieces = lines.into_iter().filter(|l| l.chars().any(is_thai)).map(thai).collect::, _>>()?; + if pieces.is_empty() { + return Ok(Self::default()); + } + let mut taken = Dict::new(); + for page in pages.iter().filter_map(|&i| all.get(i)) { + for (name, font) in crate::place::page_fonts(doc, page).iter() { + taken.set(name.clone(), font.clone()); + } + } + let mut fonts = Dict::new(); + let shaped = + ShapedFonts::new(clusters(pieces.iter().map(Vec::as_slice)), pdfcraft_fonts::shaped_glyph, &taken, &mut fonts, "PCTh", &mut |o| { + Object::Ref(doc.add(o)) + }) + .map_err(|(e, c)| EditError::Invalid(thai_message(e, &c.text)))?; + Ok(Self { shaped, fonts }) + } + + /// Show `line` inside `BT … ET` from `x` on baseline `y` at `size`: its Thai from the fonts, + /// the rest in the page font `latin` (selected when `current` is `None`), `width` wide. + #[allow(clippy::too_many_arguments)] + pub(crate) fn show( + &self, + out: &mut Vec, + line: &str, + x: f64, + y: f64, + size: f64, + latin: &str, + width: impl Fn(&str) -> f64, + current: &mut Option, + ) { + let plain = || vec![Piece::Latin(line.into())]; + // `new` shaped every line drawn, so this shapes. + let pieces = if self.fonts.is_empty() || !line.chars().any(is_thai) { plain() } else { thai_pieces(line).unwrap_or_else(|_| plain()) }; + self.shaped.show_line(out, &pieces, x, y, size, latin, pdfcraft_fonts::win_ansi, width, current); + } + + /// Add the fonts to `page`'s own resources. + pub(crate) fn add_to(&self, doc: &mut Document, page: &pdfcraft_model::Page) -> Result<(), EditError> { + if self.fonts.is_empty() { + return Ok(()); + } + crate::place::add_page_fonts(doc, page, &self.fonts) + } +} /// The kinds of page marks. #[derive(Clone, Copy, Debug, PartialEq, Eq)] @@ -6,6 +77,8 @@ pub enum MarkKind { HeaderFooter, Watermark, Background, + /// Printer marks: trim, bleed, registration, colour bars, page information. + PrinterMarks, } impl MarkKind { @@ -14,6 +87,7 @@ impl MarkKind { MarkKind::HeaderFooter => "HeaderFooter", MarkKind::Watermark => "Watermark", MarkKind::Background => "Background", + MarkKind::PrinterMarks => "PrinterMarks", } } } @@ -61,13 +135,13 @@ impl Default for HeaderFooter { } impl HeaderFooter { - /// The first character [`add_header_footer`] would refuse, because the standard fonts can't - /// draw it (#125). Tokens only add ASCII (Bates prefixes and suffixes are part of the - /// template), so checking the templates, split as `expand`'s output is, covers every page - /// (and a `\r` that would be drawn is refused too). The dialog asks this too, so its preview - /// never offers text that applying refuses (#2013). + /// The first character [`add_header_footer`] would refuse, because neither the standard + /// fonts nor (for Thai) the Thai face can draw it (#125). Tokens only add ASCII (Bates + /// prefixes and suffixes are part of the template), so checking the templates, split as + /// `expand`'s output is, covers every page (and a `\r` that would be drawn is refused too). + /// The dialog asks this too, so its preview never offers text that applying refuses (#2013). pub fn first_undrawable(&self) -> Option { - self.text.iter().flat_map(|t| t.lines()).find_map(pdfcraft_fonts::first_non_win_ansi) + self.text.iter().flat_map(|t| t.lines()).find_map(first_undrawable_in_line) } /// The room the header and the footer take at the top and the bottom of a page, in points: @@ -139,12 +213,13 @@ pub struct Watermark { } impl Watermark { - /// The first character of a text watermark [`add_watermark`] would refuse, because the - /// standard fonts can't draw it (#125). A picture watermark draws no text. + /// The first character of a text watermark [`add_watermark`] would refuse, because neither + /// the standard fonts nor (for Thai) the Thai face can draw it (#125). A picture watermark + /// draws no text. pub fn first_undrawable(&self) -> Option { match self.source { Some(_) => None, - None => self.text.trim().lines().find_map(pdfcraft_fonts::first_non_win_ansi), + None => self.text.trim().lines().find_map(first_undrawable_in_line), } } } diff --git a/crates/edit/src/objects.rs b/crates/edit/src/objects.rs index 6a64e3cfce..5b1a36caef 100644 --- a/crates/edit/src/objects.rs +++ b/crates/edit/src/objects.rs @@ -9,6 +9,9 @@ use pdfcraft_cos::{DecodeBudget, Dict, Document, Object}; use crate::text::ContentStream; use crate::{Content, EditError}; +mod clip; +pub use clip::{CopiedObjects, copy_objects, paste_objects}; + pub const MAX_MOVE_OBJECTS: usize = 1_000; const MAX_PAGE_OBJECTS: usize = 10_000; const MAX_COORDINATE: f64 = 1e9; diff --git a/crates/edit/src/objects/clip.rs b/crates/edit/src/objects/clip.rs new file mode 100644 index 0000000000..4402667b76 --- /dev/null +++ b/crates/edit/src/objects/clip.rs @@ -0,0 +1,283 @@ +//! Edit a PDF ▸ Copy, Cut and Paste of objects selected together (#5989): paragraphs, images, +//! Form artwork and added content, pasted as page content that draws the same. +//! +//! A copy is a content stream of its own that draws the objects and nothing else, in the page's +//! user space, with the resources it names. It replays every state operator of the page up to +//! the last object (transformations, clips, colours, ExtGStates, text state), so each object is +//! drawn in the state it had, and leaves out what paints anything else: other text, images, +//! shadings and paths (a path that clips is kept, without its paint). Each copied text-showing +//! operator is placed by its own text matrix, which is where its glyphs start, so the text +//! positioning between operators is left out. Marked content is left out too: a paste is +//! untagged and in no layer. +//! +//! A paste draws that stream on top of the page through Ungroup's merge ([`ungroup_ops`]): the +//! copy is read as Form artwork whose resources join the page's under names that don't clash, +//! moved by the offset, with the patterns it paints moved along. Its resources must already be +//! the target document's: the engine copies them over first when the copy came from another +//! document. + +use std::collections::{BTreeMap, BTreeSet, HashMap}; + +use pdfcraft_content::{Matrix, Op, serialize_ops}; +use pdfcraft_cos::{Dict, Document, ObjRef, Object, Stream}; + +use super::{EditableObject, MAX_COORDINATE, MAX_MOVE_OBJECTS, ObjectKind, ObjectTarget, checked_streams, editable_objects, valid_rect}; +use crate::EditError; +use crate::ungroup::{CATEGORIES, DEFAULT_SPACES, MAX_FORM_BYTES, Ungrouped, page_blend_space, resource_operand, ungroup_ops}; + +/// How deep the copied graphics state may nest (`q`), as the page's own may. +const MAX_NESTING: usize = 1024; + +/// Objects copied from one page: what [`paste_objects`] draws again. +#[derive(Clone, Debug, PartialEq)] +pub struct CopiedObjects { + /// A content stream that draws the objects alone, in the source page's user space: its `q` + /// and `Q` balance and its text objects close. + pub content: Vec, + /// The resources `content` names, as the source page has them: references are into the + /// document the copy came from. + pub resources: Dict, + /// The union of the objects' boxes (user space of the source page). + pub rect: [f64; 4], + /// The objects as [`editable_objects`] listed them, in inventory order. + pub objects: Vec, +} + +impl CopiedObjects { + /// The text of the copied paragraphs and text boxes, one per line: what the system clipboard + /// gets for other applications. + pub fn text(&self) -> String { + self.objects.iter().filter_map(|o| o.text.as_deref()).filter(|t| !t.is_empty()).collect::>().join("\n") + } +} + +/// Copy `targets` (references from one [`editable_objects`] inventory of `page`). The document +/// is not changed. +pub fn copy_objects(doc: &Document, page: usize, targets: &[ObjectTarget]) -> Result { + if targets.is_empty() || targets.len() > MAX_MOVE_OBJECTS { + return Err(EditError::Invalid("select between 1 and 1000 objects to copy".into())); + } + let mut inventory: BTreeMap<_, _> = editable_objects(doc, page)?.into_iter().map(|o| (o.target, o)).collect(); + let mut objects = Vec::with_capacity(targets.len()); + for target in targets.iter().collect::>() { + objects + .push(inventory.remove(target).ok_or_else(|| EditError::Invalid("a selected object no longer exists; select the objects again".into()))?); + } + if objects.len() != targets.len() { + return Err(EditError::Invalid("an object was selected more than once".into())); + } + let indexes = |kind| objects.iter().filter(|o| o.target.kind == kind).map(|o| o.target.index).collect::>(); + let streams = checked_streams(doc, page)?; + let pieces = crate::text::joined(&streams); + let ops = pieces.parse().ops; + let mut keep = crate::text::paragraph_ops(doc, page, &indexes(ObjectKind::Text), false)?; + keep.extend(crate::images::image_ops(doc, page, &indexes(ObjectKind::Image))?); + // Added content owns its stream: everything that stream paints is the item. + let added: Vec<_> = crate::list_added(doc).into_iter().filter(|a| a.page == page).collect(); + let owned = indexes(ObjectKind::Added) + .into_iter() + .map(|i| { + added.get(i).map(|a| a.obj).ok_or_else(|| EditError::Invalid("a selected added item no longer exists; select the objects again".into())) + }) + .collect::, _>>()?; + if !owned.is_empty() { + for (i, op) in ops.iter().enumerate() { + let stream = streams.get(pieces.pieces_of(op).0).and_then(|s| s.entry.as_ref()); + if paints(op) && stream.is_some_and(|r| owned.contains(&r)) { + keep.insert(i); + } + } + } + let p = pdfcraft_model::pages(doc).into_iter().nth(page).ok_or(EditError::NoSuchPage(page))?; + let res = p.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default(); + let shows = crate::text::show_starts(doc, &ops, &res)?; + let drawn = replay(&ops, &keep, &shows)?; + let rect = objects.iter().map(|o| o.rect).reduce(|a, b| [a[0].min(b[0]), a[1].min(b[1]), a[2].max(b[2]), a[3].max(b[3])]).unwrap_or_default(); + Ok(CopiedObjects { resources: used_resources(doc, &res, &drawn), content: serialize_ops(&drawn), rect, objects }) +} + +/// Whether `op` paints (rather than sets state or builds a path). +fn paints(op: &Op) -> bool { + matches!(op.op.as_slice(), b"S" | b"s" | b"f" | b"F" | b"f*" | b"B" | b"B*" | b"b" | b"b*" | b"Tj" | b"TJ" | b"'" | b"\"" | b"Do" | b"sh" | b"BI") +} + +/// The page's operators up to the last of `keep`, painting only `keep` (see the module docs). +/// `shows`: where each text-showing operator starts ([`crate::text::show_starts`]). +fn replay(ops: &[Op], keep: &BTreeSet, shows: &HashMap) -> Result, EditError> { + let Some(&last) = keep.last() else { return Err(EditError::Invalid("the selected objects draw nothing to copy".into())) }; + let mut out = Vec::new(); + let (mut depth, mut in_text) = (0usize, false); + let (mut path, mut clips) = (Vec::new(), false); + for (i, op) in ops.iter().enumerate().take(last.saturating_add(1)) { + let kept = keep.contains(&i); + match op.op.as_slice() { + b"q" => { + if depth >= MAX_NESTING { + return Err(EditError::Invalid("the page graphics state is nested too deeply".into())); + } + depth += 1; + out.push(op.clone()); + } + // An unmatched `Q` restores nothing; the renderer ignores it too. + b"Q" if depth > 0 => { + depth -= 1; + out.push(op.clone()); + } + b"BT" if !in_text => { + in_text = true; + out.push(op.clone()); + } + b"ET" if in_text => { + in_text = false; + out.push(op.clone()); + } + b"Q" | b"BT" | b"ET" => {} + b"m" | b"l" | b"c" | b"v" | b"y" | b"re" | b"h" => path.push(op.clone()), + b"W" | b"W*" => { + clips = true; + path.push(op.clone()); + } + b"n" | b"S" | b"s" | b"f" | b"F" | b"f*" | b"B" | b"B*" | b"b" | b"b*" => { + if kept || clips { + out.append(&mut path); + out.push(if kept { op.clone() } else { Op::new("n", Vec::new()) }); + } + path.clear(); + clips = false; + } + b"Tj" | b"TJ" | b"'" | b"\"" => { + // `"` sets the spacing for what follows, shown or not. + if op.is("\"") { + for (name, value) in ["Tw", "Tc"].into_iter().zip(op.operands.iter().take(2)) { + if value.as_f64().is_some() { + out.push(Op::new(name, vec![value.clone()])); + } + } + } + match shows.get(&i) { + Some((tm, _)) if kept => { + out.push(Op::new("Tm", tm.0.map(Object::Real).to_vec())); + out.push(if op.is("TJ") { op.clone() } else { Op::new("Tj", op.operands.last().cloned().into_iter().collect()) }); + } + // Its glyphs clip what follows, which leaving them out would not. + Some((_, mode)) if *mode >= 4 => { + return Err(EditError::Invalid( + "text used as a clipping path is drawn before the selected objects, so they can't be copied".into(), + )); + } + _ => {} + } + } + // Each copied show sets its own text matrix; `TD` also sets the leading. + b"TD" => { + if let Some([_, y]) = op.nums::<2>() { + out.push(Op::new("TL", vec![Object::Real(-y)])); + } + } + b"Td" | b"Tm" | b"T*" | b"BDC" | b"BMC" | b"EMC" | b"MP" | b"DP" => {} + b"Do" | b"sh" | b"BI" => { + if kept { + out.push(op.clone()); + } + } + _ => out.push(op.clone()), + } + } + if in_text { + out.push(Op::new("ET", Vec::new())); + } + out.extend(std::iter::repeat_with(|| Op::new("Q", Vec::new())).take(depth)); + Ok(out) +} + +/// The entries of the page's resources `res` that `ops` name, and the default colour spaces, +/// which remap the device colours they paint (ISO 32000-2 §8.6.5.6). +fn used_resources(doc: &Document, res: &Dict, ops: &[Op]) -> Dict { + let mut used: BTreeMap<&[u8], BTreeSet<&[u8]>> = BTreeMap::new(); + for op in ops { + if let Ok(Some((cat, i))) = resource_operand(op) + && let Some(Object::Name(name)) = op.operands.get(i) + { + used.entry(cat).or_default().insert(name.as_slice()); + } + } + let mut out = Dict::new(); + for cat in CATEGORIES { + let Some(all) = res.get(cat).map(|d| doc.resolve(d)).and_then(|d| d.as_dict().cloned()) else { continue }; + let mut mine = Dict::new(); + for (key, value) in all.iter() { + if used.get(cat).is_some_and(|u| u.contains(key.as_slice())) || (cat == b"ColorSpace" && DEFAULT_SPACES.contains(&key.as_slice())) { + mine.set(key.clone(), value.clone()); + } + } + if !mine.is_empty() { + out.set(cat.to_vec(), Object::Dict(mine)); + } + } + out +} + +/// Paste `copied` on top of `page`, moved by `offset` (user space), as one change: its content +/// and the page's resources merged with its own. Its resources must be this document's. +pub fn paste_objects(doc: &mut Document, page: usize, copied: &CopiedObjects, offset: [f64; 2]) -> Result<(), EditError> { + let r = copied.rect; + if !offset.iter().all(|v| v.is_finite() && v.abs() <= MAX_COORDINATE) + || !valid_rect([r[0] + offset[0], r[1] + offset[1], r[2] + offset[0], r[3] + offset[1]]) + { + return Err(EditError::Invalid("the paste position is out of range".into())); + } + let p = pdfcraft_model::pages(doc).into_iter().nth(page).ok_or(EditError::NoSuchPage(page))?; + crate::check_on_top(doc, &p)?; + let res = p.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default(); + let mut xobjects = res.get(b"XObject").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()).unwrap_or_default(); + let name = (1..=xobjects.len().saturating_add(1)) + .map(|n| format!("PCPaste{n}").into_bytes()) + .find(|n| xobjects.get(n).is_none()) + .ok_or_else(|| EditError::Invalid("the page has too many resources".into()))?; + // The copy as Form artwork, to draw inline as Ungroup does: its resources merge into the + // page's, and patterns move with it. The form itself is never added to the document. + let number = |v: f64| Object::Real(v); + let mut form = Dict::new(); + form.set(b"Type".to_vec(), Object::name("XObject")); + form.set(b"Subtype".to_vec(), Object::name("Form")); + // Nothing of the copy is clipped: its glyphs and strokes may reach past its objects' boxes. + form.set(b"BBox".to_vec(), Object::Array([-MAX_COORDINATE, -MAX_COORDINATE, MAX_COORDINATE, MAX_COORDINATE].map(number).to_vec())); + form.set(b"Matrix".to_vec(), Object::Array(Matrix::translate(offset[0], offset[1]).0.map(number).to_vec())); + form.set(b"Resources".to_vec(), Object::Dict(copied.resources.clone())); + xobjects.set(name.clone(), Object::Stream(Stream::from_raw(form, copied.content.clone()))); + let mut with_copy = res; + with_copy.set(b"XObject".to_vec(), Object::Dict(xobjects)); + let space = page_blend_space(doc, &p.dict); + // Drawn by a `Do` of its own: not translucent, and in no path or text object. + let draw = [Op::new("Do", vec![Object::Name(name.clone())])]; + let mut budget = pdfcraft_cos::DecodeBudget::new(MAX_FORM_BYTES, 0); + let Ungrouped { ops, resources: mut merged, patterns, .. } = + ungroup_ops(doc, &with_copy, &draw, 0, space.as_ref(), Matrix::translate(offset[0], offset[1]), &mut budget).map_err(|r| match r.error { + EditError::Invalid(why) => { + EditError::Invalid(why.replacen("this artwork can't be ungrouped", "the copied objects can't be pasted here", 1)) + } + other => other, + })?; + let mut work = doc.clone(); + if let Some(mut x) = merged.get(b"XObject").and_then(Object::as_dict).cloned() { + x.remove(&name); + merged.set(b"XObject".to_vec(), Object::Dict(x)); + } + if !patterns.is_empty() { + let mut table = merged.get(b"Pattern").map(|x| work.resolve(x)).and_then(|x| x.as_dict().cloned()).unwrap_or_default(); + for (key, pattern) in patterns { + table.set(key, Object::Ref(work.add(pattern))); + } + merged.set(b"Pattern".to_vec(), Object::Dict(table)); + } + work.update_dict(p.obj, |d| d.set(b"Resources".to_vec(), Object::Dict(merged)))?; + crate::place_tagged(&mut work, &p, "Pasted", serialize_ops(&ops), false)?; + // Keep the page's objects selectable together, as a move does. + editable_objects(&work, page)?; + *doc = work; + Ok(()) +} + +#[cfg(test)] +#[path = "clip_tests.rs"] +mod tests; diff --git a/crates/edit/src/objects/clip_tests.rs b/crates/edit/src/objects/clip_tests.rs new file mode 100644 index 0000000000..980bb9ba4c --- /dev/null +++ b/crates/edit/src/objects/clip_tests.rs @@ -0,0 +1,228 @@ +use std::sync::Arc; + +use pdfcraft_cos::{Document, SaveOptions, write_incremental}; + +use super::*; + +/// A document of `objs` (object 1 the catalog), numbered in order. +fn pdf(objs: &[String]) -> Document { + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offs = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offs.push(out.len()); + out.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes()); + } + let x = out.len(); + out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offs { + out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{x}\n%%EOF\n", objs.len() + 1).as_bytes()); + Document::open(Arc::new(out)).unwrap() +} + +fn stream(content: &str) -> String { + format!("<< /Length {} >>\nstream\n{content}\nendstream", content.len()) +} + +/// Two 300 × 300 pages. The first draws `source` with Helvetica as /F1, a 4 × 2 RGB image /Im0, +/// a half-opaque ExtGState /GS0 and Courier as /F2; the second draws `target` with Courier +/// under the same name /F1, so a paste from the first must rename the font it brings. +fn two_pages(source: &str, target: &str) -> Document { + pdf(&[ + "<< /Type /Catalog /Pages 2 0 R >>".into(), + "<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 /MediaBox [0 0 300 300] >>".into(), + "<< /Type /Page /Parent 2 0 R /Contents 5 0 R /Resources << /Font << /F1 7 0 R /F2 8 0 R >> /XObject << /Im0 9 0 R >> /ExtGState << /GS0 << /ca 0.5 /CA 0.5 >> >> >> >>" + .into(), + "<< /Type /Page /Parent 2 0 R /Contents 6 0 R /Resources << /Font << /F1 8 0 R >> >> >>".into(), + stream(source), + stream(target), + "<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>".into(), + "<< /Type /Font /Subtype /Type1 /BaseFont /Courier /Encoding /WinAnsiEncoding >>".into(), + "<< /Type /XObject /Subtype /Image /Width 4 /Height 2 /ColorSpace /DeviceRGB /BitsPerComponent 8 /Length 24 >>\nstream\n\u{7f}\u{0}\u{0}\u{0}\u{7f}\u{0}\u{0}\u{0}\u{7f}\u{7f}\u{7f}\u{0}\u{0}\u{7f}\u{7f}\u{7f}\u{0}\u{7f}@@@\u{0}\u{0}\u{0}\nendstream".into(), + ]) +} + +/// What the page state, the copied paragraph and picture and the objects left out are drawn with. +const SOURCE: &str = "q 1 0 0 1 10 10 cm 0 0 270 270 re W n 0 0 1 rg /GS0 gs \ + BT /F2 10 Tf 200 20 Td (Skip) Tj ET \ + BT /F1 24 Tf 1.5 Tc 20 170 Td (Copied) Tj 0 -30 Td (Too) Tj ET \ + q 1 0 0 rg 150 230 100 40 re f Q \ + q 80 0 0 40 20 60 cm /Im0 Do Q Q \ + BT /F1 12 Tf 200 250 Td (After) Tj ET"; + +fn target(doc: &Document, page: usize, want: impl Fn(&EditableObject) -> bool) -> ObjectTarget { + editable_objects(doc, page).unwrap().into_iter().find(|o| want(o)).unwrap().target +} + +fn render(doc: &Document, page: usize) -> pdfcraft_render::RenderedPage { + let bytes = write_incremental(doc, &SaveOptions::default()).unwrap(); + let mut renderer = pdfcraft_render::PageRenderer::new(Arc::new(bytes), pdfcraft_render::RenderConfig::default()); + renderer.render(pdfcraft_render::RenderRequest { page, kind: pdfcraft_render::RequestKind::Pixels, scale: 1.0, ..Default::default() }) +} + +/// The pixels of `page` in user-space box `r` (a page 300 high at scale 1), shifted by `by`. +fn pixels(page: &pdfcraft_render::RenderedPage, r: [f64; 4], by: [f64; 2]) -> Vec { + let (x0, x1) = ((r[0] + by[0]) as u32, (r[2] + by[0]) as u32); + let (y0, y1) = ((300.0 - r[3] - by[1]) as u32, (300.0 - r[1] - by[1]) as u32); + let mut out = Vec::new(); + for y in y0..y1 { + for x in x0..x1 { + let i = ((y * page.width + x) * 4) as usize; + out.extend_from_slice(&page.rgba[i..i + 4]); + } + } + out +} + +/// The paragraph "Copied" / "Too" (one or two of them, as the lines group) and the picture. +fn copied_paragraph_and_picture(doc: &Document) -> CopiedObjects { + let targets: Vec<_> = editable_objects(doc, 0) + .unwrap() + .into_iter() + .filter(|o| o.target.kind == ObjectKind::Image || o.text.as_deref().is_some_and(|t| t.contains("Copied") || t.contains("Too"))) + .map(|o| o.target) + .collect(); + copy_objects(doc, 0, &targets).unwrap() +} + +#[test] +fn a_copy_takes_the_selected_objects_their_state_and_only_the_resources_they_use() { + let doc = two_pages(SOURCE, ""); + let copied = copied_paragraph_and_picture(&doc); + let content = String::from_utf8_lossy(&copied.content).into_owned(); + assert!(content.contains("(Copied)") && content.contains("(Too)") && content.contains("/Im0 Do"), "{content}"); + // The page's clip and its colour and transparency still apply; nothing else is painted. + assert!(content.contains("W\nn") && content.contains("/GS0 gs") && content.contains("rg"), "{content}"); + assert!(!content.contains("Skip") && !content.contains("After") && !content.contains("\nf\n"), "{content}"); + let names = |cat: &[u8]| copied.resources.get(cat).and_then(Object::as_dict).map(|d| d.iter().map(|(k, _)| k.clone()).collect::>()); + assert_eq!(names(b"Font"), Some(vec![b"F1".to_vec(), b"F2".to_vec()]), "the font of the skipped text stays selected for what follows"); + assert_eq!(names(b"XObject"), Some(vec![b"Im0".to_vec()])); + assert_eq!(names(b"ExtGState"), Some(vec![b"GS0".to_vec()])); + assert!(copied.text().contains("Copied") && copied.text().contains("Too") && !copied.text().contains("Skip"), "{}", copied.text()); + assert_eq!(pdfcraft_content::parse(&copied.content).ops.iter().filter(|o| o.is("q")).count(), 3, "balanced: {content}"); +} + +#[test] +fn pasted_objects_draw_the_same_on_another_page_under_renamed_resources() { + let mut doc = two_pages(SOURCE, "BT /F1 10 Tf 5 5 Td (x) Tj ET"); + let copied = copied_paragraph_and_picture(&doc); + paste_objects(&mut doc, 1, &copied, [0.0, 0.0]).unwrap(); + let (source, pasted) = (render(&doc, 0), render(&doc, 1)); + let area = [copied.rect[0] - 4.0, copied.rect[1] - 4.0, copied.rect[2] + 4.0, copied.rect[3] + 4.0]; + let (before, after) = (pixels(&source, area, [0.0, 0.0]), pixels(&pasted, area, [0.0, 0.0])); + assert!(before.chunks(4).any(|p| p != [255, 255, 255, 255]), "the copied objects are drawn"); + assert_eq!(before, after, "the paste draws the same pixels"); + let red = pixels(&pasted, [160.0, 240.0, 260.0, 280.0], [0.0, 0.0]); + assert!(red.chunks(4).all(|p| p == [255, 255, 255, 255]), "what wasn't selected isn't pasted"); + // Its own /F1 (Courier) is left alone; the copy's Helvetica comes under another name. + let fonts = + pdfcraft_model::pages(&doc)[1].dict.get(b"Resources").map(|r| doc.resolve(r)).unwrap().as_dict().unwrap().get(b"Font").cloned().unwrap(); + let fonts = doc.resolve(&fonts).as_dict().cloned().unwrap(); + assert_eq!(fonts.get(b"F1"), Some(&Object::Ref(pdfcraft_cos::ObjRef::new(8, 0)))); + assert!(fonts.iter().any(|(k, v)| k != b"F1" && *v == Object::Ref(pdfcraft_cos::ObjRef::new(7, 0))), "{fonts:?}"); + let texts: Vec<_> = editable_objects(&doc, 1).unwrap().into_iter().filter_map(|o| o.text).collect(); + assert!(texts.iter().any(|t| t.contains("Copied")) && !texts.iter().any(|t| t.contains("Skip")), "{texts:?}"); + assert_eq!(crate::images::page_images(&doc, 1).unwrap().len(), 1); +} + +#[test] +fn a_paste_onto_its_own_page_moves_by_the_offset_and_draws_the_same() { + let mut doc = two_pages(SOURCE, ""); + let copied = copied_paragraph_and_picture(&doc); + let before = render(&doc, 0); + paste_objects(&mut doc, 0, &copied, [150.0, 0.0]).unwrap(); + let after = render(&doc, 0); + let area = [copied.rect[0] - 2.0, copied.rect[1] - 2.0, copied.rect[2] + 2.0, copied.rect[3] + 2.0]; + assert!(area[2] + 150.0 < 300.0, "{area:?}"); + // The copy (and the clip it is drawn in) lands 150 pt to the right, the original stays. + // Antialiasing elsewhere on the page may round an edge pixel one step differently. + let (moved, original) = (pixels(&after, area, [150.0, 0.0]), pixels(&before, area, [0.0, 0.0])); + assert_eq!(moved.len(), original.len()); + assert!(original.iter().any(|v| *v != 255), "the copied objects are drawn"); + assert!(moved.iter().zip(&original).all(|(a, b)| a.abs_diff(*b) <= 2), "the paste draws the same pixels, moved"); + assert_eq!(pixels(&after, area, [0.0, 0.0]), original); + // Within the document and the page, the resources are the page's own, under their names. + let res = pdfcraft_model::pages(&doc)[0].dict.get(b"Resources").map(|r| doc.resolve(r)).unwrap(); + let fonts = res.as_dict().unwrap().get(b"Font").and_then(Object::as_dict).cloned().unwrap(); + assert_eq!(fonts.len(), 2, "{fonts:?}"); + assert_eq!(editable_objects(&doc, 0).unwrap().iter().filter(|o| o.target.kind == ObjectKind::Image).count(), 2); +} + +/// Artwork without resources of its own takes the names of the page drawing it. Pasted onto a +/// page whose names differ, artwork that names nothing still draws the same; artwork that names +/// a font would find another one, so the paste is refused. +#[test] +fn artwork_inheriting_resources_pastes_only_when_it_names_none() { + for (art, pastes) in [("0 0 1 rg 0 0 60 30 re f", true), ("BT /F1 9 Tf (n) Tj ET", false)] { + let mut doc = pdf(&[ + "<< /Type /Catalog /Pages 2 0 R >>".into(), + "<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 /MediaBox [0 0 300 300] >>".into(), + "<< /Type /Page /Parent 2 0 R /Contents 5 0 R /Resources << /Font << /F1 7 0 R >> /XObject << /Art 8 0 R >> >> >>".into(), + "<< /Type /Page /Parent 2 0 R /Contents 6 0 R /Resources << /Font << /F1 9 0 R >> >> >>".into(), + stream("BT /F1 12 Tf 20 250 Td (Title) Tj ET q 1 0 0 1 20 100 cm /Art Do Q"), + stream(""), + "<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>".into(), + format!("<< /Type /XObject /Subtype /Form /BBox [0 0 60 30] /Length {} >>\nstream\n{art}\nendstream", art.len()), + "<< /Type /Font /Subtype /Type1 /BaseFont /Courier >>".into(), + ]); + let all: Vec<_> = editable_objects(&doc, 0).unwrap().into_iter().map(|o| o.target).collect(); + let copied = copy_objects(&doc, 0, &all).unwrap(); + let pasted = paste_objects(&mut doc, 1, &copied, [0.0, 0.0]); + assert_eq!(pasted.is_ok(), pastes, "{art}: {pasted:?}"); + if let Err(EditError::Invalid(why)) = pasted { + assert!(why.starts_with("the copied objects can't be pasted here") && why.contains("uses its resources"), "{why}"); + } + } +} + +#[test] +fn added_content_is_copied_with_everything_its_stream_paints() { + let mut doc = two_pages(SOURCE, ""); + let item = crate::Content::Text(crate::AddedText { + rect: [20.0, 20.0, 200.0, 60.0], + text: "Added box".into(), + size: 12.0, + ..crate::AddedText::default() + }); + crate::add_content(&mut doc, 0, &item).unwrap(); + let added = target(&doc, 0, |o| o.target.kind == ObjectKind::Added); + let copied = copy_objects(&doc, 0, &[added]).unwrap(); + assert_eq!(copied.text(), "Added box"); + paste_objects(&mut doc, 1, &copied, [0.0, 0.0]).unwrap(); + let texts: Vec<_> = editable_objects(&doc, 1).unwrap().into_iter().filter_map(|o| o.text).collect(); + assert_eq!(texts, ["Added box"], "pasted as the page's own text"); + assert!(crate::list_added(&doc).iter().all(|a| a.page == 0)); +} + +#[test] +fn copying_and_pasting_refuse_what_they_cannot_do_and_change_nothing() { + let doc = two_pages(SOURCE, ""); + let text = target(&doc, 0, |o| o.text.as_deref().is_some_and(|t| t.contains("Copied"))); + assert!(copy_objects(&doc, 0, &[]).is_err()); + assert!(copy_objects(&doc, 0, &[text, text]).is_err(), "selected twice"); + assert!(copy_objects(&doc, 0, &[ObjectTarget { kind: ObjectKind::Image, index: 9 }]).is_err(), "stale"); + assert!(copy_objects(&doc, 5, &[text]).is_err(), "no such page"); + let copied = copy_objects(&doc, 0, &[text]).unwrap(); + for offset in [[f64::NAN, 0.0], [f64::INFINITY, 0.0], [2e9, 0.0]] { + let mut work = doc.clone(); + assert!(paste_objects(&mut work, 1, &copied, offset).is_err(), "{offset:?}"); + assert!(!work.is_modified(), "a refused paste changes nothing"); + } + assert!(paste_objects(&mut doc.clone(), 7, &copied, [0.0, 0.0]).is_err()); + + // Text drawn as a clipping path before the selection would clip it: refused, not dropped. + let clipped = two_pages("BT /F1 20 Tf 7 Tr 20 20 Td (Mask) Tj ET 0 Tr BT /F1 20 Tf 20 200 Td (Shown) Tj ET", ""); + let shown = target(&clipped, 0, |o| o.text.as_deref() == Some("Shown")); + assert!(copy_objects(&clipped, 0, &[shown]).is_err()); + + // Stray `Q`, `ET` and an unfinished path or text object: the copy is still balanced. + let stray = two_pages("Q Q ET 10 10 m 20 20 l BT /F1 20 Tf 20 200 Td (Stray) Tj", ""); + let shown = target(&stray, 0, |o| o.text.as_deref() == Some("Stray")); + let copied = copy_objects(&stray, 0, &[shown]).unwrap(); + let ops = pdfcraft_content::parse(&copied.content).ops; + assert_eq!(ops.iter().filter(|o| o.is("q")).count(), ops.iter().filter(|o| o.is("Q")).count()); + assert_eq!(ops.iter().filter(|o| o.is("BT")).count(), ops.iter().filter(|o| o.is("ET")).count()); + let mut work = stray.clone(); + paste_objects(&mut work, 1, &copied, [0.0, 0.0]).unwrap(); +} diff --git a/crates/edit/src/page_streams.rs b/crates/edit/src/page_streams.rs new file mode 100644 index 0000000000..eae00399da --- /dev/null +++ b/crates/edit/src/page_streams.rs @@ -0,0 +1,69 @@ +//! A page's content streams rewritten one at a time: each stream keeps its own dictionary +//! (and so its marks, such as `/PCMark`), and only the streams that changed are replaced. + +use pdfcraft_cos::{Dict, Document, ObjRef, Object, Stream}; + +use crate::EditError; + +/// The decoded content streams of one page, in drawing order. +pub(crate) struct PageStreams { + /// Each stream's object and dictionary. + entries: Vec<(ObjRef, Dict)>, + /// Each stream's decoded data, in the same order. + pub(crate) data: Vec>, +} + +impl PageStreams { + /// The streams of `page_dict`'s `/Contents`. `None` when one of them is not a stream or + /// cannot be decoded: rewriting the others would draw the page differently, so the page is + /// left alone. A page without content has no streams. + pub(crate) fn read(doc: &Document, page_dict: &Dict) -> Option { + let refs: Vec = match page_dict.get(b"Contents") { + None => Vec::new(), + Some(c) => match &*doc.resolve(c) { + Object::Array(a) => a.clone(), + _ => vec![c.clone()], + }, + }; + let mut entries = Vec::with_capacity(refs.len()); + let mut data = Vec::with_capacity(refs.len()); + for e in &refs { + let r = e.as_ref()?; + let (dict, bytes) = match &*doc.get(r) { + Object::Stream(s) => (s.dict.clone(), s.decoded().ok()?), + _ => return None, + }; + data.push(bytes); + entries.push((r, dict)); + } + Some(Self { entries, data }) + } + + /// Put `new` in place of the streams that changed (`None` keeps a stream as it is). Each + /// replacement is a new object with the old dictionary (minus its encoding), so a stream + /// another page shares is not touched. Returns whether anything changed. + pub(crate) fn write(self, doc: &mut Document, page: ObjRef, new: Vec>>) -> Result { + if new.iter().all(Option::is_none) { + return Ok(false); + } + let mut refs: Vec = Vec::with_capacity(self.entries.len()); + for (i, (r, dict)) in self.entries.into_iter().enumerate() { + match new.get(i).and_then(Option::as_ref) { + Some(bytes) => { + let mut dict = dict; + for k in [&b"Length"[..], b"Filter", b"DecodeParms", b"DL"] { + dict.remove(k); + } + refs.push(Object::Ref(doc.add(Object::Stream(Stream::flate(dict, bytes))))); + } + None => refs.push(Object::Ref(r)), + } + } + let contents = match <[Object; 1]>::try_from(refs) { + Ok([one]) => one, + Err(many) => Object::Array(many), + }; + doc.update_dict(page, |d| d.set(b"Contents".to_vec(), contents))?; + Ok(true) + } +} diff --git a/crates/edit/src/place.rs b/crates/edit/src/place.rs index 6460aab066..ae2faf8386 100644 --- a/crates/edit/src/place.rs +++ b/crates/edit/src/place.rs @@ -1,5 +1,5 @@ -use super::marks::{MarkKind, matrix_number, n}; -use super::{Dict, Document, EditError, ObjRef, Object, Stream, literal, win_ansi}; +use super::marks::{MarkKind, matrix_number}; +use super::{Dict, Document, EditError, ObjRef, Object, Stream}; pub(crate) fn page_list(doc: &Document) -> Vec { pdfcraft_model::pages(doc) @@ -68,6 +68,19 @@ pub(crate) fn free_font_name(doc: &Document, fonts: &Dict, preferred: &str, base (1..=fonts.len().saturating_add(1)).map(|k| format!("{preferred}_{k}")).find(|name| usable(name)).unwrap_or_else(|| preferred.to_string()) } +/// Add `fonts` (resource name, font) to the page's own resources. +pub(crate) fn add_page_fonts(doc: &mut Document, page: &pdfcraft_model::Page, fonts: &Dict) -> Result<(), EditError> { + let mut res = page.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default(); + let mut all = res.get(b"Font").map(|f| doc.resolve(f)).and_then(|f| f.as_dict().cloned()).unwrap_or_default(); + for (name, font) in fonts.iter() { + all.set(name.clone(), font.clone()); + } + res.set(b"Font".to_vec(), Object::Dict(all)); + // The page gets its own (possibly copied) resources; shared dictionaries stay untouched. + doc.update_dict(page.obj, |d| d.set(b"Resources".to_vec(), Object::Dict(res)))?; + Ok(()) +} + /// Add the standard `faces` (resource name, base font) and an opacity state to the page's own /// resources. pub(crate) fn add_page_resources( @@ -575,10 +588,3 @@ pub(crate) fn undrawable(c: char) -> EditError { pub fn undrawable_message(c: char) -> String { format!("the standard fonts can't draw \"{c}\" (U+{:04X}); only Western European characters can be added as text for now", u32::from(c)) } - -pub(crate) fn text_op(x: f64, y: f64, text: &str) -> Vec { - let mut v = format!("1 0 0 1 {} {} Tm ", n(x), n(y)).into_bytes(); - v.extend(literal(&win_ansi(text))); - v.extend_from_slice(b" Tj\n"); - v -} diff --git a/crates/edit/src/printer_marks.rs b/crates/edit/src/printer_marks.rs new file mode 100644 index 0000000000..a247f65d30 --- /dev/null +++ b/crates/edit/src/printer_marks.rs @@ -0,0 +1,340 @@ +//! Printer marks (Acrobat's Print production ▸ Add Printer Marks): trim and +//! bleed marks, registration targets, colour bars and page information drawn outside the trim +//! box, as a page mark like a header, so they can be removed or replaced. The page is widened +//! when asked and the marks would otherwise fall outside it; the trim box stays where it is. +//! +//! Lines, targets and the information line are in registration colour, the `/All` Separation +//! (ISO 32000-2, 8.6.6.4), so they print on every plate, spot plates too; viewers show its +//! four-ink alternate. The colour bars are process inks and stay DeviceCMYK. + +use pdfcraft_cos::{Dict, Document, Object}; +use pdfcraft_fonts::{literal, win_ansi}; + +use crate::marks::{MarkKind, n}; +use crate::place::{END, add_resources, begin, check, page_list, place}; +use crate::{EditError, remove_marks}; + +/// Which marks, and how heavy. +#[derive(Clone, Debug, PartialEq)] +pub struct PrinterMarks { + pub trim: bool, + pub bleed: bool, + pub registration: bool, + pub color_bars: bool, + pub page_info: bool, + /// Line weight in points (Acrobat: 0.25, 0.5, 1). + pub line_weight: f64, + /// Widen the page by this many points on each side (at least what the marks need) when + /// they would fall outside it; `None` leaves the page and clips the marks. + pub make_room: Option, + /// What the page information line says before the page number (the file's name). + pub label: String, + /// The date on the page information line. + pub date: String, +} + +impl Default for PrinterMarks { + fn default() -> Self { + Self { + trim: true, + bleed: true, + registration: true, + color_bars: true, + page_info: true, + line_weight: 0.25, + make_room: Some(36.0), + label: String::new(), + date: String::new(), + } + } +} + +/// How far a mark reaches beyond the trim box: a gap, then the mark itself. +const GAP: f64 = 6.0; +const LENGTH: f64 = 18.0; +/// Registration targets sit this far beyond the trim box, centred on each side. +const TARGET_OFFSET: f64 = 16.0; +const TARGET_RADIUS: f64 = 4.0; +const BAR: f64 = 10.0; +const INFO_SIZE: f64 = 7.0; +/// The colour space resource holding registration colour. +const REGISTRATION: &str = "PCReg"; +/// The largest page side PDF viewers take (Annex C of ISO 32000-1 lists it as Acrobat's limit). +const MAX_PAGE_SIDE: f64 = 14_400.0; + +/// `[/Separation /All /DeviceCMYK ]`: full tint is all four process inks. +fn registration() -> Object { + let reals = |v: &[f64]| Object::Array(v.iter().map(|x| Object::Real(*x)).collect()); + let mut tint = Dict::new(); + tint.set(b"FunctionType".to_vec(), Object::Int(2)); + tint.set(b"Domain".to_vec(), reals(&[0.0, 1.0])); + tint.set(b"C0".to_vec(), reals(&[0.0; 4])); + tint.set(b"C1".to_vec(), reals(&[1.0; 4])); + tint.set(b"N".to_vec(), Object::Int(1)); + Object::Array(vec![Object::name("Separation"), Object::name("All"), Object::name("DeviceCMYK"), Object::Dict(tint)]) +} + +impl PrinterMarks { + /// The line weights a run accepts, in points. + pub const WEIGHTS: std::ops::RangeInclusive = 0.1..=4.0; + /// The most a run widens a page by on each side at the caller's asking, in points. + pub const MAX_ROOM: f64 = 288.0; + + fn any(&self) -> bool { + self.trim || self.bleed || self.registration || self.color_bars || self.page_info + } + + /// How far beyond the trim box the chosen marks reach. + pub fn reach(&self) -> f64 { + let mut r: f64 = 0.0; + if self.trim || self.bleed { + r = r.max(GAP + LENGTH); + } + if self.registration { + r = r.max(TARGET_OFFSET + TARGET_RADIUS + 2.0); + } + if self.color_bars || self.page_info { + r = r.max(GAP + BAR + 6.0); + } + r + } + + /// The marks' content for a page whose trim box is `trim` and bleed box `bleed` (user + /// space), `page` of `count` (1-based). + pub fn content(&self, trim: [f64; 4], bleed: [f64; 4], page: usize, count: usize) -> String { + let w = self.line_weight.clamp(*Self::WEIGHTS.start(), *Self::WEIGHTS.end()); + // Registration colour: every plate. + let mut c = format!("/{REGISTRATION} CS 1 SCN /{REGISTRATION} cs 1 scn {} w 0 J\n", n(w)); + let [x0, y0, x1, y1] = trim; + let line = |c: &mut String, ax: f64, ay: f64, bx: f64, by: f64| { + c.push_str(&format!("{} {} m {} {} l S\n", n(ax), n(ay), n(bx), n(by))); + }; + // Crop (trim) marks: at each corner, outside the box, along both edges. + let corner_marks = |c: &mut String, b: [f64; 4]| { + let [bx0, by0, bx1, by1] = b; + for (x, dir) in [(bx0, -1.0), (bx1, 1.0)] { + for y in [by0, by1] { + // Vertical mark beyond the top or bottom edge. + let (ya, yb) = if y == by0 { (y - GAP, y - GAP - LENGTH) } else { (y + GAP, y + GAP + LENGTH) }; + line(c, x, ya, x, yb); + // Horizontal mark beyond the left or right edge. + line(c, x + dir * GAP, y, x + dir * (GAP + LENGTH), y); + } + } + }; + if self.trim { + corner_marks(&mut c, trim); + } + if self.bleed && bleed != trim { + c.push_str(&format!("[{} {}] 0 d\n", n(w * 4.0), n(w * 4.0))); + corner_marks(&mut c, bleed); + c.push_str("[] 0 d\n"); + } + if self.registration { + let r = TARGET_RADIUS; + let k = 0.5523 * r; + let target = |c: &mut String, cx: f64, cy: f64| { + c.push_str(&format!( + "{} {} m {} {} {} {} {} {} c {} {} {} {} {} {} c {} {} {} {} {} {} c {} {} {} {} {} {} c S\n", + n(cx + r), + n(cy), + n(cx + r), + n(cy + k), + n(cx + k), + n(cy + r), + n(cx), + n(cy + r), + n(cx - k), + n(cy + r), + n(cx - r), + n(cy + k), + n(cx - r), + n(cy), + n(cx - r), + n(cy - k), + n(cx - k), + n(cy - r), + n(cx), + n(cy - r), + n(cx + k), + n(cy - r), + n(cx + r), + n(cy - k), + n(cx + r), + n(cy) + )); + line(c, cx - r * 1.6, cy, cx + r * 1.6, cy); + line(c, cx, cy - r * 1.6, cx, cy + r * 1.6); + }; + let (mx, my) = ((x0 + x1) / 2.0, (y0 + y1) / 2.0); + target(&mut c, mx, y1 + TARGET_OFFSET); + target(&mut c, mx, y0 - TARGET_OFFSET); + target(&mut c, x0 - TARGET_OFFSET, my); + target(&mut c, x1 + TARGET_OFFSET, my); + } + if self.color_bars { + // Above the top edge, from the left: C, M, Y, K at 100%, then 75, 50, 25% K. + let y = y1 + GAP + 2.0; + let x = x0 + LENGTH + GAP + 12.0; + let swatches: [(f64, f64, f64, f64); 7] = [ + (1.0, 0.0, 0.0, 0.0), + (0.0, 1.0, 0.0, 0.0), + (0.0, 0.0, 1.0, 0.0), + (0.0, 0.0, 0.0, 1.0), + (0.0, 0.0, 0.0, 0.75), + (0.0, 0.0, 0.0, 0.5), + (0.0, 0.0, 0.0, 0.25), + ]; + for (i, (cy, m, yl, k)) in swatches.iter().enumerate() { + c.push_str(&format!("{} {} {} {} k {} {} {} {} re f\n", n(*cy), n(*m), n(*yl), n(*k), n(x + i as f64 * BAR), n(y), n(BAR), n(BAR))); + } + c.push_str(&format!("/{REGISTRATION} cs 1 scn\n")); + } + if self.page_info { + let mut info = String::new(); + if !self.label.trim().is_empty() { + info.push_str(self.label.trim()); + info.push_str(" "); + } + info.push_str(&format!("{page} of {count}")); + if !self.date.trim().is_empty() { + info.push_str(" "); + info.push_str(self.date.trim()); + } + let info: String = info.chars().filter(|ch| pdfcraft_fonts::win_ansi_encodable(*ch)).collect(); + c.push_str(&format!("BT /PCHelv {} Tf 1 0 0 1 {} {} Tm ", n(INFO_SIZE), n(x0 + LENGTH + GAP + 12.0), n(y0 - GAP - BAR))); + c.push_str(&ascii_literal(&literal(&win_ansi(&info)))); + c.push_str(" Tj ET\n"); + } + c + } +} + +/// A PDF literal string made of WinAnsi bytes, as ASCII text: bytes from 0x80 up are written as +/// octal escapes, so they reach the content stream unchanged instead of being replaced when the +/// bytes are turned into a `String`. +fn ascii_literal(bytes: &[u8]) -> String { + let mut out = String::with_capacity(bytes.len()); + for &b in bytes { + if b < 0x80 { + out.push(char::from(b)); + } else { + out.push_str(&format!("\\{b:03o}")); + } + } + out +} + +/// How far the chosen marks reach beyond the trim box on each side (left, bottom, right, top): +/// `reach`, or more where the bleed marks, which start at the bleed box, go further out. +fn side_reach(marks: &PrinterMarks, trim: [f64; 4], bleed: [f64; 4]) -> [f64; 4] { + let base = marks.reach(); + if !(marks.bleed && bleed != trim) { + return [base; 4]; + } + let beyond = |bleed_edge: f64, trim_edge: f64| base.max((trim_edge - bleed_edge).abs() + GAP + LENGTH); + [beyond(bleed[0], trim[0]), beyond(bleed[1], trim[1]), beyond(bleed[2], trim[2]), beyond(bleed[3], trim[3])] +} + +/// Add printer marks to `pages` (0-based). With `replace`, marks already there are removed +/// first. Returns the pages that were widened (media or crop box) to make room. +pub fn add_printer_marks(doc: &mut Document, pages: &[usize], marks: &PrinterMarks, replace: bool) -> Result, EditError> { + let all = page_list(doc); + check(pages, all.len())?; + if !marks.any() { + return Err(EditError::Invalid("choose at least one kind of mark".into())); + } + if !PrinterMarks::WEIGHTS.contains(&marks.line_weight) { + let (low, high) = (PrinterMarks::WEIGHTS.start(), PrinterMarks::WEIGHTS.end()); + return Err(EditError::Invalid(format!("the line weight must be from {low} to {high} points"))); + } + if marks.make_room.is_some_and(|room| !(0.0..=PrinterMarks::MAX_ROOM).contains(&room)) { + return Err(EditError::Invalid(format!("the room to make must be from 0 to {} points", PrinterMarks::MAX_ROOM))); + } + if replace { + remove_marks(doc, pages, MarkKind::PrinterMarks)?; + } + let count = all.len(); + let mut widened = Vec::new(); + for &i in pages { + let [media, crop, bleed, trim, art] = pdfcraft_organize_boxes(doc, i)?; + let side = side_reach(marks, trim, bleed); + let needed = [trim[0] - side[0], trim[1] - side[1], trim[2] + side[2], trim[3] + side[3]]; + let within = |b: [f64; 4]| b[0] <= needed[0] && b[1] <= needed[1] && b[2] >= needed[2] && b[3] >= needed[3]; + // The crop box is what shows, so the marks need room inside it, not only inside the media box. + if !within(crop) + && let Some(room) = marks.make_room + { + let new_media = if within(media) { + None + } else { + let room = |s: f64| room.max(s); + Some([ + media[0].min(trim[0] - room(side[0])), + media[1].min(trim[1] - room(side[1])), + media[2].max(trim[2] + room(side[2])), + media[3].max(trim[3] + room(side[3])), + ]) + }; + if new_media.is_some_and(|m| m[2] - m[0] > MAX_PAGE_SIDE || m[3] - m[1] > MAX_PAGE_SIDE) { + return Err(EditError::Invalid(format!("page {} would be wider than the {MAX_PAGE_SIDE} points a PDF page can be", i + 1))); + } + // Without a wider media box, only the crop box grows, to the marks and no further than the media box. + let new_crop = match new_media { + Some(m) => m, + None => [ + crop[0].min(needed[0]).max(media[0]), + crop[1].min(needed[1]).max(media[1]), + crop[2].max(needed[2]).min(media[2]), + crop[3].max(needed[3]).min(media[3]), + ], + }; + let rect = |r: [f64; 4]| Object::Array(r.iter().map(|v| Object::Real(*v)).collect()); + let page = page_list(doc)[i].clone(); + doc.update_dict(page.obj, |d| { + if let Some(m) = new_media { + d.set(b"MediaBox".to_vec(), rect(m)); + } + // The crop box shows the marks too. The trim, bleed and art boxes default to the + // crop box and are clipped to the media box, so each is written out as it was in + // effect, or it would move with the wider boxes and a second run would widen the + // page again. + d.set(b"CropBox".to_vec(), rect(new_crop)); + for (key, value) in [(&b"BleedBox"[..], bleed), (&b"TrimBox"[..], trim), (&b"ArtBox"[..], art)] { + d.set(key.to_vec(), rect(value)); + } + })?; + widened.push(i); + } + let page = page_list(doc)[i].clone(); + let content = marks.content(trim, bleed, i + 1, count); + let body = format!("{}{}{}", begin(MarkKind::PrinterMarks, "PrinterMarks", [1.0, 0.0, 0.0, 1.0, 0.0, 0.0]), content, END); + add_resources(doc, &page, None, Some(body.as_bytes()))?; + let page = page_list(doc)[i].clone(); + // add_resources gave the page resources of its own. + let mut res = page.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default(); + let mut spaces = res.get(b"ColorSpace").map(|c| doc.resolve(c)).and_then(|c| c.as_dict().cloned()).unwrap_or_default(); + spaces.set(REGISTRATION.as_bytes().to_vec(), registration()); + res.set(b"ColorSpace".to_vec(), Object::Dict(spaces)); + doc.update_dict(page.obj, |d| d.set(b"Resources".to_vec(), Object::Dict(res)))?; + let page = page_list(doc)[i].clone(); + place(doc, &page, MarkKind::PrinterMarks, body.into_bytes(), false)?; + } + Ok(widened) +} + +/// A page's boxes: media, crop, bleed, trim, art (each inherited or falling back as the +/// standard says), without depending on the organize crate. +fn pdfcraft_organize_boxes(doc: &Document, page: usize) -> Result<[[f64; 4]; 5], EditError> { + let p = page_list(doc).get(page).cloned().ok_or_else(|| EditError::Invalid(format!("page {} does not exist", page + 1)))?; + let rect = |k: &[u8]| -> Option<[f64; 4]> { + let a = p.dict.get(k).map(|o| doc.resolve(o))?; + let v: Vec = a.as_array()?.iter().filter_map(|x| doc.resolve(x).as_f64()).collect(); + (v.len() == 4 && v.iter().all(|x| x.is_finite())).then(|| [v[0].min(v[2]), v[1].min(v[3]), v[0].max(v[2]), v[1].max(v[3])]) + }; + let media = rect(b"MediaBox").unwrap_or([0.0, 0.0, 612.0, 792.0]); + let clip = |r: [f64; 4]| [r[0].max(media[0]), r[1].max(media[1]), r[2].min(media[2]), r[3].min(media[3])]; + let crop = rect(b"CropBox").map(clip).unwrap_or(media); + let (bleed, trim, art) = (rect(b"BleedBox").map(clip), rect(b"TrimBox").map(clip), rect(b"ArtBox").map(clip)); + Ok([media, crop, bleed.unwrap_or(crop), trim.unwrap_or(crop), art.unwrap_or(crop)]) +} diff --git a/crates/edit/src/tests.rs b/crates/edit/src/tests.rs index 4b3b6633e4..d56240ee87 100644 --- a/crates/edit/src/tests.rs +++ b/crates/edit/src/tests.rs @@ -549,7 +549,7 @@ fn flattening_keeps_a_no_rotate_widget_upright_on_a_rotated_page() { /// #1317: a cloudy square this builder cannot draw stays on the page. A solid square, a hidden /// cloudy square and a cloudy square that already has an appearance keep their previous results. #[test] -fn flattening_keeps_a_cloudy_square_it_cannot_draw() { +fn flattening_draws_cloudy_squares_and_keeps_one_it_cannot_draw() { let page = stream("", " "); let form = "<< /Type /XObject /Subtype /Form /BBox [100 600 200 650] /Length 28 >>\nstream\n1 0 0 RG 100 600 100 50 re S\nendstream"; let case = |extra: &str, drawn: usize, left: bool| { @@ -571,10 +571,90 @@ fn flattening_keeps_a_cloudy_square_it_cannot_draw() { assert_eq!(streams(&doc, 0), before, "nothing was drawn in place of the comment"); } }; - case("/BE << /S /C /I 2 >>", 0, true); + // A cloudy square is drawn (ISO 32000-2 Table 169); one whose /RD leaves no square inside + // its /Rect (Table 180: the sums "shall be less than" the width and height) cannot be, and stays. + case("/BE << /S /C /I 2 >>", 1, false); + case("/BE << /S /C /I 2 >> /RD [60 30 60 30]", 0, true); case("/BE << /S /S >>", 1, false); - case("/BE << /S /C /I 2 >> /F 2", 0, true); - case("/BE << /S /C /I 2 >> /AP << /N 6 0 R >>", 1, false); + case("/BE << /S /C /I 2 >> /RD [60 30 60 30] /F 2", 0, true); + case("/BE << /S /C /I 2 >> /RD [60 30 60 30] /AP << /N 6 0 R >>", 1, false); +} + +/// #3213: flattening wraps transferable optional-content memberships (`/OC` indirect OCGs and +/// valid OCMDs, including ISO 32000-1 §8.11.2.2 `/VE` expressions within depth 16 and the +/// 4,096-node visit budget) in `/OC /PCFlOC0 BDC … EMC` and registers them in +/// `/Resources /Properties`, while retaining annotations with untransferable, malformed, +/// overly deep, or over-budget `/OC` definitions in `/Annots`. +#[test] +fn flattening_transfers_annotation_optional_content_to_marked_content() { + let page = stream("", " "); + let form = "<< /Type /XObject /Subtype /Form /BBox [100 600 200 650] /Length 28 >>\nstream\n1 0 0 RG 100 600 100 50 re S\nendstream"; + let case = |extra: &str, drawn: usize, left: bool, bdc: bool| { + let annot = format!("<< /Type /Annot /Subtype /Square /Rect [100 600 200 650] /C [1 0 0] /AP << /N 6 0 R >> {extra} >>"); + let mut doc = build(&[ + "<< /Type /Catalog /Pages 2 0 R /OCProperties << /OCGs [7 0 R 8 0 R 13 0 R] /D << /OFF [7 0 R] /ON [8 0 R 13 0 R] >> >> >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>", + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 600 800] /Resources << >> /Contents 5 0 R /Annots [4 0 R] >>", + &annot, + &page, + form, + "<< /Type /OCG /Name (Hidden layer) >>", + "<< /Type /OCG /Name (Shown layer) >>", + "<< /Type /OCMD /OCGs [7 0 R 8 0 R] /P /AllOn >>", + "<< /Type /OCMD /OCGs [7 0 R 8 0 R] /P /AnyOn >>", + "<< /Type /OCMD /OCGs [7 0 R 8 0 R] /P /AnyOff >>", + "<< /Type /OCMD /OCGs [7 0 R 8 0 R] /P /AllOff >>", + "<< /Type /OCG /Name (Second shown layer) >>", + "[ /Or 15 0 R 15 0 R 15 0 R 15 0 R ]", + "[ /Or 16 0 R 16 0 R 16 0 R 16 0 R ]", + "[ /Or 17 0 R 17 0 R 17 0 R 17 0 R ]", + "[ /Or 18 0 R 18 0 R 18 0 R 18 0 R ]", + "[ /Or 8 0 R 8 0 R 8 0 R 8 0 R ]", + ]); + let before = streams(&doc, 0); + let expected_oc = doc.get(pdfcraft_cos::ObjRef::new(4, 0)).as_dict().and_then(|d| d.get(b"OC").cloned()); + assert_eq!(flatten(&mut doc, &[0], true, false).unwrap(), drawn, "{extra}"); + let doc = reopen(&doc); + let p = &pdfcraft_model::pages(&doc)[0]; + assert_eq!(p.dict.contains(b"Annots"), left, "{extra}"); + if drawn == 0 { + assert_eq!(streams(&doc, 0), before, "nothing was drawn in place of the retained comment: {extra}"); + } else { + let flat = streams(&doc, 0).pop().unwrap(); + assert_eq!(flat.contains("/OC /PCFlOC0 BDC") && flat.contains("EMC"), bdc, "{extra}: {flat}"); + if bdc { + let res = doc.resolve(p.dict.get(b"Resources").unwrap()); + let props = doc.resolve(res.as_dict().unwrap().get(b"Properties").unwrap()); + assert_eq!(props.as_dict().unwrap().get(b"PCFlOC0").cloned(), expected_oc, "{extra}"); + } + } + }; + case("/OC 7 0 R", 1, false, true); + case("/OC 8 0 R", 1, false, true); + case("/OC 9 0 R", 1, false, true); + case("/OC 10 0 R", 1, false, true); + case("/OC 11 0 R", 1, false, true); + case("/OC 12 0 R", 1, false, true); + case("", 1, false, false); + // Valid ISO 32000-1 §8.11.2.2 /VE visibility expressions are transferred with their OCMD. + case("/OC << /Type /OCMD /VE [ /Not 7 0 R ] >>", 1, false, true); + case("/OC << /Type /OCMD /VE [ /Not 8 0 R ] >>", 1, false, true); + case("/OC << /Type /OCMD /VE [ /And 8 0 R 7 0 R ] >>", 1, false, true); + case("/OC << /Type /OCMD /VE [ /Or 8 0 R 7 0 R ] >>", 1, false, true); + case("/OC << /Type /OCMD /VE [ /Or 7 0 R [ /And 8 0 R 13 0 R ] ] >>", 1, false, true); + case("/OC << /Type /OCMD /VE [ /And 8 0 R [ /Not 13 0 R ] ] >>", 1, false, true); + case("/OC << /Type /OCMD /VE [ /Not 8 0 R ] /OCGs [8 0 R] /P /AllOn >>", 1, false, true); + case("/OC << /Type /OCMD /VE [ /Not 7 0 R ] /OCGs [7 0 R] /P /AllOn >>", 1, false, true); + // Malformed /VE, invalid /P, inline /OCG, and /VE exceeding depth 16 or the node budget fail closed and stay in /Annots. + case("/OC << /Type /OCG /Name (Inline layer) >>", 0, true, false); + case("/OC << /Type /OCMD /OCGs [8 0 R] /P /InvalidPolicy >>", 0, true, false); + case("/OC << /Type /OCMD /VE 8 0 R /OCGs [8 0 R] /P /AllOn >>", 0, true, false); + case("/OC << /Type /OCMD /VE [] /OCGs [7 0 R] /P /AllOn >>", 0, true, false); + case("/OC << /Type /OCMD /VE [ /Not 7 0 R 8 0 R ] /OCGs [7 0 R] /P /AllOn >>", 0, true, false); + case("/OC << /Type /OCMD /VE [ /Xor 8 0 R ] /OCGs [7 0 R] /P /AllOn >>", 0, true, false); + let deep_ve = format!("{}8 0 R{}", "[ /Not ".repeat(17), " ]".repeat(17)); + case(&format!("/OC << /Type /OCMD /VE {deep_ve} /OCGs [8 0 R] /P /AllOn >>"), 0, true, false); + case("/OC << /Type /OCMD /VE [ /Or 14 0 R 14 0 R 14 0 R 14 0 R ] /OCGs [8 0 R] /P /AllOn >>", 0, true, false); } /// #805: a supported note with no stored appearance is still drawn by viewers. Flattening must @@ -725,10 +805,12 @@ fn flattening_keeps_a_widget_whose_value_is_past_the_parent_limit() { assert_eq!(annots, Some(1), "the widget stays"); } -/// #1794: a reply this builder cannot draw stays on the page even when the comment it replies to -/// is flattened; the comment's pop-up still goes with it. +/// #1794, #5631: a reply this builder cannot draw goes with the comment it replies to, as its +/// pop-up does. A reply of type R is not shown on the page (ISO 32000-2 Table 172), so it has no +/// ink of its own to keep; before #5631 it stayed, the orphan of a comment that was gone. (A +/// `/RT /Group` member that cannot be drawn still stays: it is drawn on the page.) #[test] -fn flattening_keeps_an_undrawn_reply_to_a_flattened_comment() { +fn flattening_takes_an_undrawable_reply_with_its_comment() { let page = stream("", " "); let mut doc = build(&[ "<< /Type /Catalog /Pages 2 0 R >>", @@ -736,15 +818,212 @@ fn flattening_keeps_an_undrawn_reply_to_a_flattened_comment() { "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 600 800] /Resources << >> /Contents 4 0 R /Annots [5 0 R 6 0 R 7 0 R] >>", &page, "<< /Type /Annot /Subtype /Square /Rect [100 600 200 650] /C [1 0 0] /Contents (Synthetic comment) /AP << /N 8 0 R >> /Popup 7 0 R >>", - "<< /Type /Annot /Subtype /Square /Rect [100 500 200 550] /C [1 0 0] /Contents (Synthetic reply) /BE << /S /C /I 2 >> /IRT 5 0 R >>", + "<< /Type /Annot /Subtype /Square /Rect [100 500 200 550] /C [1 0 0] /Contents (Synthetic reply) /BE << /S /C /I 2 >> /RD [60 30 60 30] /IRT 5 0 R >>", "<< /Type /Annot /Subtype /Popup /Rect [300 600 400 700] /Parent 5 0 R >>", "<< /Type /XObject /Subtype /Form /BBox [100 600 200 650] /Length 28 >>\nstream\n1 0 0 RG 100 600 100 50 re S\nendstream", ]); - assert_eq!(flatten(&mut doc, &[0], true, false).unwrap(), 1, "the comment is drawn; the cloudy reply cannot be"); + assert_eq!(flatten(&mut doc, &[0], true, false).unwrap(), 1, "the comment is drawn; the reply is not"); + let doc = reopen(&doc); + let p = &pdfcraft_model::pages(&doc)[0]; + assert!(!p.dict.contains(b"Annots"), "the comment, its reply and its pop-up are gone"); +} + +/// #5631: flattening draws no reply of type R into the page, whatever its appearance, and builds +/// none for one without (a note reply would get an icon); every reply goes with its comment, a +/// reply to a reply too, even listed before the reply it answers. A `/RT /Group` member is +/// drawn as any annotation. +#[test] +fn flattening_draws_no_reply_and_takes_the_thread_with_its_comment() { + let page = stream("", " "); + let form = |rgb: &str| stream("/Type /XObject /Subtype /Form /BBox [0 0 20 18]", &format!("{rgb} rg 0 0 20 18 re f")); + let (blue, red, green) = (form("0 0 1"), form("1 0 0"), form("0 1 0")); + let mut doc = build(&[ + "<< /Type /Catalog /Pages 2 0 R >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>", + // The reply to the reply (8) is listed before the reply it answers (6). + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 600 800] /Resources << >> /Contents 4 0 R /Annots [5 0 R 8 0 R 6 0 R 7 0 R 9 0 R] >>", + &page, + // The comment (blue), its reply (red), its pop-up, a reply to the reply with no + // appearance, and a member of its group (green). + "<< /Type /Annot /Subtype /Text /F 28 /Rect [100 500 120 518] /Contents (Comment) /AP << /N 10 0 R >> /Popup 7 0 R >>", + "<< /Type /Annot /Subtype /Text /F 28 /Rect [105 495 125 513] /Contents (Reply) /IRT 5 0 R /AP << /N 11 0 R >> >>", + "<< /Type /Annot /Subtype /Popup /Rect [300 500 400 600] /Parent 5 0 R >>", + "<< /Type /Annot /Subtype /Text /F 28 /Rect [110 490 130 508] /Contents (Reply to the reply) /IRT 6 0 R /RT /R >>", + "<< /Type /Annot /Subtype /Square /F 4 /Rect [200 500 220 518] /IRT 5 0 R /RT /Group /AP << /N 12 0 R >> >>", + &blue, + &red, + &green, + ]); + let n = flatten(&mut doc, &[0], true, false).unwrap(); + assert_eq!(n, 2, "the comment and the group member are drawn; the replies are not"); + let doc = reopen(&doc); + let p = &pdfcraft_model::pages(&doc)[0]; + assert!(!p.dict.contains(b"Annots"), "the whole thread, its pop-up and the group member are gone"); + let res = doc.resolve(p.dict.get(b"Resources").unwrap()); + let xo = doc.resolve(res.as_dict().unwrap().get(b"XObject").unwrap()); + let mut drawn: Vec = xo.as_dict().unwrap().iter().filter_map(|(_, v)| v.as_ref()).map(|r| r.num).collect(); + drawn.sort_unstable(); + assert_eq!(drawn, [10, 12], "the comment's and the member's appearances; not the reply's, and none built for the other"); +} + +/// #5632 review: a reply's text never blocks flattening (a reply is not drawn into the page, so +/// it can't bake in a `?`), and a `/RT /Group` member that flattening skipped (Hidden here) +/// stays when its primary is flattened: only replies go with what they answer. +#[test] +fn flattening_skips_replies_in_the_question_mark_check_and_keeps_skipped_group_members() { + let page = stream("", " "); + let mut doc = build(&[ + "<< /Type /Catalog /Pages 2 0 R >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>", + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Resources << >> /Contents 4 0 R /Annots [5 0 R 6 0 R 7 0 R] >>", + &page, + "<< /Type /Annot /Subtype /Square /Rect [10 10 60 40] /F 4 /AP << /N 8 0 R >> >>", + // A text-box reply whose text Helvetica can't draw, with no appearance. + "<< /Type /Annot /Subtype /FreeText /Rect [10 50 150 80] /DA (0 g /Helv 12 Tf) /Contents /F 4 /IRT 5 0 R >>", + // A hidden member of the square's group. + "<< /Type /Annot /Subtype /Circle /Rect [70 10 100 40] /F 6 /IRT 5 0 R /RT /Group /AP << /N 8 0 R >> >>", + &stream("/Type /XObject /Subtype /Form /BBox [0 0 50 30]", "0 0 1 rg 0 0 50 30 re f"), + ]); + assert_eq!(flatten(&mut doc, &[0], true, false).unwrap(), 1, "flattened, the square drawn"); let doc = reopen(&doc); let p = &pdfcraft_model::pages(&doc)[0]; let left: Vec = doc.resolve(p.dict.get(b"Annots").unwrap()).as_array().unwrap().iter().filter_map(|f| f.as_ref()).map(|r| r.num).collect(); - assert_eq!(left, [6], "the undrawn reply stays; the comment and its pop-up are gone"); + assert_eq!(left, [7], "the reply went with the square; the hidden group member stays"); +} + +/// #5632 review: a thread three replies deep, listed in `/Annots` in reverse (the deepest reply +/// first, the comment last), goes whole with its comment, none of the replies drawn; a reply to a +/// `/RT /Group` member goes with that member, which is drawn. +#[test] +fn flattening_takes_a_thread_three_deep_in_reverse_order_and_a_group_members_reply() { + let page = stream("", " "); + let form = |rgb: &str| stream("/Type /XObject /Subtype /Form /BBox [0 0 20 18]", &format!("{rgb} rg 0 0 20 18 re f")); + let (blue, red, green) = (form("0 0 1"), form("1 0 0"), form("0 1 0")); + let mut doc = build(&[ + "<< /Type /Catalog /Pages 2 0 R >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>", + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 600 800] /Resources << >> /Contents 4 0 R /Annots [8 0 R 7 0 R 6 0 R 5 0 R 9 0 R 10 0 R] >>", + &page, + // The comment (blue), and three replies, each to the one before, all with appearances (red). + "<< /Type /Annot /Subtype /Text /F 28 /Rect [100 500 120 518] /Contents (Comment) /AP << /N 11 0 R >> >>", + "<< /Type /Annot /Subtype /Text /F 28 /Rect [105 495 125 513] /Contents (Reply 1) /IRT 5 0 R /AP << /N 12 0 R >> >>", + "<< /Type /Annot /Subtype /Text /F 28 /Rect [110 490 130 508] /Contents (Reply 2) /IRT 6 0 R /RT /R /AP << /N 12 0 R >> >>", + "<< /Type /Annot /Subtype /Text /F 28 /Rect [115 485 135 503] /Contents (Reply 3) /IRT 7 0 R /AP << /N 12 0 R >> >>", + // A member of the comment's group (green), and a reply to the member. + "<< /Type /Annot /Subtype /Square /F 4 /Rect [200 500 220 518] /IRT 5 0 R /RT /Group /AP << /N 13 0 R >> >>", + "<< /Type /Annot /Subtype /Text /F 28 /Rect [205 495 225 513] /Contents (Reply to the member) /IRT 9 0 R /AP << /N 12 0 R >> >>", + &blue, + &red, + &green, + ]); + assert_eq!(flatten(&mut doc, &[0], true, false).unwrap(), 2, "the comment and the group member are drawn; no reply is"); + let doc = reopen(&doc); + let p = &pdfcraft_model::pages(&doc)[0]; + assert!(!p.dict.contains(b"Annots"), "the three replies and the member's reply go with what they answer"); + let res = doc.resolve(p.dict.get(b"Resources").unwrap()); + let xo = doc.resolve(res.as_dict().unwrap().get(b"XObject").unwrap()); + let mut drawn: Vec = xo.as_dict().unwrap().iter().filter_map(|(_, v)| v.as_ref()).map(|r| r.num).collect(); + drawn.sort_unstable(); + assert_eq!(drawn, [11, 13], "the comment's and the member's appearances, not the replies'"); +} + +/// #5632 review: `/Annots` should hold references (ISO 32000-2 Table 31), but a reply or pop-up +/// written inline in it still goes with the comment it points at, as a referenced one does. One +/// pointing at a comment that stays, stays. +#[test] +fn flattening_takes_inline_replies_and_popups_with_their_comment() { + let page = stream("", " "); + let blue = stream("/Type /XObject /Subtype /Form /BBox [0 0 20 18]", "0 0 1 rg 0 0 20 18 re f"); + let mut doc = build(&[ + "<< /Type /Catalog /Pages 2 0 R >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>", + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 600 800] /Resources << >> /Contents 4 0 R /Annots [5 0 R \ + << /Type /Annot /Subtype /Text /F 28 /Rect [105 495 125 513] /Contents (Inline reply) /IRT 5 0 R >> \ + << /Type /Annot /Subtype /Popup /Rect [200 400 300 500] /Parent 5 0 R >> \ + << /Type /Annot /Subtype /Text /F 28 /Rect [105 395 125 413] /Contents (Reply to a hidden note) /IRT 6 0 R >> 6 0 R] >>", + &page, + "<< /Type /Annot /Subtype /Text /F 28 /Rect [100 500 120 518] /Contents (Comment) /AP << /N 7 0 R >> >>", + // Hidden: not flattened, so it stays, and so does its inline reply. + "<< /Type /Annot /Subtype /Text /F 30 /Rect [100 400 120 418] /Contents (Hidden note) /AP << /N 7 0 R >> >>", + &blue, + ]); + assert_eq!(flatten(&mut doc, &[0], true, false).unwrap(), 1, "only the comment is drawn"); + let doc = reopen(&doc); + let p = &pdfcraft_model::pages(&doc)[0]; + let annots = doc.resolve(p.dict.get(b"Annots").unwrap()); + let left: Vec = annots + .as_array() + .unwrap() + .iter() + .map(|a| doc.resolve(a).as_dict().unwrap().get(b"Contents").and_then(|c| c.as_string().map(|s| s.to_text())).unwrap_or_default()) + .collect(); + assert_eq!(left, ["Reply to a hidden note", "Hidden note"], "the inline reply and pop-up went with the comment"); +} + +/// #5632 review: replies whose `/IRT` runs in a loop (two replying to each other, one replying to +/// itself, and a square and a note each answering the other) end the flattening. None of them is +/// drawn: each is a reply. Those that answer nothing flattened stay; the comment beside them is +/// flattened and its reply goes with it. +#[test] +fn flattening_ends_on_replies_whose_irt_loops() { + let page = stream("", " "); + let blue = stream("/Type /XObject /Subtype /Form /BBox [0 0 20 18]", "0 0 1 rg 0 0 20 18 re f"); + let mut doc = build(&[ + "<< /Type /Catalog /Pages 2 0 R >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>", + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 600 800] /Resources << >> /Contents 4 0 R /Annots [5 0 R 6 0 R 7 0 R 8 0 R 9 0 R 10 0 R 11 0 R] >>", + &page, + // A comment and its reply. + "<< /Type /Annot /Subtype /Square /F 4 /Rect [100 500 120 518] /AP << /N 12 0 R >> >>", + "<< /Type /Annot /Subtype /Text /F 28 /Rect [105 495 125 513] /IRT 5 0 R /AP << /N 12 0 R >> >>", + // Two replies to each other. + "<< /Type /Annot /Subtype /Text /F 28 /Rect [200 500 220 518] /IRT 8 0 R /AP << /N 12 0 R >> >>", + "<< /Type /Annot /Subtype /Text /F 28 /Rect [205 495 225 513] /IRT 7 0 R /AP << /N 12 0 R >> >>", + // A reply to itself. + "<< /Type /Annot /Subtype /Text /F 28 /Rect [300 500 320 518] /IRT 9 0 R /AP << /N 12 0 R >> >>", + // A square and a note each answering the other: both are replies. + "<< /Type /Annot /Subtype /Square /F 4 /Rect [400 500 420 518] /IRT 11 0 R /AP << /N 12 0 R >> >>", + "<< /Type /Annot /Subtype /Text /F 28 /Rect [405 495 425 513] /IRT 10 0 R /AP << /N 12 0 R >> >>", + &blue, + ]); + assert_eq!(flatten(&mut doc, &[0], true, false).unwrap(), 1, "only the comment is drawn"); + let doc = reopen(&doc); + let p = &pdfcraft_model::pages(&doc)[0]; + let left: Vec = doc.resolve(p.dict.get(b"Annots").unwrap()).as_array().unwrap().iter().filter_map(|f| f.as_ref()).map(|r| r.num).collect(); + assert_eq!(left, [7, 8, 9, 10, 11], "the comment's reply went with it; the looping replies stay"); +} + +/// #5632 review: an `/RT` that isn't `/Group` reads as the default R, as the comment list and the +/// page read it, so replies with an unknown name or a string there are neither drawn nor kept +/// once their comment is flattened. A form field with a stray `/IRT` is no reply: it is flattened +/// as a field. +#[test] +fn flattening_reads_an_unknown_or_malformed_rt_as_a_reply_and_a_widget_with_irt_as_a_field() { + let page = stream("", " "); + let form = |rgb: &str| stream("/Type /XObject /Subtype /Form /BBox [0 0 20 18]", &format!("{rgb} rg 0 0 20 18 re f")); + let (blue, red, green) = (form("0 0 1"), form("1 0 0"), form("0 1 0")); + let mut doc = build(&[ + "<< /Type /Catalog /Pages 2 0 R >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>", + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 600 800] /Resources << >> /Contents 4 0 R /Annots [5 0 R 6 0 R 7 0 R 8 0 R] >>", + &page, + "<< /Type /Annot /Subtype /Text /F 28 /Rect [100 500 120 518] /Contents (Comment) /AP << /N 9 0 R >> >>", + "<< /Type /Annot /Subtype /Text /F 28 /Rect [105 495 125 513] /Contents (Reply) /IRT 5 0 R /RT /Answer /AP << /N 10 0 R >> >>", + "<< /Type /Annot /Subtype /Text /F 28 /Rect [110 490 130 508] /Contents (Reply) /IRT 5 0 R /RT (Group) /AP << /N 10 0 R >> >>", + "<< /Type /Annot /Subtype /Widget /F 4 /Rect [200 500 220 518] /IRT 5 0 R /AP << /N 11 0 R >> >>", + &blue, + &red, + &green, + ]); + assert_eq!(flatten(&mut doc, &[0], true, true).unwrap(), 2, "the comment and the field are drawn; neither reply is"); + let doc = reopen(&doc); + let p = &pdfcraft_model::pages(&doc)[0]; + assert!(!p.dict.contains(b"Annots"), "both replies went with the comment, and the field was flattened"); + let res = doc.resolve(p.dict.get(b"Resources").unwrap()); + let xo = doc.resolve(res.as_dict().unwrap().get(b"XObject").unwrap()); + let mut drawn: Vec = xo.as_dict().unwrap().iter().filter_map(|(_, v)| v.as_ref()).map(|r| r.num).collect(); + drawn.sort_unstable(); + assert_eq!(drawn, [9, 11], "the comment's and the field's appearances, not the replies'"); } /// Flattening never makes a `?` permanent: it refuses, changing nothing, a text box without an @@ -1521,6 +1800,45 @@ fn text_page(content: &str) -> Document { Document::open(Arc::new(out)).unwrap() } +/// ISO 32000-2 §9.6.4: an Edit-text line set in a Type 3 bitmap font is as high as the boxes its +/// glyphs declare with `d1`, whatever its `/FontBBox` says and whichever way its font matrix +/// turns glyph space, so two lines 8 points apart are two lines with 7-point boxes, not slivers. +#[test] +fn edit_lines_in_a_type3_font_are_as_high_as_their_glyphs() { + // Glyph `a` is drawn 60 units wide, from 20 below the baseline to 50 above it, at 0.1 Tf: + // 7 points high, 2 below the baseline. + for (matrix, flip) in [("1 0 0 1 0 0", 1), ("1 0 0 -1 0 0", -1)] { + for bbox in ["[0 0 0 0]", "[0 0 1 1]"] { + let mut doc = text_page(&format!("BT /T3 0.1 Tf 1 0 0 {flip} 72 700 Tm (aa) Tj 1 0 0 {flip} 72 692 Tm (aa) Tj ET")); + let glyph = doc.add(Object::Stream(Stream::from_raw(Dict::new(), b"60 0 0 -20 60 50 d1 0 -20 60 70 re f".to_vec()))); + let font = doc.add( + pdfcraft_cos::parse_indirect( + format!( + "1 0 obj << /Type /Font /Subtype /Type3 /FontMatrix [{matrix}] /FontBBox {bbox} /FirstChar 97 /LastChar 97 /Widths [60] /Encoding << /Differences [97 /a] >> /CharProcs << /a {} 0 R >> >> endobj", + glyph.num + ) + .as_bytes(), + 0, + &|_| None, + ) + .unwrap() + .1, + ); + let p = pdfcraft_model::pages(&doc)[0].obj; + doc.update_dict(p, |d| { + d.set( + b"Resources".to_vec(), + Object::Dict(Dict::from_iter([(b"Font".to_vec(), Object::Dict(Dict::from_iter([(b"T3".to_vec(), Object::Ref(font))])))])), + ) + }) + .unwrap(); + let lines = text::text_lines(&doc, 0).unwrap(); + let rects: Vec<_> = lines.iter().map(|l| (l.text.as_str(), l.rect.map(|v| (v * 100.0).round() / 100.0))).collect(); + assert_eq!(rects, [("aa", [72.0, 698.0, 84.0, 705.0]), ("aa", [72.0, 690.0, 84.0, 697.0])], "{matrix} {bbox}"); + } + } +} + #[test] fn tracked_text_keeps_words_in_paragraphs_and_objects() { let expected = "TRACKED SAMPLE TEXT"; @@ -1588,6 +1906,30 @@ fn tracked_text_preserves_larger_word_gaps() { } } +/// LibreOffice stretches a justified line on both sides of each space glyph, so a gap that reads +/// as a space comes right before the space glyph it widens: one space, not two. A gap after the +/// space glyph, a space glyph with no gap before it, and two space glyphs after a gap read as +/// before. +#[test] +fn a_gap_before_a_space_glyph_reads_as_one_space() { + for show in ["[(Word) -250 ( next) -250 ( last)] TJ", "[(Word ) -250 (next ) -250 (last)] TJ", "[(Word) -250 (next) 0 ( last)] TJ"] { + let doc = text_page(&format!("BT /F1 10 Tf 72 700 Td {show} ET")); + assert_eq!(text::text_lines(&doc, 0).unwrap()[0].text, "Word next last", "{show}"); + } + // Only the first space glyph after the gap is the gap's: two real spaces stay two. + let doc = text_page("BT /F1 10 Tf 72 700 Td [(Word) -250 ( next)] TJ ET"); + assert_eq!(text::text_lines(&doc, 0).unwrap()[0].text, "Word next"); + // The same line split over operators: the gap ends one run, its space glyph starts the next. + let doc = text_page("BT /F1 10 Tf 72 700 Td [(Word) -250] TJ [( next) -250] TJ ( last) Tj ET"); + assert_eq!(text::text_lines(&doc, 0).unwrap()[0].text, "Word next last"); + // Space glyphs on both sides of a break between runs, with no gap, are two spaces. + let doc = text_page("BT /F1 10 Tf 72 700 Td (Word ) Tj ( next) Tj ET"); + assert_eq!(text::text_lines(&doc, 0).unwrap()[0].text, "Word next"); + // Down a vertical column the same. + let doc = vertical_page("BT /V1 20 Tf 500 720 Td [<0041> 300 <00200042> 300 <00200043>] TJ ET"); + assert_eq!(text::text_lines(&doc, 0).unwrap()[0].text, "A B C"); +} + #[test] fn tracked_text_positions_do_not_count_character_spacing_twice() { let expected = "TRACKED SAMPLE TEXT"; @@ -5445,6 +5787,20 @@ fn replacing_a_line_keeps_a_distinct_nearby_line() { } } +#[test] +fn text_blocks_split_at_headings_and_blank_lines() { + let page = |size: u32, gap: u32, words: &[&str], top: u32| -> String { + words.iter().enumerate().map(|(i, w)| format!("BT /F1 {size} Tf 40 {} Td ({w}) Tj ET\n", top - i as u32 * gap)).collect() + }; + let count = |content: String| text::text_blocks(&text_page(&content), 0).unwrap().len(); + let heading = page(12, 14, &["First line of text", "second line here", "third line ends"], 340) + + &page(12, 14, &["HEADING"], 340 - 28 - 24) + + &page(12, 14, &["Next paragraph one", "next two", "next three"], 340 - 28 - 24 - 24); + assert_eq!(count(heading), 3); + assert_eq!(count(page(12, 14, &["Para one a", "para one b"], 340) + &page(12, 14, &["Para two a", "para two b"], 340 - 42)), 2); + assert_eq!(count(page(24, 29, &["VISIBLE", "HIDDEN"], 340)), 1); +} + #[test] fn editing_text_keeps_an_overlapping_watermark_and_covered_paragraphs() { let listed = |doc: &Document| text::text_lines(doc, 0).unwrap().into_iter().map(|l| l.text).collect::>(); @@ -6199,6 +6555,55 @@ fn header_and_watermark_drawability_matches_what_applying_refuses() { } } +/// Thai in a header, footer or text watermark is drawn from the Thai face as Add text draws it: +/// Type 3 glyphs with a ToUnicode map, one set of fonts shared by every page the mark goes on, +/// beside the mark's standard face. A build without a Thai face refuses it, saying what to do. +#[test] +fn thai_headers_footers_and_watermarks_are_drawn_from_the_thai_face() { + let blank = String::new; + let footer = HeaderFooter { + text: [blank(), "หน้า <<1>> บุญฤทธิ์".into(), blank(), blank(), blank(), "น้ำ <>".into()], + ..HeaderFooter::default() + }; + let wm = Watermark { text: "ลับ ปั้น\nDRAFT".into(), ..Watermark::default() }; + if pdfcraft_fonts::document_thai_font().is_none() { + let mut doc = fixture(); + for err in [add_header_footer(&mut doc, &[0], &footer, false, &cx()).unwrap_err(), add_watermark(&mut doc, &[0], &wm, false).unwrap_err()] { + assert!(matches!(&err, EditError::Invalid(m) if m.contains("Thai") && m.contains("CRAFT_FONTS_DIR")), "{err:?}"); + } + assert!(!doc.is_modified(), "nothing is written when the font is missing"); + assert_eq!((footer.first_undrawable(), wm.first_undrawable()), (Some('ห'), Some('ล'))); + return; + } + assert_eq!((footer.first_undrawable(), wm.first_undrawable()), (None, None)); + let mut doc = fixture(); + add_header_footer(&mut doc, &[0, 1], &footer, false, &cx()).unwrap(); + add_watermark(&mut doc, &[0, 1], &wm, false).unwrap(); + let doc = reopen(&doc); + let mut shared = Vec::new(); + for page in 0..2 { + let drawn = streams(&doc, page).concat(); + assert!(drawn.contains("/PCTh") && drawn.contains("(DRAFT) Tj") && !drawn.contains("(?"), "{drawn}"); + let p = &pdfcraft_model::pages(&doc)[page]; + let res = doc.resolve(p.dict.get(b"Resources").unwrap()); + let fonts = doc.resolve(res.as_dict().unwrap().get(b"Font").unwrap()).as_dict().unwrap().clone(); + let thai: Vec<_> = fonts.iter().filter(|(k, _)| k.starts_with(b"PCTh")).map(|(k, v)| (k.clone(), v.clone())).collect(); + assert_eq!(thai.len(), 2, "the footer's fonts and the watermark's"); + let mut mapped = String::new(); + for (_, f) in &thai { + let f = doc.resolve(f).as_dict().unwrap().clone(); + assert_eq!(f.name(b"Subtype"), Some(&b"Type3"[..])); + mapped += &String::from_utf8_lossy(&stream_bytes(&doc, f.get(b"ToUnicode").unwrap()).unwrap()); + } + // น้ำ, ก (the Bates prefix) and ปั้น copy as typed. + for text in ["<0E190E490E33>", "<0E01>", "<0E1B0E310E49>"] { + assert!(mapped.contains(text), "{text} in {mapped}"); + } + shared.push(thai); + } + assert_eq!(shared[0], shared[1], "every page draws with the same fonts"); +} + /// #125: page-content tools refuse text the standard fonts can't draw instead of writing `?`, /// and leave the document untouched. #[test] @@ -6439,6 +6844,142 @@ fn reading_follows_forms_however_their_resources_are_stored() { } } +/// #3503: a form XObject's `/BBox` clips its content, so text drawn outside it is not shown and +/// should not be read either -- matching the renderer, which already honours the clip. +#[test] +fn form_bbox_clips_text_from_reading() { + let page = stream("", "/Fm Do"); + let fm = stream( + "/Type /XObject /Subtype /Form /BBox [0 0 50 50] /Resources << /Font << /F1 6 0 R >> >>", + "BT /F1 12 Tf 5 5 Td (INSIDE) Tj ET BT /F1 12 Tf 100 100 Td (OUTSIDE) Tj ET", + ); + let doc = build(&[ + "<< /Type /Catalog /Pages 2 0 R >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>", + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Resources << /XObject << /Fm 5 0 R >> >> /Contents 4 0 R >>", + &page, + &fm, + "<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>", + ]); + let texts: Vec = text::reading_blocks(&doc, 0).unwrap().into_iter().map(|b| b.text).collect(); + assert_eq!(texts, ["INSIDE"]); +} + +/// #3503: a page's own `re`/`W`/`n` clip (not a form's `/BBox`) hides text outside it the same +/// way, and `Q` lifts it again -- the clip is scoped to its `q`/`Q` like the rest of the graphics +/// state, not a one-way switch. +#[test] +fn page_re_w_n_clip_hides_text_from_reading_until_q() { + let page = stream( + "", + "q 0 0 50 50 re W n BT /F1 12 Tf 5 5 Td (INSIDE) Tj ET BT /F1 12 Tf 100 100 Td (HIDDEN) Tj ET Q BT /F1 12 Tf 100 100 Td (AFTER) Tj ET", + ); + let doc = build(&[ + "<< /Type /Catalog /Pages 2 0 R >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>", + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Resources << /Font << /F1 5 0 R >> >> /Contents 4 0 R >>", + &page, + "<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>", + ]); + let texts: Vec = text::reading_blocks(&doc, 0).unwrap().into_iter().map(|b| b.text).collect(); + assert_eq!(texts, ["INSIDE", "AFTER"]); +} + +/// The text `reading_blocks` reads on a 400 × 400 pt page drawing `content` with Helvetica as +/// `/F1`, and forms `/Fa` and `/Fb` (objects 6 and 7) with the given dictionary entries and content. +fn read_clipped(content: &str, forms: [(&str, &str); 2]) -> Vec { + let font = "/Font << /F1 5 0 R >>"; + let resources = format!("<< {font} /XObject << /Fa 6 0 R /Fb 7 0 R >> >>"); + let form = |(dict, body): (&str, &str)| stream(&format!("/Type /XObject /Subtype /Form {dict} /Resources {resources}"), body); + let doc = build(&[ + "<< /Type /Catalog /Pages 2 0 R >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>", + &format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 400 400] /Resources {resources} /Contents 4 0 R >>"), + &stream("", content), + "<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>", + &form(forms[0]), + &form(forms[1]), + ]); + text::reading_blocks(&doc, 0).unwrap().into_iter().map(|b| b.text).collect() +} + +/// #5075 review: a `/BBox` with anything but four numbers is no bound. Reading the numbers it +/// happens to hold (`[0 /Bad 0 50 50]` as `[0 0 50 50]`) invented a clip and hid real text. +#[test] +fn a_malformed_form_bbox_clips_nothing() { + let body = "BT /F1 12 Tf 5 5 Td (INSIDE) Tj ET BT /F1 12 Tf 100 100 Td (OUTSIDE) Tj ET"; + for bbox in ["[0 /Bad 0 50 50]", "[0 0 50]", "[0 0 50 50 60]", "[0 (x) 50 50]"] { + let texts = read_clipped("/Fa Do", [(&format!("/BBox {bbox}"), body), ("", "")]); + assert_eq!(texts, ["INSIDE", "OUTSIDE"], "/BBox {bbox}"); + } +} + +/// #5075 review: a clipping path of several rectangles clips to all of them, not to the last one +/// drawn (text seen through the first rectangle was hidden), and not to the box around them +/// either (text in the gap between them was read, though nothing of it shows). +#[test] +fn a_clip_of_several_rectangles_keeps_text_in_each() { + let content = "q 0 0 50 50 re 100 100 50 50 re W n BT /F1 12 Tf 5 5 Td (FIRST) Tj ET BT /F1 12 Tf 105 105 Td (SECOND) Tj ET \ + BT /F1 12 Tf 60 60 Td (GAP) Tj ET BT /F1 12 Tf 300 300 Td (OUTSIDE) Tj ET Q"; + assert_eq!(read_clipped(content, [("", ""), ("", "")]), ["FIRST", "SECOND"]); +} + +/// #5075 review: a form's `/BBox` clips however thin it is and wherever its matrix puts it. A box +/// one point by 1e-10 was taken for no box (its area fell under a fixed threshold), and one moved +/// 1e12 points away lost its area to rounding; either way the text outside it was read. +#[test] +fn a_thin_or_far_off_form_bbox_still_clips() { + let after = "BT /F1 12 Tf 5 300 Td (AFTER) Tj ET"; + let body = "BT /F1 12 Tf 5 5 Td (INSIDE) Tj ET BT /F1 12 Tf 300 300 Td (OUTSIDE) Tj ET"; + let sorted = |mut v: Vec| { + v.sort(); + v + }; + let thin = read_clipped(&format!("/Fa Do {after}"), [("/BBox [0 0 1 1e-10]", body), ("", "")]); + assert_eq!(thin, ["AFTER"], "a thin /BBox"); + let far = read_clipped(&format!("/Fa Do {after}"), [("/Matrix [1 0 0 1 1e12 1e12] /BBox [0 0 100 100]", body), ("", "")]); + assert_eq!(sorted(far), ["AFTER", "INSIDE"], "a /BBox 1e12 points away"); +} + +/// #5075 review: a clip of no area has no inside, so nothing drawn under it shows (ISO 32000-2 +/// §8.5.4). It was taken for no clip at all, and everything under it was read. +#[test] +fn a_clip_of_no_area_shows_nothing() { + let content = "q 0 0 100 0 re W n BT /F1 12 Tf 5 5 Td (HIDDEN) Tj ET Q BT /F1 12 Tf 5 300 Td (AFTER) Tj ET"; + assert_eq!(read_clipped(content, [("", ""), ("", "")]), ["AFTER"]); +} + +/// #5075 review: what shows is the intersection of every clip in force. A long run that crosses a +/// page clip and, apart from it, a form's `/BBox` overlaps each of them but shows in neither, as it +/// does not when one form's `/BBox` lies apart from the one of the form it is drawn in. +#[test] +fn disjoint_clips_show_nothing_between_them() { + let run = "BT /F1 12 Tf 5 10 Td (AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA) Tj ET"; + let after = "BT /F1 12 Tf 5 300 Td (AFTER) Tj ET"; + // A page clip and a form's `/BBox` side by side. + let page_clip = format!("q 0 0 50 50 re W n /Fa Do Q {after}"); + assert_eq!(read_clipped(&page_clip, [("/BBox [100 0 150 50]", run), ("", "")]), ["AFTER"]); + // A form's `/BBox` and the `/BBox` of the form inside it, side by side. + let nested = format!("/Fa Do {after}"); + assert_eq!(read_clipped(&nested, [("/BBox [0 0 50 50]", "/Fb Do"), ("/BBox [100 0 150 50]", run)]), ["AFTER"]); +} + +/// #5075 review: a clip turned with its matrix keeps its turned shape. Its upright bounding box +/// takes in corners it does not cover, and text there was read although it is clipped away. +#[test] +fn a_turned_clip_hides_text_in_the_corners_of_its_bounding_box() { + // A 100 pt square turned 45° about (200, 0): a diamond with its lower corner at (200, 0). + // Undone straight after the clip, so the text is drawn upright on the page. + let (turn, undo) = + ("0.70710678 0.70710678 -0.70710678 0.70710678 200 0 cm", "0.70710678 -0.70710678 0.70710678 0.70710678 -141.421356 141.421356 cm"); + let texts = "BT /F1 6 Tf 132 3 Td (CORNER) Tj ET BT /F1 6 Tf 195 65 Td (CENTRE) Tj ET"; + let page_clip = format!("q {turn} 0 0 100 100 re W n {undo} {texts} Q"); + assert_eq!(read_clipped(&page_clip, [("", ""), ("", "")]), ["CENTRE"], "a turned page clip"); + // The same diamond as a form's `/BBox`, under its `/Matrix`. + let matrix = "/Matrix [0.70710678 0.70710678 -0.70710678 0.70710678 200 0] /BBox [0 0 100 100]"; + assert_eq!(read_clipped("/Fa Do", [(matrix, &format!("{undo} {texts}")), ("", "")]), ["CENTRE"], "a turned form /BBox"); +} + /// A chain of forms, each drawing the next twenty times, twelve deep: within the depth cap but /// 20^11 visits without a budget. Reading stops at the page's visit budget instead of hanging. #[test] @@ -6553,24 +7094,28 @@ fn forms_that_fan_out_a_hundred_ways_stop_quickly_and_say_so() { /// Forms that draw themselves or each other, have no resources, a malformed matrix, or are /// missing are read once (or skipped) without panicking or looping. +/// +/// `/BBox [0 0 20 420]` is wide enough to hold every form's own text below (#3503 clips a form's +/// content to its `/BBox`; this test is about the loop and resource handling, not clipping). #[test] fn hostile_form_xobjects_are_read_once() { let page = stream("", "/Loop Do /A Do /NoRes Do /Bad Do /Missing Do /F1 Do"); let font = "/Font << /F1 5 0 R >>"; let lp = stream( - &format!("/Subtype /Form /BBox [0 0 9 9] /Resources << {font} /XObject << /Loop 6 0 R >> >>"), + &format!("/Subtype /Form /BBox [0 0 20 420] /Resources << {font} /XObject << /Loop 6 0 R >> >>"), "/Loop Do BT /F1 10 Tf 10 10 Td (once) Tj ET", ); let a = stream( - &format!("/Subtype /Form /BBox [0 0 9 9] /Resources << {font} /XObject << /B 8 0 R >> >>"), + &format!("/Subtype /Form /BBox [0 0 20 420] /Resources << {font} /XObject << /B 8 0 R >> >>"), "/B Do BT /F1 10 Tf 10 100 Td (from a) Tj ET", ); let b = stream( - &format!("/Subtype /Form /BBox [0 0 9 9] /Resources << {font} /XObject << /A 7 0 R >> >>"), + &format!("/Subtype /Form /BBox [0 0 20 420] /Resources << {font} /XObject << /A 7 0 R >> >>"), "/A Do BT /F1 10 Tf 10 200 Td (from b) Tj ET", ); - let nores = stream("/Subtype /Form /BBox [0 0 9 9]", "BT /F1 10 Tf 10 300 Td (inherited) Tj ET"); - let bad = stream(&format!("/Subtype /Form /BBox [0 0 9 9] /Matrix [1 0 0] /Resources << {font} >>"), "BT /F1 10 Tf 10 400 Td (bad matrix) Tj ET"); + let nores = stream("/Subtype /Form /BBox [0 0 20 420]", "BT /F1 10 Tf 10 300 Td (inherited) Tj ET"); + let bad = + stream(&format!("/Subtype /Form /BBox [0 0 20 420] /Matrix [1 0 0] /Resources << {font} >>"), "BT /F1 10 Tf 10 400 Td (bad matrix) Tj ET"); let doc = build(&[ "<< /Type /Catalog /Pages 2 0 R >>", "<< /Type /Pages /Kids [3 0 R] /Count 1 >>", @@ -6619,6 +7164,57 @@ fn added_thai_text_wraps_clusters_and_survives_reopen() { assert!(arabic_font_names(&doc, 0).is_empty()); } +#[test] +fn added_thai_text_wraps_between_dictionary_words() { + if pdfcraft_fonts::document_thai_font().is_none() { + eprintln!("skipping the Thai word wrapping checks: built without craft-fonts (set CRAFT_FONTS_DIR)"); + return; + } + let text = "การประปาส่วนภูมิภาคให้บริการน้ำประปาที่สะอาดแก่ประชาชนทั่วประเทศ"; + let t = AddedText { rect: [0.0, 0.0, 90.0, 400.0], text: text.into(), size: 16.0, ..AddedText::default() }; + let rows = super::added::lines(&t); + assert!(rows.len() > 2, "{rows:?}"); + assert_eq!(rows.concat(), text, "no space is added or lost between the rows"); + // Every row ends where the dictionary lets one end, never inside a word such as ประปา. + let cuts = crate::linebreak::breaks(text); + let mut at = 0; + for row in rows.iter().take(rows.len() - 1) { + at += row.len(); + assert!(cuts.contains(&at), "row {row:?} ends inside a word: {rows:?}"); + } + // Spaces where a row ends are dropped, as for Latin text, and a word wider than the row is + // still broken between clusters. + let spaced = + AddedText { text: "น้ำประปา สะอาด ปลอดภัย".into(), rect: [0.0, 0.0, 60.0, 400.0], ..t.clone() }; + let rows = super::added::lines(&spaced); + assert!(rows.iter().all(|r| !r.starts_with(' ') && !r.ends_with(' ')), "{rows:?}"); + assert_eq!(rows.concat(), "น้ำประปาสะอาดปลอดภัย"); + let narrow = AddedText { rect: [0.0, 0.0, 16.0, 200.0], text: "กิ่กิ่กิ่".into(), ..t.clone() }; + assert_eq!(super::added::lines(&narrow), ["กิ่", "กิ่", "กิ่"]); + // Spaces that open a paragraph are kept, but they are not a row of their own: a first word + // too wide to fit after them stays on the first row (or breaks between clusters there). + for width in [16.0, 40.0, 60.0, 90.0] { + let indented = AddedText { rect: [0.0, 0.0, width, 400.0], text: " ประปาสะอาด".into(), ..t.clone() }; + let rows = super::added::lines(&indented); + assert!(rows.iter().all(|r| !r.trim().is_empty()), "an empty row at width {width}: {rows:?}"); + assert!(rows[0].starts_with(" "), "the opening spaces are kept at width {width}: {rows:?}"); + assert_eq!(rows.concat().trim_start(), "ประปาสะอาด"); + } + // A first word that fits a row alone but not after the opening spaces breaks between clusters + // on the first row rather than running past its end. + let word = "ประปา"; + let word_w = super::added::shaped_width(&t, word, false); + let indent_w = super::added::shaped_width(&t, " ", false); + let width = word_w + indent_w / 2.0; + let indented = AddedText { rect: [0.0, 0.0, width, 400.0], text: format!(" {word}"), ..t.clone() }; + let rows = super::added::lines(&indented); + for row in &rows { + assert!(super::added::shaped_width(&t, row, false) <= width + 0.5, "{row:?} overflows {width}: {rows:?}"); + } + assert!(rows.len() > 1 && rows[0].starts_with(" "), "{rows:?}"); + assert_eq!(rows.concat().trim_start(), word); +} + #[test] fn group_moves_preserve_glyph_bytes_and_unselected_text_state() { for content in [ @@ -8803,6 +9399,159 @@ fn removing_marks_strips_wrappers_that_closed_open_state() { assert_eq!(streams(&doc, 0), ["q 2 0 0 2 0 0 cm"]); } +/// Printer marks: drawn outside the trim box as a removable page mark; the page +/// is widened to make room when asked and the trim box stays; running again replaces them. +#[test] +fn printer_marks_are_drawn_outside_the_trim_box_and_removable() { + use crate::printer_marks::PrinterMarks; + let mut doc = fixture(); + let page_content_text = |doc: &Document, page: usize| String::from_utf8_lossy(&page_content_bytes(doc, page)).into_owned(); + let page = page_list(&doc)[0].clone(); + // A trim box inside a letter-size page, 36 pt in: room for the marks, bleed marks included + // (the bleed box is 9 pt out, so they reach 33 pt beyond the trim box), without widening. + doc.update_dict(page.obj, |d| { + d.set(b"MediaBox".to_vec(), Object::Array([0.0, 0.0, 612.0, 792.0].iter().map(|v| Object::Real(*v)).collect())); + d.set(b"TrimBox".to_vec(), Object::Array([36.0, 36.0, 576.0, 756.0].iter().map(|v| Object::Real(*v)).collect())); + d.set(b"BleedBox".to_vec(), Object::Array([27.0, 27.0, 585.0, 765.0].iter().map(|v| Object::Real(*v)).collect())); + }) + .unwrap(); + let marks = PrinterMarks { label: "Quote 1042.pdf".into(), date: "2026-10-13".into(), ..PrinterMarks::default() }; + let widened = add_printer_marks(&mut doc, &[0, 1], &marks, false).unwrap(); + assert_eq!(widened, vec![1], "page 2 has no trim box inside its media box, so it was widened"); + assert_eq!(marks_present(&doc), vec![MarkKind::PrinterMarks]); + let text = page_content_text(&doc, 0); + assert!(text.contains("/PCMark /PrinterMarks"), "{text}"); + // A trim mark at the lower-left corner: vertical below the box, horizontal to its left. + assert!(text.contains("36 30 m 36 12 l S") && text.contains("30 36 m 12 36 l S"), "{text}"); + assert!(text.contains("/PCReg CS 1 SCN /PCReg cs 1 scn"), "registration colour: {text}"); + assert!(!text.contains("1 1 1 1 K"), "not four process inks alone: {text}"); + assert!(text.contains("[1 1] 0 d") && text.contains("27 21 m 27 3 l S"), "dashed bleed marks: {text}"); + assert!(text.contains("1 0 0 0 k") && text.contains("0 0 0 0.25 k"), "colour bars: {text}"); + assert!(text.contains("(Quote 1042.pdf 1 of 3 2026-10-13) Tj"), "page information: {text}"); + // Page 2: the media box grew by 36 pt a side around its (whole-page) trim box. + let p2 = page_list(&doc)[1].clone(); + let media: Vec = p2.dict.get(b"MediaBox").and_then(|m| m.as_array()).unwrap().iter().filter_map(|v| v.as_f64()).collect(); + assert_eq!(media, vec![-36.0, -36.0, 636.0, 836.0]); + // Again with replace: still one set of marks per page; without replace, two. + add_printer_marks(&mut doc, &[0], &marks, true).unwrap(); + assert_eq!(page_content_text(&doc, 0).matches("/PCMark /PrinterMarks").count(), 1); + add_printer_marks(&mut doc, &[0], &marks, false).unwrap(); + assert_eq!(page_content_text(&doc, 0).matches("/PCMark /PrinterMarks").count(), 2); + assert_eq!(remove_marks(&mut doc, &[0, 1], MarkKind::PrinterMarks).unwrap(), 3); + assert!(marks_present(&doc).is_empty()); + // Nothing chosen, or a bad weight, is refused. + let none = PrinterMarks { trim: false, bleed: false, registration: false, color_bars: false, page_info: false, ..PrinterMarks::default() }; + assert!(add_printer_marks(&mut doc, &[0], &none, false).is_err()); + assert!(add_printer_marks(&mut doc, &[0], &PrinterMarks { line_weight: 0.0, ..PrinterMarks::default() }, false).is_err()); +} + +/// Convert colours: device, ICC-based and separation colours in page content +/// and forms become the target's operators; black is preserved; patterns stay; a second run +/// finds nothing. +#[test] +fn colours_are_converted_to_gray_rgb_and_cmyk() { + use crate::convert_colours::{ColourTarget, ConvertColours, convert_colours}; + let mut doc = fixture(); + let page = page_list(&doc)[0].clone(); + // An ICCBased RGB space and a spot colour (Separation to CMYK, Type 2 tint). + let mut icc = Dict::new(); + icc.set(b"N".to_vec(), Object::Int(3)); + let icc_ref = doc.add(Object::Stream(Stream::flate(icc, b"not a real profile"))); + let mut fun = Dict::new(); + fun.set(b"FunctionType".to_vec(), Object::Int(2)); + fun.set(b"Domain".to_vec(), Object::Array(vec![Object::Int(0), Object::Int(1)])); + fun.set(b"C0".to_vec(), Object::Array(vec![Object::Int(0); 4])); + fun.set(b"C1".to_vec(), Object::Array(vec![Object::Int(0), Object::Int(1), Object::Int(1), Object::Int(0)])); + fun.set(b"N".to_vec(), Object::Int(1)); + let mut cs = Dict::new(); + cs.set(b"CS0".to_vec(), Object::Array(vec![Object::name("ICCBased"), Object::Ref(icc_ref)])); + cs.set(b"Spot".to_vec(), Object::Array(vec![Object::name("Separation"), Object::name("Red"), Object::name("DeviceCMYK"), Object::Dict(fun)])); + let mut res = Dict::new(); + res.set(b"ColorSpace".to_vec(), Object::Dict(cs)); + let content = b"1 0 0 rg 10 10 50 50 re f 0 0 1 RG 0 0 m 100 100 l S /CS0 cs 0 1 0 scn 0 0 20 20 re f /Spot cs 1 scn 30 30 20 20 re f 0 g 60 60 10 10 re f 0.5 g 70 70 5 5 re f"; + let c = doc.add(Object::Stream(Stream::flate(Dict::new(), content))); + doc.update_dict(page.obj, |d| { + d.set(b"Contents".to_vec(), Object::Ref(c)); + d.set(b"Resources".to_vec(), Object::Dict(res)); + }) + .unwrap(); + let mut no_images = |_: &Document, _: &Stream, _: ColourTarget| None; + let flat = |b: &[u8]| String::from_utf8_lossy(b).split_whitespace().collect::>().join(" "); + // Grayscale. + let mut gray = doc.clone(); + let done = convert_colours(&mut gray, &[0], &ConvertColours::default(), &mut no_images).unwrap(); + assert_eq!((done.colours, done.pages), (6, 1), "{done:?}"); + let t = flat(&page_content_bytes(&gray, 0)); + assert!(t.contains("0.299 g 10 10 50 50 re f"), "red → its luma: {t}"); + assert!(t.contains("0.114 G 0 0 m 100 100 l S"), "blue stroke: {t}"); + assert!(t.contains("0 g 0 1 0 scn") || t.contains("0.587 g 0 0 20 20 re f"), "ICC green through its space: {t}"); + assert!(t.contains("0.587 g 0 0 20 20 re f"), "{t}"); + assert!(t.contains("0.299 g 30 30 20 20 re f"), "the spot's tint through its CMYK alternate (0 1 1 0 = red): {t}"); + assert!(t.contains("0 g 60 60 10 10 re f") && t.contains("0.5 g 70 70 5 5 re f"), "gray stays gray: {t}"); + assert!(!t.contains("/CS0 cs") && !t.contains("/Spot cs"), "named spaces are gone: {t}"); + assert_eq!(convert_colours(&mut gray, &[0], &ConvertColours::default(), &mut no_images).unwrap().colours, 0, "nothing left"); + // CMYK with black preserved; without, black is still 100% K by the formula. + let mut cmyk = doc.clone(); + let s = ConvertColours { target: ColourTarget::Cmyk, ..ConvertColours::default() }; + convert_colours(&mut cmyk, &[0], &s, &mut no_images).unwrap(); + let t = flat(&page_content_bytes(&cmyk, 0)); + assert!(t.contains("0 1 1 0 k 10 10 50 50 re f"), "red: {t}"); + assert!(t.contains("0 0 0 1 k 60 60 10 10 re f"), "black preserved: {t}"); + assert!(t.contains("0 0 0 0.5 k 70 70 5 5 re f"), "gray 0.5 → 50% K: {t}"); + // RGB: the spot becomes its red. + let mut rgbd = doc.clone(); + convert_colours(&mut rgbd, &[0], &ConvertColours { target: ColourTarget::Rgb, ..ConvertColours::default() }, &mut no_images).unwrap(); + let t = flat(&page_content_bytes(&rgbd, 0)); + assert!(t.contains("1 0 0 rg 30 30 20 20 re f"), "{t}"); + assert!(t.contains("1 0 0 rg 10 10 50 50 re f"), "already RGB, kept: {t}"); +} + +/// Fix hairlines: zero-width strokes and strokes thinner than the threshold on +/// the page (after the CTM) get the chosen width, in page content and the forms it draws; wider +/// lines, and glyphs and patterns unless asked, are left alone. +#[test] +fn hairlines_are_widened_in_page_content_and_forms() { + use crate::hairlines::{Hairlines, fix_hairlines}; + let mut doc = fixture(); + let page = page_list(&doc)[0].clone(); + // A form drawn at half scale with a 0.1 pt line (0.05 pt on the page) and a 4 pt line. + let mut fd = Dict::new(); + fd.set(b"Type".to_vec(), Object::name("XObject")); + fd.set(b"Subtype".to_vec(), Object::name("Form")); + fd.set(b"BBox".to_vec(), Object::Array([0.0, 0.0, 100.0, 100.0].iter().map(|v| Object::Real(*v)).collect())); + let form = doc.add(Object::Stream(Stream::flate(fd, b"0.1 w 0 0 m 50 50 l S 4 w 0 50 m 50 0 l S"))); + let content = b"0 w 10 10 m 100 10 l S 2 w 10 20 m 100 20 l S q 0.5 0 0 0.5 0 0 cm 0.2 w 0 0 m 10 10 l S /Fm1 Do Q 0.5 w 10 30 m 100 30 l S"; + let cs = doc.add(Object::Stream(Stream::flate(Dict::new(), content))); + let mut xo = Dict::new(); + xo.set(b"Fm1".to_vec(), Object::Ref(form)); + let mut res = Dict::new(); + res.set(b"XObject".to_vec(), Object::Dict(xo)); + doc.update_dict(page.obj, |d| { + d.set(b"Contents".to_vec(), Object::Ref(cs)); + d.set(b"Resources".to_vec(), Object::Dict(res)); + }) + .unwrap(); + let fixed = fix_hairlines(&mut doc, &[0], &Hairlines::default()).unwrap(); + // Page: the 0 w line and the 0.2 w line at half scale (0.1 pt); the 2 pt and 0.5 pt lines stay. + // Form: the 0.1 w line (0.05 pt on the page); the 4 pt line stays. + assert_eq!(fixed, crate::hairlines::Fixed { lines: 3, pages: 1, elsewhere: 0 }); + let flat = |b: &[u8]| String::from_utf8_lossy(b).split_whitespace().collect::>().join(" "); + let text = flat(&page_content_bytes(&doc, 0)); + assert!(text.contains("0.72 w 10 10 m 100 10 l S"), "zero width → 0.72 pt: {text}"); + assert!(text.contains("2 w 10 20 m 100 20 l S") && text.contains("0.5 w 10 30 m 100 30 l S"), "wider lines untouched: {text}"); + assert!(text.contains("1.44 w 0 0 m 10 10 l S"), "under a half-scale CTM the width is doubled: {text}"); + let Object::Stream(fs) = &*doc.get(form) else { panic!() }; + let ft = flat(&fs.decoded().unwrap()); + assert!(ft.contains("1.44 w 0 0 m 50 50 l S") && ft.contains("4 w 0 50 m 50 0 l S"), "{ft}"); + // Nothing left to fix: a second run changes nothing. + assert_eq!(fix_hairlines(&mut doc, &[0], &Hairlines::default()).unwrap().lines, 0); + // A stricter threshold catches the 0.5 pt line too, with a custom width. + let fixed = fix_hairlines(&mut doc, &[0], &Hairlines { narrower_than: 0.6, width: 1.0, ..Hairlines::default() }).unwrap(); + assert_eq!(fixed.lines, 1); + assert!(flat(&page_content_bytes(&doc, 0)).contains("1 w 10 30 m 100 30 l S")); + assert!(fix_hairlines(&mut doc, &[0], &Hairlines { width: 0.0, ..Hairlines::default() }).is_err()); +} + /// #3087: flattening comments leaves alone what isn't a comment: links, and the non-markup /// multimedia and print-production annotations (3D, Screen, Movie, RichMedia, PrinterMark, /// TrapNet, Watermark), which it used to bake into the page and delete with their data. @@ -9098,7 +9847,7 @@ fn vertical_page(content: &str) -> Document { ]); let unicode = doc.add(Object::Stream(Stream::flate( Dict::new(), - b"1 begincodespacerange <0000> endcodespacerange 6 beginbfchar <0041> <0041> <0042> <0042> <0043> <0043> <0044> <0044> <0045> <0045> <0046> <0046> endbfchar", + b"1 begincodespacerange <0000> endcodespacerange 7 beginbfchar <0020> <0020> <0041> <0041> <0042> <0042> <0043> <0043> <0044> <0044> <0045> <0045> <0046> <0046> endbfchar", ))); font.set(b"ToUnicode".to_vec(), Object::Ref(unicode)); font.set( @@ -9648,6 +10397,20 @@ fn added_text_is_wrapped_once_per_write() { assert!(calls <= 4, "the text was wrapped {calls} times for 40 lines"); } +/// Negative character spacing keeps a line from ever filling its box. Wrapping then holds every +/// word on one line, appended and measured once each rather than copied and measured again for +/// each word (which took minutes for this text); the widths still decide the breaks. +#[test] +fn added_text_that_never_fills_its_line_wraps_in_linear_time() { + let words = vec!["word"; 200_000].join(" "); + let packed = AddedText { rect: [0.0, 0.0, 100.0, 100.0], text: words.clone(), char_spacing: -10.0, ..AddedText::default() }; + let start = std::time::Instant::now(); + assert_eq!(added::lines(&packed), [words]); + assert!(start.elapsed() < std::time::Duration::from_secs(10), "{:?}", start.elapsed()); + let plain = AddedText { rect: [0.0, 0.0, 60.0, 100.0], text: "one two three four".into(), size: 10.0, ..AddedText::default() }; + assert_eq!(added::lines(&plain), ["one two", "three four"]); +} + /// Character spacing on added text (#4245) is the PDF's `Tc`, kept in `/PCAdded` so the item /// reopens with it; an item written without it reads 0. #[test] @@ -10750,8 +11513,250 @@ fn carriage_returns_break_rows_on_the_styled_path() { fn a_run_of_spaces_in_a_subset_without_a_space_glyph_keeps_its_width() { let mut doc = text_page("BT /F2 12 Tf 72 700 Td (ab) Tj /F1 12 Tf ( ) Tj /F2 12 Tf (ba) Tj ET"); assert_eq!(text::replace_block(&mut doc, 0, 0, "ab ba ab").unwrap().substituted, None); - let content = String::from_utf8_lossy(&page_content_bytes(&reopen(&doc), 0)).into_owned(); + let reopened = reopen(&doc); + let content = String::from_utf8_lossy(&page_content_bytes(&reopened, 0)).into_owned(); assert_eq!(content.matches("[-330] TJ").count(), 5, "{content}"); + assert_eq!(text::text_lines(&reopened, 0).unwrap()[0].text, "ab ba ab"); +} + +/// Spaces drawn as TJ gaps in a subset without a space glyph are wrapped in `/ActualText`, so +/// repeated spaces and leading indents read back through `text_lines`, `editor_text`, and text +/// extraction, and survive a second edit without shifting the line or miscalibrating its scale +/// (#4229). +#[test] +fn spaces_and_indents_in_a_subset_without_a_space_glyph_round_trip_across_edits() { + for paragraph in [true, false] { + for typed in [" ab ba", "ab ba ab", " ab ba ab "] { + let mut doc = text_page("BT /F2 12 Tf 72 700 Td (ab) Tj /F1 12 Tf ( ) Tj /F2 12 Tf (ba) Tj ET"); + let edit = if paragraph { text::replace_block(&mut doc, 0, 0, typed) } else { text::replace_line(&mut doc, 0, 0, typed) }; + assert_eq!(edit.unwrap().substituted, None, "{typed:?}"); + let mut doc = reopen(&doc); + let lines = text::text_lines(&doc, 0).unwrap(); + assert_eq!(lines.len(), 1, "{typed:?}: {lines:?}"); + assert_eq!(lines[0].text, typed, "{typed:?}"); + assert!((lines[0].rect[0] - 72.0).abs() < 0.01, "{typed:?}: {:?}", lines[0].rect); + let blocks = text::text_blocks(&doc, 0).unwrap(); + assert_eq!(blocks[0].editor_text, typed, "{typed:?}"); + + // Re-edit the reopened document and verify the origin, gap count, and scale stay exact. + let next = " ba ab"; + let edit2 = if paragraph { text::replace_block(&mut doc, 0, 0, next) } else { text::replace_line(&mut doc, 0, 0, next) }; + assert_eq!(edit2.unwrap().substituted, None, "{typed:?}"); + let doc = reopen(&doc); + let lines = text::text_lines(&doc, 0).unwrap(); + assert_eq!(lines[0].text, next, "{typed:?}"); + assert!((lines[0].rect[0] - 72.0).abs() < 0.01, "{typed:?}: {:?}", lines[0].rect); + let content = String::from_utf8_lossy(&page_content_bytes(&doc, 0)).into_owned(); + assert_eq!(content.matches("[-330] TJ").count(), 5, "{content}"); + } + } + + // Styled multi-row path: untouched calibration and row breaks keep 100 Tz and every space. + let mut doc = text_page("BT /F2 12 Tf 72 700 Td (ab) Tj ET"); + assert_eq!(text::replace_block(&mut doc, 0, 0, " ab ba\nba ab").unwrap().substituted, None); + let mut doc = reopen(&doc); + assert_eq!(text::text_blocks(&doc, 0).unwrap()[0].editor_text, " ab ba\nba ab"); + assert_eq!(text::replace_block(&mut doc, 0, 0, " ab ba\nba ab").unwrap().substituted, None); + let doc = reopen(&doc); + let content = String::from_utf8_lossy(&page_content_bytes(&doc, 0)).into_owned(); + let scales: Vec<&str> = content.lines().filter(|l| l.ends_with(" Tz")).collect(); + assert!(scales.iter().all(|l| *l == "100 Tz"), "{scales:?}: {content}"); + let lines: Vec = text::text_lines(&doc, 0).unwrap().into_iter().map(|l| l.text).collect(); + assert_eq!(lines, [" ab ba", "ba ab"]); + assert_eq!(text::text_blocks(&doc, 0).unwrap()[0].editor_text, " ab ba\nba ab"); + + // Two subsets of the same face (/F2 holds 'a' only, /F3 holds 'b' only, neither has ' '): + // spaces adopt their neighbour's subset on the row so no space is stranded in a glyph-less run. + let mut doc = text_page("BT /F2 12 Tf 72 700 Td (a) Tj /F1 12 Tf ( ) Tj /F3 12 Tf (b) Tj ET"); + doc.update_dict(pdfcraft_cos::ObjRef::new(6, 0), |f| { + f.set(b"FirstChar".to_vec(), Object::Int(97)); + f.set(b"LastChar".to_vec(), Object::Int(97)); + f.set(b"Widths".to_vec(), Object::Array(vec![Object::Int(500)])); + }) + .unwrap(); + doc.update_dict(pdfcraft_cos::ObjRef::new(7, 0), |f| { + f.set(b"FirstChar".to_vec(), Object::Int(98)); + f.set(b"LastChar".to_vec(), Object::Int(98)); + f.set(b"Widths".to_vec(), Object::Array(vec![Object::Int(520)])); + }) + .unwrap(); + assert_eq!(text::replace_block(&mut doc, 0, 0, " b a \n a b ").unwrap().substituted, None); + let doc = reopen(&doc); + let lines: Vec = text::text_lines(&doc, 0).unwrap().into_iter().map(|l| l.text).collect(); + assert_eq!(lines, [" b a ", " a b "]); + assert_eq!(text::text_blocks(&doc, 0).unwrap()[0].editor_text, " b a \n a b "); + + // A RangeStyle covering only spaces in a subset without a space glyph joins an adjacent + // glyph-bearing segment so its space count survives save/reopen. + let mut doc = text_page("BT /F2 12 Tf 72 700 Td (ab) Tj ET"); + let style = text::BlockStyle { + ranges: vec![text::RangeStyle { + range: 2..5, + family: None, + resource: None, + size: None, + color: Some([1.0, 0.0, 0.0]), + underline: None, + char_spacing: None, + scale: None, + }], + ..Default::default() + }; + assert_eq!(text::rewrite_block(&mut doc, 0, 0, Some("ab ba"), &style).unwrap().substituted, None); + let doc = reopen(&doc); + assert_eq!(text::text_lines(&doc, 0).unwrap()[0].text, "ab ba"); +} + +/// A space that joins a neighbour's subset for `/ActualText` keeps the underline of its own +/// style, not the neighbour's, and still reads back as that space. +#[test] +fn a_reassigned_subset_space_keeps_its_own_underline() { + let mut doc = text_page("BT /F2 12 Tf 72 700 Td (ab) Tj ET"); + let style = text::BlockStyle { + ranges: vec![text::RangeStyle { + range: 2..5, + family: None, + resource: None, + size: None, + color: None, + underline: Some(true), + char_spacing: None, + scale: None, + }], + ..Default::default() + }; + assert_eq!(text::rewrite_block(&mut doc, 0, 0, Some("ab ba"), &style).unwrap().substituted, None); + let doc = reopen(&doc); + let line = text::text_lines(&doc, 0).unwrap().into_iter().next().expect("a line"); + assert_eq!(line.text, "ab ba"); + let content = String::from_utf8_lossy(&page_content_bytes(&doc, 0)).into_owned(); + let mut widths = Vec::new(); + let mut from = None; + for row in content.lines() { + if let Some(rest) = row.strip_suffix(" m") { + from = rest.split_whitespace().next().and_then(|s| s.parse::().ok()); + } else if let Some(rest) = row.strip_suffix(" l") + && let (Some(x0), Some(x1)) = (from.take(), rest.split_whitespace().next().and_then(|s| s.parse::().ok())) + { + widths.push((x1 - x0).abs()); + } + } + let text_width = line.rect[2] - line.rect[0]; + assert!(widths.iter().any(|w| *w > 1.0), "the spaces lost their underline: {widths:?}\n{content}"); + assert!( + widths.iter().all(|w| *w < text_width * 0.6), + "the space underline covered the neighbouring glyphs ({text_width}): {widths:?}\n{content}" + ); +} + +/// #6131: a subset that writes width 0 for its unused space (pdfTeX does) has no space glyph. A word added to +/// the line takes a positioning gap for its spaces, not code 32 at width 0, which would run the words together. +#[test] +fn a_space_in_a_subset_that_gives_it_no_width_is_a_gap_not_code_32() { + for subtype in ["TrueType", "Type1"] { + // Codes 32..=98: the space and everything else 0 wide, but a and b (97, 98). + let widths: Vec = (32..=98).map(|c| if c == 97 || c == 98 { "500" } else { "0" }.to_string()).collect(); + let font = format!( + "<< /Type /Font /Subtype /{subtype} /BaseFont /ABCDEF+Sample /FirstChar 32 /LastChar 98 /Widths [{}] /Encoding /WinAnsiEncoding >>", + widths.join(" ") + ); + let content = "BT /F1 12 Tf 72 700 Td (ab) Tj ET"; + let objs = [ + "<< /Type /Catalog /Pages 2 0 R >>".to_string(), + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_string(), + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 600 800] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >>".to_string(), + format!( + "<< /Length {} >> +stream +{content} +endstream", + content.len() + ), + font, + ]; + let mut out = b"%PDF-1.7 +" + .to_vec(); + let mut offs = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offs.push(out.len()); + out.extend_from_slice( + format!( + "{} 0 obj +{o} +endobj +", + i + 1 + ) + .as_bytes(), + ); + } + let x = out.len(); + out.extend_from_slice( + format!( + "xref +0 {} +0000000000 65535 f +", + objs.len() + 1 + ) + .as_bytes(), + ); + for o in offs { + out.extend_from_slice( + format!( + "{o:010} 00000 n +" + ) + .as_bytes(), + ); + } + out.extend_from_slice( + format!( + "trailer +<< /Size {} /Root 1 0 R >> +startxref +{x} +%%EOF +", + objs.len() + 1 + ) + .as_bytes(), + ); + let mut doc = Document::open(Arc::new(out)).unwrap(); + assert_eq!(text::replace_block(&mut doc, 0, 0, "ab ba").unwrap().substituted, None, "{subtype}"); + // Every byte the text-showing operators draw (strings inside `TJ` arrays too) and every + // `TJ` adjustment, read from the saved content rather than matched as text. + let ops = pdfcraft_content::parse(&page_content_bytes(&reopen(&doc), 0)).ops; + let mut shown: Vec = Vec::new(); + let mut adjustments: Vec = Vec::new(); + // How many bytes had been drawn when each adjustment came: the gap must sit between the words. + let mut positions: Vec = Vec::new(); + for op in ops.iter().filter(|o| matches!(o.op.as_slice(), b"Tj" | b"TJ" | b"'" | b"\"")) { + for operand in &op.operands { + match operand { + Object::String(s) => shown.extend_from_slice(&s.bytes), + Object::Array(items) => { + for item in items { + match item { + Object::String(s) => shown.extend_from_slice(&s.bytes), + number => { + if let Some(adjustment) = number.as_f64() { + adjustments.push(adjustment); + positions.push(shown.len()); + } + } + } + } + } + _ => {} + } + } + } + assert_eq!(shown, b"abba", "{subtype}: the words are drawn and no code 32 at width 0 between them"); + assert!(matches!(adjustments.as_slice(), [gap] if *gap < 0.0), "{subtype}: one adjustment moves the second word right: {adjustments:?}"); + assert_eq!(positions, [2], "{subtype}: the gap is between the words, after \"ab\""); + } } #[test] @@ -11016,6 +12021,38 @@ fn q_inside_a_text_object_restores_the_text_matrices() { } } +#[test] +fn variation_public_line_and_block_edits_preserve_selectors_after_reopen() { + let replacement = "1\u{FE0F}"; + let mut results = Vec::new(); + for kind in ["line", "block"] { + let mut doc = text_page("BT /F2 12 Tf 72 700 Td (ab) Tj ET"); + if kind == "line" { + text::replace_line(&mut doc, 0, 0, replacement).unwrap(); + } else { + text::replace_block(&mut doc, 0, 0, replacement).unwrap(); + } + let lines = text::text_lines(&reopen(&doc), 0).unwrap(); + results.push((kind, lines[0].text.clone())); + } + assert_eq!(results, vec![("line", replacement.to_owned()), ("block", replacement.to_owned())]); +} + +#[test] +fn variation_public_selector_does_not_shift_centered_text_with_character_spacing() { + // қ requires the same bundled fallback for both drafts, without optional font inputs. + for spacing in [0.0, 2.0] { + let mut plain = text_page("BT /F2 12 Tf 72 700 Td (ab) Tj ET"); + let mut selected = plain.clone(); + let style = text::BlockStyle { width: Some(100.0), align: Some(added::Align::Center), char_spacing: Some(spacing), ..Default::default() }; + text::rewrite_block(&mut plain, 0, 0, Some("1қ"), &style).unwrap(); + text::rewrite_block(&mut selected, 0, 0, Some("1\u{FE0F}қ"), &style).unwrap(); + let a = text::text_lines(&reopen(&plain), 0).unwrap(); + let b = text::text_lines(&reopen(&selected), 0).unwrap(); + assert_eq!(a[0].base_font, b[0].base_font); + assert!(close(a[0].rect, b[0].rect), "Tc={spacing}: plain {:?}, selector {:?}", a[0].rect, b[0].rect); + } +} /// Whole-paragraph alignment retains both soft wraps and typed hard breaks after save/reopen. /// Selected-row alignment for existing PDF content is withheld until it can persist safely. #[test] @@ -13657,3 +14694,84 @@ fn editing_legacy_thai_keeps_the_embedded_face_weight_size_spacing_and_paint_aft assert_eq!(page_font_programs(&doc), fonts, "no replacement font program may be added"); } } + +/// The Unicode row breaks are invisible moves, not glyphs. A draft carrying them (text pasted +/// from a browser or Word) breaks the row on both rewrite paths, rather than being refused +/// because no fallback face has an outline for U+2028 (#6272). The paragraph is one wide row, so +/// a break that was dropped would leave the words on one row instead of two. A one-line edit has +/// no next row, so there each break is written as a space. +#[test] +fn unicode_row_breaks_break_the_row_instead_of_needing_a_glyph() { + let red = text::RangeStyle { + range: 0..3, + family: None, + resource: None, + size: None, + color: Some([1.0, 0.0, 0.0]), + underline: None, + char_spacing: None, + scale: None, + }; + for separator in ['\u{2028}', '\u{2029}', '\u{0085}', '\u{000B}', '\u{000C}', '\u{001C}', '\u{001D}', '\u{001E}'] { + let draft = format!("one{separator}two"); + for style in [ + text::BlockStyle::default(), + text::BlockStyle { ranges: vec![red.clone()], ..Default::default() }, + text::BlockStyle { ranges: vec![text::RangeStyle { range: 0..draft.len(), ..red.clone() }], ..Default::default() }, + text::BlockStyle { bold: Some(true), ..Default::default() }, + ] { + let mut doc = text_page("BT /F1 12 Tf 72 700 Td (Hello world) Tj ET"); + text::rewrite_block(&mut doc, 0, 0, Some(&draft), &style).unwrap_or_else(|e| panic!("U+{:04X}: {e}", u32::from(separator))); + let doc = reopen(&doc); + let lines: Vec = text::text_lines(&doc, 0).unwrap().into_iter().map(|l| l.text).collect(); + assert_eq!(lines, ["one", "two"], "U+{:04X}", u32::from(separator)); + } + let mut doc = text_page("BT /F1 12 Tf 72 700 Td (Hello world) Tj ET"); + text::replace_line(&mut doc, 0, 0, &draft).unwrap_or_else(|e| panic!("U+{:04X}: {e}", u32::from(separator))); + let doc = reopen(&doc); + let lines: Vec = text::text_lines(&doc, 0).unwrap().into_iter().map(|l| l.text).collect(); + assert_eq!(lines, ["one two"], "a one-line edit keeps U+{:04X} on its line", u32::from(separator)); + } +} + +/// Thai text on the styled path has its own coverage checks and row splits. A Unicode row break +/// in a Thai draft breaks the row there too, whether the paragraph keeps its legacy subset, takes +/// the bundled Thai face by choice or is completed by the matching one, instead of being refused +/// for a glyph no face has (#6272). +#[test] +fn thai_paragraphs_break_the_row_on_unicode_row_breaks() { + let rows = |doc: &Document| -> Vec { text::text_lines(&reopen(doc), 0).unwrap().into_iter().map(|l| l.text).collect() }; + let thai_face = pdfcraft_fonts::document_thai_font().is_some(); + if !thai_face { + eprintln!("thai_paragraphs_break_the_row_on_unicode_row_breaks: built without craft-fonts, so only the legacy subset is checked"); + } + for separator in ['\u{2028}', '\u{001C}'] { + let draft = format!("กข{separator}ค"); + let mut doc = text_page("BT /F1 16 Tf 50 650 Td <00d6> Tj ET"); + let font = legacy_thai_font(&mut doc, "ABCDEF+THSarabunPSK", &legacy_thai_program(), LEGACY_THAI_UNICODE); + doc.set(pdfcraft_cos::ObjRef::new(5, 0), Object::Dict(font)); + let style = BlockStyle { width: Some(400.0), ..Default::default() }; + text::rewrite_block(&mut doc, 0, 0, Some(&draft), &style).unwrap_or_else(|e| panic!("legacy U+{:04X}: {e}", u32::from(separator))); + assert_eq!(rows(&doc), ["กข", "ค"], "legacy U+{:04X}", u32::from(separator)); + if !thai_face { + continue; + } + let mut doc = text_page("BT /F1 12 Tf 72 700 Td (Hello world) Tj ET"); + let chosen = BlockStyle { resource: Some("builtin:thai".into()), ..Default::default() }; + text::rewrite_block(&mut doc, 0, 0, Some(&draft), &chosen).unwrap_or_else(|e| panic!("chosen U+{:04X}: {e}", u32::from(separator))); + assert_eq!(rows(&doc), ["กข", "ค"], "chosen U+{:04X}", u32::from(separator)); + let mut doc = text_page("BT /F1 16 Tf 50 700 Td (A) Tj ET"); + let cmap = doc.add(Object::Stream(Stream::flate( + Dict::new(), + b"1 begincodespacerange <00> endcodespacerange 1 beginbfchar <41> <0E19> endbfchar", + ))); + doc.update_dict(pdfcraft_cos::ObjRef::new(5, 0), |f| { + f.set(b"BaseFont".to_vec(), Object::name("ABCDEF+NotoSansThai-Regular")); + f.set(b"ToUnicode".to_vec(), Object::Ref(cmap)); + }) + .unwrap(); + text::rewrite_block(&mut doc, 0, 0, Some(&draft), &BlockStyle::default()) + .unwrap_or_else(|e| panic!("matching U+{:04X}: {e}", u32::from(separator))); + assert_eq!(rows(&doc), ["กข", "ค"], "matching U+{:04X}", u32::from(separator)); + } +} diff --git a/crates/edit/src/text.rs b/crates/edit/src/text.rs index 06a8b62646..ada90d8484 100644 --- a/crates/edit/src/text.rs +++ b/crates/edit/src/text.rs @@ -25,7 +25,7 @@ mod regions; #[path = "text_session.rs"] mod session; pub use fallback::strip_subset_tag; -use regions::{alignment_frame, group_blocks, is_centred, rewrite_styled_block}; +use regions::{alignment_frame, group_blocks, is_centred, rewrite_changed_words, rewrite_styled_block}; pub use session::TextSession; mod blocks; @@ -36,13 +36,13 @@ mod interp; mod lines; mod rewrite; -use blocks::paragraph_separator; #[cfg(test)] use blocks::wrap; pub use blocks::{ BlockKey, BlockStyle, RangeStyle, ReadingBlocks, TextBlock, block_overlapping, clear_block, reading_blocks, reading_blocks_checked, replace_block, text_blocks, }; +use blocks::{kept_words, paragraph_separator, row_break}; pub use editor::{TextEditLayout, TextEditorFont, register_text_font, text_edit_layout, text_editor_fonts}; use editor::{builtin_thai_choice, layout_draft}; #[cfg(test)] @@ -55,17 +55,17 @@ use fallback::{ use fallback::{add_type3_font, is_type3_fallback}; pub(crate) use fallback::{glyph_path, type3_font_dict, unique_font_name}; use interp::{Origin, Paint, union}; -use lines::page_dict; pub(crate) use lines::{ ContentStream, FormBudget, Read, content_streams, content_streams_within, dict_in_place, drawn_before_damage, first_op_after_damage, form_call, joined, resource_dict, rewritten_contents, splice, }; pub use lines::{MAX_FORM_DEPTH, MAX_FORM_VISITS, glyph_text_lines, text_lines}; +use lines::{page_dict, word_lines}; pub use rewrite::{LineEdit, replace_line, rewrite_block}; use rewrite::{coincident_ops, rewrite_block_from}; +pub(crate) use rewrite::{paragraph_ops, reveal_ops, selectable_blocks, show_starts, translate_blocks}; #[cfg(test)] pub(crate) use rewrite::{replace_line_with_limit, rewrite_block_with_limit}; -pub(crate) use rewrite::{reveal_ops, selectable_blocks, translate_blocks}; /// One line of existing text. #[derive(Clone, Debug, PartialEq)] @@ -155,6 +155,66 @@ impl TextLine { mod tests { use super::*; + /// The clip regions reading uses (#3503): intersected regions keep only what both cover, + /// regions apart leave nothing, turned ones keep their shape, and touching is not showing. + #[test] + fn clip_regions_intersect_and_show_what_they_cover() { + use interp::{intersect, region, shows}; + let square = |x: f64, y: f64| region(&Matrix::IDENTITY, [x, y, x + 10.0, y + 10.0]).unwrap(); + let area = |p: &[[f64; 2]]| p.iter().zip(p.iter().cycle().skip(1)).map(|(a, b)| a[0] * b[1] - b[0] * a[1]).sum::().abs() / 2.0; + // Squares overlapping by half each way share a quarter. + let both = intersect(&square(0.0, 0.0), &square(5.0, 5.0)); + assert!((area(&both) - 25.0).abs() < 1e-9, "{both:?}"); + assert!(shows([6.0, 6.0, 7.0, 7.0], &both) && !shows([1.0, 1.0, 2.0, 2.0], &both)); + // Squares apart share nothing, so nothing shows, not even a box across both. + let none = intersect(&square(0.0, 0.0), &square(20.0, 0.0)); + assert!(!shows([-100.0, -100.0, 100.0, 100.0], &none), "{none:?}"); + // Squares that meet only along an edge share no area either: a box across that edge + // does not show (#5075 review). + let edge = intersect(&square(0.0, 0.0), &square(10.0, 0.0)); + assert!(!shows([8.0, 2.0, 12.0, 4.0], &edge), "{edge:?}"); + // Nothing intersected with anything is nothing, whichever side it is on. + assert!(!shows([1.0, 1.0, 2.0, 2.0], &intersect(&square(0.0, 0.0), &Vec::new()))); + assert!(!shows([1.0, 1.0, 2.0, 2.0], &intersect(&Vec::new(), &square(0.0, 0.0)))); + // A box touching an edge shares no area; one a hair inside does. + assert!(!shows([10.0, 0.0, 12.0, 5.0], &square(0.0, 0.0))); + assert!(shows([9.99, 0.0, 12.0, 5.0], &square(0.0, 0.0))); + // A square turned 45° keeps its diamond shape: a box in the corner of its bounding box is + // outside it, the centre is inside. + let c = std::f64::consts::FRAC_1_SQRT_2; + let diamond = region(&Matrix([c, c, -c, c, 0.0, 0.0]), [0.0, 0.0, 10.0, 10.0]).unwrap(); + assert!(!shows([-7.0, 0.0, -6.5, 0.5], &diamond) && shows([-0.5, 6.5, 0.5, 7.5], &diamond)); + // A corner repeated (an edge of no length) does not hide what the region covers. + let doubled = vec![[0.0, 0.0], [10.0, 0.0], [10.0, 0.0], [10.0, 10.0], [0.0, 10.0]]; + assert!(shows([4.0, 4.0, 6.0, 6.0], &doubled)); + // A rectangle of no area is a region that covers nothing; a corner that isn't finite + // gives no region to clip with at all. + let flat = region(&Matrix::IDENTITY, [0.0, 0.0, 10.0, 0.0]).unwrap(); + assert!(!shows([-100.0, -100.0, 100.0, 100.0], &flat), "{flat:?}"); + assert!(region(&Matrix::IDENTITY, [0.0, 0.0, f64::INFINITY, 10.0]).is_none()); + // However thin, a rectangle with area keeps it, and so does one far from the origin. + let thin = region(&Matrix::IDENTITY, [0.0, 0.0, 1.0, 1e-10]).unwrap(); + assert!(shows([0.25, 0.0, 0.75, 1.0], &thin) && !shows([0.25, 1.0, 0.75, 2.0], &thin)); + let far = region(&Matrix([1.0, 0.0, 0.0, 1.0, 1e12, 1e12]), [0.0, 0.0, 100.0, 100.0]).unwrap(); + assert!(shows([1e12 + 10.0, 1e12 + 10.0, 1e12 + 20.0, 1e12 + 20.0], &far) && !shows([1e12 + 200.0, 1e12, 1e12 + 300.0, 1e12 + 10.0], &far)); + } + + /// A stream that sets clip after clip, each turned a little more, keeps its region bounded: + /// past the corner cap a region becomes its bounding box, which still holds all it held. + #[test] + fn clip_after_clip_keeps_the_region_bounded() { + use interp::{intersect, region, shows}; + let mut clip = region(&Matrix::IDENTITY, [-100.0, -100.0, 100.0, 100.0]).unwrap(); + for i in 1..=10_000 { + let (s, c) = (f64::from(i) * 0.001).sin_cos(); + let turned = region(&Matrix([c, s, -s, c, 0.0, 0.0]), [-60.0, -60.0, 60.0, 60.0]).unwrap(); + clip = intersect(&clip, &turned); + assert!(clip.len() <= 32 && clip.iter().flatten().all(|v| v.is_finite()), "clip {i}: {} corners", clip.len()); + } + // Every square is centred on the origin, so the middle still shows. + assert!(shows([-1.0, -1.0, 1.0, 1.0], &clip)); + } + fn block(text: &str, rect: [f64; 4]) -> TextBlock { TextBlock { text: text.into(), @@ -277,6 +337,18 @@ mod tests { assert_eq!(wrapped("一行目　二行目", 4.0), ["一行目　", "二行目"]); } + #[test] + fn thai_text_wraps_between_dictionary_words_and_keeps_marks_on_their_word() { + // Without word breaks the whole run is one word and overflows a single row. + assert_eq!(wrapped("ภาษาไทยไม่มีช่องว่าง", 8.0), ["ภาษาไทย", "ไม่มี", "ช่องว่าง"]); + // Mai Yamok stays with the word it repeats; pieces join without a space, words keep theirs. + assert_eq!(wrapped("เด็กๆชอบกิน ขนม", 6.0), ["เด็กๆ", "ชอบกิน", "ขนม"]); + assert_eq!(wrapped("ไฟล์ PDF ประปาส่วนภูมิภาค", 9.0), ["ไฟล์ PDF", "ประปาส่วน", "ภูมิภาค"]); + // A wrap between Thai words consumed no separator, so a rejoin puts back no space. + let rows = wrap("ภาษาไทยไม่มีช่องว่าง", 8.0, |s| s.chars().count() as f64); + assert_eq!(rows.iter().map(|(_, soft, gap)| (*soft, *gap)).collect::>(), [(false, 0), (true, 0), (true, 0)]); + } + #[test] fn fallback_face_moves_on_when_the_best_face_lacks_a_character() { static JPAN: &[&str] = &["Jpan"]; diff --git a/crates/edit/src/text/blocks.rs b/crates/edit/src/text/blocks.rs index dcebb050dd..b714bcd592 100644 --- a/crates/edit/src/text/blocks.rs +++ b/crates/edit/src/text/blocks.rs @@ -44,6 +44,12 @@ impl TextBlock { (self.invisible && self.size.is_finite() && self.size > 0.0).then_some(self.size) } + /// Whether `draft` says what the paragraph says, give or take space around it. Spacing inside + /// it has to match: a typed second space is a change. + pub fn same_words(&self, draft: &str) -> bool { + draft.trim() == self.editor_text.trim() + } + /// Preserve styles outside the changed ranges; inserted characters inherit the surrounding /// source run. The texts are aligned word by word, so words left untouched between two edits /// keep their own runs (#2198); past [`MAX_CHANGED_WORDS`] the draft counts as one change. @@ -163,6 +169,23 @@ fn edit_script(a: &[T], b: &[T], max: usize) -> Option> None } +/// For each of `old`'s words, the index in `new` of the word it stays as in the shortest edit from +/// one to the other (`None` when it goes), or `None` overall past [`MAX_CHANGED_WORDS`] changes. +pub(crate) fn kept_words(old: &[T], new: &[T]) -> Option>> { + let mut kept = vec![None; old.len()]; + let (mut o, mut n) = (0usize, 0usize); + for step in edit_script(old, new, MAX_CHANGED_WORDS)? { + if step == Step::Keep + && let Some(slot) = kept.get_mut(o) + { + *slot = Some(n); + } + o = o.saturating_add(usize::from(step != Step::Insert)); + n = n.saturating_add(usize::from(step != Step::Delete)); + } + Some(kept) +} + /// Walk Myers' saved frontiers back from the end to the edit script. fn backtrack(trace: &[Vec], n: isize, m: isize, max: isize) -> Vec { let at = |v: &[isize], k: isize| usize::try_from(k + max + 1).ok().and_then(|i| v.get(i).copied()).unwrap_or(0); @@ -344,6 +367,15 @@ pub(crate) fn paragraph_separator(c: char) -> bool { c.is_whitespace() && c != '\u{3000}' } +/// Characters that end a row rather than draw a glyph: the Unicode mandatory breaks (UAX #14 +/// class BK: vertical tab, form feed, U+2028 LINE SEPARATOR, U+2029 PARAGRAPH SEPARATOR), the +/// next line (U+0085), the information separators (bidi class B) and CR/LF. Text pasted from a +/// browser or Word carries these, so a rewrite moves to the next row instead of asking a font +/// for an outline no face has (#6272). +pub(crate) fn row_break(c: char) -> bool { + matches!(c, '\n' | '\r' | '\u{000B}' | '\u{000C}' | '\u{0085}' | '\u{001C}'..='\u{001E}' | '\u{2028}' | '\u{2029}') +} + /// Greedy word wrapping to `width` with `advance` giving a string's width. Each `\n` starts a /// new line. A row may also end inside a word of Chinese or Japanese text ([`crate::linebreak`]); /// those pieces join without a space. Every white-space character is written as one space, so @@ -435,6 +467,13 @@ pub struct BlockStyle { /// For an invisible paragraph ([`TextBlock::invisible`]): fill its old area with this colour /// (RGB 0–1) before the new text, hiding the scanned word beneath it. pub cover: Option<[f64; 3]>, + /// For an invisible paragraph whose formatting nobody asked to change: cover and redraw only + /// the words the draft changes, and leave the rest as the scan shows them (#3313). The session + /// sets this; a caller that sets it should also give `cover`. + pub changed_words_only: bool, + /// How far across [`BlockStyle::cover`] may reach, whatever its padding: the edges of the words + /// on either side that stay (user space). + pub cover_between: Option<[f64; 2]>, } #[derive(Clone, Debug, PartialEq)] diff --git a/crates/edit/src/text/fallback.rs b/crates/edit/src/text/fallback.rs index 80b20e0cda..f7c795ca65 100644 --- a/crates/edit/src/text/fallback.rs +++ b/crates/edit/src/text/fallback.rs @@ -5,7 +5,27 @@ use super::{ }; /// Text → the runs that show it: each run's font resource name and the bytes shown in that font. -pub(super) type Encoder = Box Option)>>>; +pub(super) type Encoder = Box Option>>; + +/// One run a rewrite shows: bytes drawn in one generated font. `spacing` set (a variation +/// selector's run) shows at that character spacing: its empty glyph advances nothing, and the +/// spacing in force must not add to it either. +#[derive(Clone, Debug, PartialEq)] +pub(super) struct Type3Run { + pub(super) name: String, + pub(super) bytes: Vec, + pub(super) spacing: Option, +} + +/// A Unicode variation selector (U+FE00..U+FE0F): it picks a variant of the character before it +/// and has no glyph of its own. The Type 3 fallback gives it an empty zero-width glyph instead of +/// failing the edit on it. +pub(super) fn is_variation_selector(ch: char) -> bool { + matches!(u32::from(ch), 0xFE00..=0xFE0F) +} + +/// A variation selector's glyph procedure: nothing painted, zero advance. +const SELECTOR_GLYPH_PROC: &[u8] = b"0 0 0 0 0 0 d1\n"; fn is_win_ansi_char(c: char) -> bool { c == '\t' || pdfcraft_fonts::win_ansi_byte(c).is_some() @@ -377,7 +397,13 @@ pub(super) fn type3_font( // The faces differ in coverage (e.g. of Cyrillic), so use the best face that has every // character. When none has them all, the best face reports the character it lacks. Hebrew // text keeps the Hebrew face and takes each character it lacks from the next face that has it. - let face = if hebrew { faces.first().copied() } else { fallback_face(&faces, &chars, |face, ch| japanese_glyph_from(face, ch).is_ok()) } + // A variation selector has no outline anywhere: it takes a glyph slot of its own below, so the + // face is chosen by the characters that have outlines. + let named: Vec = chars.iter().copied().filter(|ch| !is_variation_selector(*ch)).collect(); + if named.is_empty() { + return Err(EditError::Invalid("replacement text is empty".into())); + } + let face = if hebrew { faces.first().copied() } else { fallback_face(&faces, &named, |face, ch| japanese_glyph_from(face, ch).is_ok()) } .ok_or_else(|| no_fallback_font(text))?; let others: &[&CraftFont] = if hebrew { faces.get(1..).unwrap_or_default() } else { &[] }; // Every outline is read before the document is touched, so a character that fails leaves it @@ -385,15 +411,21 @@ pub(super) fn type3_font( let per_font = max_glyphs.clamp(1, MAX_TYPE3_GLYPHS); let mut glyphs = Vec::with_capacity(chars.len()); for (i, ch) in chars.into_iter().enumerate() { - // The chosen face first, then the faces after it (Hebrew text takes a character the Hebrew - // face lacks from the next face that has it); a character no face draws is reported once. - let mut drawn = type3_glyph(face, ch)?; - for f in others.iter().copied() { - if drawn.is_some() { - break; + let drawn = if is_variation_selector(ch) { + Some((SELECTOR_GLYPH_PROC.to_vec(), 0.0)) + } else { + // The chosen face first, then the faces after it (Hebrew text takes a character the + // Hebrew face lacks from the next face that has it); a character no face draws is + // reported once. + let mut drawn = type3_glyph(face, ch)?; + for f in others.iter().copied() { + if drawn.is_some() { + break; + } + drawn = type3_glyph(f, ch)?; } - drawn = type3_glyph(f, ch)?; - } + drawn + }; let Some((path, width)) = drawn else { return Err(no_glyph_for(face, ch)) }; let code = u8::try_from((i % per_font).saturating_add(1)) .map_err(|_| EditError::Invalid("replacement text has too many unique characters".into()))?; @@ -580,17 +612,22 @@ pub(crate) fn type3_font_dict(doc: &mut Document, glyphs: &[(char, Vec, f64) } /// The runs that show `text` in the fallback: consecutive characters in one font form one run, so -/// a font is switched only where the text moves to another font. `None` if a character has no code. -pub(super) fn type3_encode(fallback: &Type3Fallback, text: &str) -> Option)>> { - let mut runs: Vec<(usize, Vec)> = Vec::new(); +/// a font is switched only where the text moves to another font. A variation selector forms its +/// own run, drawn at character spacing zero (see [`Type3Run`]). `None` if a character has no code. +pub(super) fn type3_encode(fallback: &Type3Fallback, text: &str) -> Option> { + let mut runs: Vec<(usize, Vec, Option)> = Vec::new(); for ch in text.chars() { let (font, code) = *fallback.codes.get(&ch)?; + if is_variation_selector(ch) { + runs.push((font, vec![code], Some(0.0))); + continue; + } match runs.last_mut() { - Some((last, bytes)) if *last == font => bytes.push(code), - _ => runs.push((font, vec![code])), + Some((last, bytes, None)) if *last == font => bytes.push(code), + _ => runs.push((font, vec![code], None)), } } - runs.into_iter().map(|(font, bytes)| Some((fallback.names.get(font)?.clone(), bytes))).collect() + runs.into_iter().map(|(font, bytes, spacing)| Some(Type3Run { name: fallback.names.get(font)?.clone(), bytes, spacing })).collect() } /// The source font's own TrueType/OpenType program, when the document embeds a complete one whose @@ -683,5 +720,35 @@ pub(crate) fn source_show(metrics: &Metrics, text: &str, size: f64, space: f64, out.push(Op::new("Tj", vec![Object::String(PdfString::literal(bytes))])); } } + if out.iter().any(|op| op.is("Tj")) { + let logical = crate::bidi::to_logical(&if metrics.visual_order { crate::indic::to_logical(text) } else { text.to_owned() }); + let mut properties = Dict::new(); + properties.set(b"ActualText".to_vec(), Object::String(PdfString::text(&logical))); + out.insert(0, Op::new("BDC", vec![Object::name("Span"), Object::Dict(properties)])); + out.push(Op::new("EMC", Vec::new())); + } Ok(out) } + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn variation_selectors_show_an_empty_glyph_at_zero_spacing() { + assert!(is_variation_selector('\u{FE0F}')); + assert!(!is_variation_selector('\u{27A1}')); + let fallback = Type3Fallback { + family: "Inter", + names: vec!["PCLat".into()], + widths: vec![('1', 0.556), ('\u{FE0F}', 0.0)], + codes: [('1', (0, 1)), ('\u{FE0F}', (0, 2))].into_iter().collect(), + }; + let runs = type3_encode(&fallback, "1\u{FE0F}").expect("the selector after a drawn base is kept, not an error"); + assert_eq!(runs.len(), 2, "the selector forms its own run"); + assert_eq!(runs[0], Type3Run { name: "PCLat".into(), bytes: vec![1], spacing: None }); + assert_eq!(runs[1], Type3Run { name: "PCLat".into(), bytes: vec![2], spacing: Some(0.0) }, "the selector draws at spacing zero"); + let runs = type3_encode(&fallback, "\u{FE0F}").expect("a lone selector keeps its empty glyph rather than failing"); + assert_eq!(runs, vec![Type3Run { name: "PCLat".into(), bytes: vec![2], spacing: Some(0.0) }]); + } +} diff --git a/crates/edit/src/text/interp.rs b/crates/edit/src/text/interp.rs index 5114a76263..0bc9373899 100644 --- a/crates/edit/src/text/interp.rs +++ b/crates/edit/src/text/interp.rs @@ -218,9 +218,139 @@ pub(crate) fn union(a: [f64; 4], b: [f64; 4]) -> [f64; 4] { [a[0].min(b[0]), a[1].min(b[1]), a[2].max(b[2]), a[3].max(b[3])] } +/// A convex part of the page, in page space: the corners of a convex polygon, in order. With fewer +/// than three corners it covers nothing. +pub(super) type Region = Vec<[f64; 2]>; + +/// A clip, as the union of the convex regions it leaves visible: one for each rectangle of its +/// path. With no parts it leaves nothing visible. +pub(super) type Clip = Vec; + +/// The most corners a clip region keeps. Each intersection with a four-cornered clip can add +/// corners, so past this a region becomes its bounding box: still all it held, and a stream that +/// sets clip after clip stays bounded. +const MAX_REGION_CORNERS: usize = 32; + +/// The most parts a clip keeps. Clips intersect part by part, so past this its parts become one, +/// the box around them: still all they held, and the work stays bounded. +const MAX_CLIP_PARTS: usize = 16; + +/// `r` (`[x0, y0, x1, y1]`, either way round) under `m`, keeping its shape when `m` turns or +/// skews it. A rectangle of no area is a region that covers nothing, however thin a rectangle +/// with area is; `None` when a corner, or its measure, is not finite. +pub(super) fn region(m: &Matrix, r: [f64; 4]) -> Option { + let corners: Region = [(r[0], r[1]), (r[2], r[1]), (r[2], r[3]), (r[0], r[3])] + .into_iter() + .map(|(x, y)| { + let (x, y) = m.apply(x, y); + [x, y] + }) + .collect(); + let finite = corners.iter().flatten().all(|v| v.is_finite()) && twice_area(&corners).is_finite() && perimeter(&corners).is_finite(); + finite.then(|| if encloses_area(&corners) { corners } else { Vec::new() }) +} + +/// The polygon's signed area, doubled: positive when its corners run anticlockwise (y up). +/// Measured from its first corner, so that far from the origin it keeps its precision. +fn twice_area(p: &[[f64; 2]]) -> f64 { + let Some(o) = p.first() else { return 0.0 }; + p.iter().zip(p.iter().cycle().skip(1)).map(|(a, b)| (a[0] - o[0]) * (b[1] - o[1]) - (b[0] - o[0]) * (a[1] - o[1])).sum() +} + +fn perimeter(p: &[[f64; 2]]) -> f64 { + p.iter().zip(p.iter().cycle().skip(1)).map(|(a, b)| (b[0] - a[0]).hypot(b[1] - a[1])).sum() +} + +/// Whether the polygon encloses any area: its area against the square of its perimeter, so the +/// answer is the same at any scale and a thin box still has one. +fn encloses_area(p: &[[f64; 2]]) -> bool { + let around = perimeter(p); + p.len() >= 3 && twice_area(p).abs() > 1e-12 * around * around +} + +/// The four corners of the box around `points`. +fn bounding<'a>(points: impl Iterator) -> Region { + let b = points.fold([f64::MAX, f64::MAX, f64::MIN, f64::MIN], |b, p| [b[0].min(p[0]), b[1].min(p[1]), b[2].max(p[0]), b[3].max(p[1])]); + vec![[b[0], b[1]], [b[2], b[1]], [b[2], b[3]], [b[0], b[3]]] +} + +/// The part of `a` inside `b`, both convex (Sutherland-Hodgman: `a` cut by each edge of `b`). Clips +/// in force intersect, so text shows only where all of them let it through. Regions that share no +/// area (apart, or meeting only along an edge) leave an empty one, as does an empty `a` or `b`. +pub(super) fn intersect(a: &Region, b: &Region) -> Region { + if a.len() < 3 || b.len() < 3 { + return Vec::new(); + } + let turn = twice_area(b).signum(); + let mut out = a.clone(); + for (p, q) in b.iter().zip(b.iter().cycle().skip(1)) { + // Positive on the side of the edge `p` → `q` that `b` lies on. + let inside = |v: &[f64; 2]| turn * ((q[0] - p[0]) * (v[1] - p[1]) - (q[1] - p[1]) * (v[0] - p[0])); + let cut = std::mem::take(&mut out); + for (s, e) in cut.iter().zip(cut.iter().cycle().skip(1)) { + let (ds, de) = (inside(s), inside(e)); + if ds >= 0.0 { + out.push(*s); + } + // The edge crosses the line: `ds - de` is not zero, as one is negative and one is not. + if (ds >= 0.0) != (de >= 0.0) { + let t = ds / (ds - de); + out.push([s[0] + t * (e[0] - s[0]), s[1] + t * (e[1] - s[1])]); + } + } + } + if !encloses_area(&out) { + return Vec::new(); + } + if out.len() > MAX_REGION_CORNERS { + out = bounding(out.iter()); + } + out +} + +/// The clip that leaves visible only what both `a` and `b` do: each part of one intersected with +/// each part of the other. +pub(super) fn clip_within(a: &Clip, b: &Clip) -> Clip { + let mut out: Clip = a.iter().flat_map(|p| b.iter().map(move |q| intersect(p, q))).filter(|r| !r.is_empty()).collect(); + if out.len() > MAX_CLIP_PARTS { + out = vec![bounding(out.iter().flatten())]; + } + out +} + +/// Whether the box `r` shows through `clip`: through any of its parts. +pub(super) fn clip_shows(r: [f64; 4], clip: &Clip) -> bool { + clip.iter().any(|part| shows(r, part)) +} + +/// Whether the box `r` (`[x0, y0, x1, y1]`) and `region` share some area, by the separating axis +/// test: two convex shapes are apart exactly when their shadows on some axis, one of the box's or +/// a normal of one of the region's edges, do not overlap. Touching is not sharing. +pub(super) fn shows(r: [f64; 4], region: &Region) -> bool { + if region.len() < 3 { + return false; + } + let corners = [[r[0], r[1]], [r[2], r[1]], [r[2], r[3]], [r[0], r[3]]]; + let apart = |axis: [f64; 2]| { + let shadow = |points: &[[f64; 2]]| { + points.iter().map(|p| p[0] * axis[0] + p[1] * axis[1]).fold((f64::INFINITY, f64::NEG_INFINITY), |(lo, hi), v| (lo.min(v), hi.max(v))) + }; + let ((a0, a1), (b0, b1)) = (shadow(&corners), shadow(region)); + a1 <= b0 || b1 <= a0 + }; + // An edge of no length (two corners in one place) has no normal to test. + let normals = region.iter().zip(region.iter().cycle().skip(1)).map(|(p, q)| [p[1] - q[1], q[0] - p[0]]).filter(|n| n[0] != 0.0 || n[1] != 0.0); + !([[1.0, 0.0], [0.0, 1.0]].into_iter().chain(normals).any(apart)) +} + #[derive(Clone)] pub(super) struct Ts { pub(super) ctm: Matrix, + /// The clip in force, if any, as what it leaves visible: the intersection of every + /// rectangular content-stream clip (`re` `W`/`W*` `n`, ISO 32000-2 §8.5.4) and every enclosing + /// form's `/BBox` ([`super::lines`]), each in the shape its matrix gives it. `q`/`Q` save and + /// restore it along with the rest of `Ts`. + pub(super) clip: Option, /// The operators that set the current fill colour (`g`, `rg`, `k`, or `cs` + `sc`/`scn`). pub(super) fill: Vec, pub(super) font: Option<(Vec, Rc)>, @@ -492,6 +622,9 @@ pub(super) struct Carry { pub(super) layers: Vec>>, /// Bytes of glyph text the rest of the page may still produce (see [`MAX_PAGE_TEXT`]). pub(super) text_left: usize, + /// Whether the last run ended in a space read from a gap, with the next operator a `Tj` or + /// `TJ` that may start with the space glyph that gap widened. + pub(super) gap_space: bool, } /// The most glyph text one page's extraction produces, in bytes. A ToUnicode destination is @@ -503,6 +636,7 @@ impl Carry { pub(super) fn new() -> Self { let ts = Ts { ctm: Matrix::IDENTITY, + clip: None, fill: Vec::new(), font: None, face: None, @@ -522,7 +656,7 @@ impl Carry { dash: (Vec::new(), 0.0), line_cap: 0, }; - Carry { ts, stack: Vec::new(), marked: Vec::new(), layers: Vec::new(), text_left: MAX_PAGE_TEXT } + Carry { ts, stack: Vec::new(), marked: Vec::new(), layers: Vec::new(), text_left: MAX_PAGE_TEXT, gap_space: false } } } @@ -567,32 +701,34 @@ impl MarkedContent { self.layers.pop(); if let Some(Some((start, open, logical))) = self.actual_text.pop() { let tail = shown.split_off(start.min(shown.len())); - let mut items = tail.into_iter(); - if let Some(mut first) = items.next() { - for item in items { - first.glyph_count = first.glyph_count.saturating_add(item.glyph_count); - first.extra_ops.push(item.op); - first.extra_ops.extend(item.extra_ops); - first.rect = [ - first.rect[0].min(item.rect[0]), - first.rect[1].min(item.rect[1]), - first.rect[2].max(item.rect[2]), - first.rect[3].max(item.rect[3]), - ]; - first.frame = union(first.frame, item.frame); - first.ink.extend(item.ink); + if tail.iter().any(|s| s.glyph_count > 0) { + let mut items = tail.into_iter(); + if let Some(mut first) = items.next() { + for item in items { + first.glyph_count = first.glyph_count.saturating_add(item.glyph_count); + first.extra_ops.push(item.op); + first.extra_ops.extend(item.extra_ops); + first.rect = [ + first.rect[0].min(item.rect[0]), + first.rect[1].min(item.rect[1]), + first.rect[2].max(item.rect[2]), + first.rect[3].max(item.rect[3]), + ]; + first.frame = union(first.frame, item.frame); + first.ink.extend(item.ink); + } + first.extra_ops.extend([open, index]); + // Only right-to-left ActualText is already in typing order and must stay out of + // bidi recovery. Word and Chrome also wrap Latin ligatures mid-word. + first.logical_order = logical.chars().any(crate::bidi::is_rtl); + first.actual_text = true; + first.text = logical; + first.start_x = first.rect[0]; + first.end_x = first.rect[2]; + first.end_y = first.baseline; + first.advance = (first.end_x - first.start_x) / first.tm.then(&first.state.ctm).0[0].abs().max(1e-6); + shown.push(first); } - first.extra_ops.extend([open, index]); - // Only right-to-left ActualText is already in typing order and must stay out of - // bidi recovery. Word and Chrome also wrap Latin ligatures mid-word. - first.logical_order = logical.chars().any(crate::bidi::is_rtl); - first.actual_text = true; - first.text = logical; - first.start_x = first.rect[0]; - first.end_x = first.rect[2]; - first.end_y = first.baseline; - first.advance = (first.end_x - first.start_x) / first.tm.then(&first.state.ctm).0[0].abs().max(1e-6); - shown.push(first); } } } @@ -654,6 +790,9 @@ fn show_text(op: &Op, i: usize, state: &mut TextRunState<'_>) -> Result<(), Edit // the next glyph continues the last word. let mut words: Vec<(f64, f64)> = Vec::new(); let mut in_word = false; + // Whether the text ends in a space read from a gap rather than from a glyph. The run just + // before may have ended in one: a justified line can be split over `TJ` and `Tj`. + let mut gap_space = std::mem::take(&mut carry.gap_space); let mut glyph_count = 0_usize; for p in &pieces { match p { @@ -662,13 +801,17 @@ fn show_text(op: &Op, i: usize, state: &mut TextRunState<'_>) -> Result<(), Edit glyph_count = glyph_count.saturating_add(1); let shown = m.text_of(code); let blank = m.is_space(code, len) || shown.as_deref().is_some_and(|t| !t.is_empty() && t.chars().all(char::is_whitespace)); + // A space glyph right after a gap that read as a space is that space: + // LibreOffice stretches a justified line on both sides of each space glyph. + let after_gap = std::mem::take(&mut gap_space); match shown { - Some(t) => { + Some(shown) => { + let t: &str = if after_gap { shown.strip_prefix(' ').unwrap_or(&shown) } else { &shown }; carry.text_left = carry .text_left .checked_sub(t.len()) .ok_or_else(|| EditError::Invalid("the page's text is too large to read".into()))?; - text.push_str(&t); + text.push_str(t); } None => decodable = false, } @@ -709,6 +852,7 @@ fn show_text(op: &Op, i: usize, state: &mut TextRunState<'_>) -> Result<(), Edit let dy = -n / 1000.0 * ts.size; if -dy > ts.size * 0.2 && !text.ends_with(' ') { text.push(' '); + gap_space = true; } if dy.abs() > ts.size.abs() * 0.2 { in_word = false; @@ -720,6 +864,7 @@ fn show_text(op: &Op, i: usize, state: &mut TextRunState<'_>) -> Result<(), Edit // A large gap inside TJ reads as a space. if dx - tracking > (ts.size * ts.scale).abs() * 0.2 && !text.ends_with(' ') { text.push(' '); + gap_space = true; } // Tracking and kerning keep a word together; a larger jump // ends it (whatever the sign of the size or `Tz`). @@ -761,8 +906,11 @@ fn show_text(op: &Op, i: usize, state: &mut TextRunState<'_>) -> Result<(), Edit .collect(); let size_user = (trm0.0[2].powi(2) + trm0.0[3].powi(2)).sqrt() * ts.size; // A numeric-only TJ adjusts the current text matrix but paints no glyphs. - // It must not create a phantom editable line or enlarge its neighbours' boxes. - if !pieces.iter().any(|p| matches!(p, Object::String(s) if !s.bytes.is_empty())) { + // Outside an `/ActualText` replacement it must not create a phantom editable + // line or enlarge its neighbours' boxes; inside one (such as a space gap in a + // subset font without a space glyph, #4229) it is part of the span's box and + // operators when the span also paints glyphs. + if !marked.actual_text.iter().any(Option::is_some) && !pieces.iter().any(|p| matches!(p, Object::String(s) if !s.bytes.is_empty())) { return Ok(()); } out.push(Shown { @@ -810,6 +958,7 @@ fn show_text(op: &Op, i: usize, state: &mut TextRunState<'_>) -> Result<(), Edit hard_break: state.hard_break, soft_gap: state.soft_gap, }); + carry.gap_space = gap_space; Ok(()) } @@ -842,7 +991,57 @@ pub(super) fn interpret( // with no properties dict (a file saved before this was tracked, or by another writer) reads // as 1, the single space a rejoin always used before #2529. let mut soft_gap = 1usize; + // A pending path of `x y w h re`s, and whether a `W`/`W*` has marked it as the path that `n` + // turns into the new clip (ISO 32000-2 §8.5.4) -- the common "set a rectangular clip" idiom. + // The path is `Some(None)` once one of its rectangles can't be read, and then clips nothing. + // Each rectangle is a part of the clip; a path of more than `MAX_CLIP_PARTS` keeps the box + // around them, which still holds them all. Any other operator between `re` and `n` ends the + // idiom: the clip is not read then. + let mut pending_clip: Option>> = None; + let mut clip_pending = false; for (i, op) in ops.iter().enumerate() { + // A gap's space carries only into a `Tj` or `TJ` right after it. + if !matches!(op.op.as_slice(), b"Tj" | b"TJ") { + carry.gap_space = false; + } + match op.op.as_slice() { + b"re" => { + let rect = op.nums::<4>().map(|[x, y, w, h]| [x.min(x + w), y.min(y + h), x.max(x + w), y.max(y + h)]); + pending_clip = Some(match (pending_clip.take(), rect) { + (None, Some(rect)) => Some(vec![rect]), + (Some(Some(mut path)), Some(rect)) => { + if path.len() >= MAX_CLIP_PARTS { + path = vec![path.iter().fold(rect, |all, r| union(all, *r))]; + } else { + path.push(rect); + } + Some(path) + } + _ => None, + }); + } + b"W" | b"W*" => clip_pending = pending_clip.is_some(), + b"n" => { + if clip_pending && let Some(Some(path)) = pending_clip.take() { + // A rectangle past what a matrix can hold leaves the clip unread. One of no + // area is a part that shows nothing, so a clip of no area hides everything. + let parts: Option = path.iter().map(|r| region(&ts.ctm, *r)).collect(); + if let Some(parts) = parts { + let new: Clip = parts.into_iter().filter(|part| !part.is_empty()).collect(); + ts.clip = Some(match &ts.clip { + Some(old) => clip_within(old, &new), + None => new, + }); + } + } + pending_clip = None; + clip_pending = false; + } + _ => { + pending_clip = None; + clip_pending = false; + } + } match op.op.as_slice() { // No depth cap: every `EMC` pops, so a capped push would pair it with the enclosing // sequence (#3573). Both stacks stay bounded by the number of operators. @@ -1006,6 +1205,7 @@ pub(super) fn interpret( carry.stack = stack; carry.marked = marked.regions; carry.layers = marked.layers; + out.retain(|s| s.glyph_count > 0 || s.actual_text); positioned_tracking(&mut out); Ok(out) } diff --git a/crates/edit/src/text/lines.rs b/crates/edit/src/text/lines.rs index 1aac66fde4..9eadb65107 100644 --- a/crates/edit/src/text/lines.rs +++ b/crates/edit/src/text/lines.rs @@ -1,7 +1,7 @@ use std::sync::Arc; use super::fallback::{same_font, same_paint}; -use super::interp::{Carry, Origin, Pen, Shown, TextState, Ts, direction, interpret, to_frame, union}; +use super::interp::{Carry, Clip, Origin, Pen, Shown, TextState, Ts, clip_shows, clip_within, direction, interpret, region, to_frame, union}; use super::{DecodeBudget, Dict, Document, EditError, HashMap, Matrix, Object, Op, Pieces, Stream, TextLine, parse, serialize_ops}; impl TextLine { @@ -291,7 +291,13 @@ fn fill_color(fill: &[Op]) -> [f64; 3] { /// The lines of text on a page (0-based), in content order. pub fn text_lines(doc: &Document, page: usize) -> Result, EditError> { - text_lines_in(doc, &page_dict(doc, page)?, true) + text_lines_in(doc, &page_dict(doc, page)?, true, false) +} + +/// [`text_lines`] with every show of invisible text a line of its own, however alike its +/// neighbours: an OCR layer's words, which an edit can keep or change one by one. +pub(crate) fn word_lines(doc: &Document, page: usize) -> Result, EditError> { + text_lines_in(doc, &page_dict(doc, page)?, true, true) } /// The lines `page`'s glyphs show, in content order, without `/ActualText` replacements: what the @@ -299,11 +305,12 @@ pub fn text_lines(doc: &Document, page: usize) -> Result, EditErro /// not trust that metadata (the redaction audit). Takes the page itself, so a caller going /// through every page doesn't walk the page tree again for each. pub fn glyph_text_lines(doc: &Document, page: &pdfcraft_model::Page) -> Result, EditError> { - text_lines_in(doc, page, false) + text_lines_in(doc, page, false, false) } -/// [`text_lines`] of `p`; with `actual_text`, `/ActualText` replaces the glyphs it spans. -fn text_lines_in(doc: &Document, p: &pdfcraft_model::Page, actual_text: bool) -> Result, EditError> { +/// [`text_lines`] of `p`; with `actual_text`, `/ActualText` replaces the glyphs it spans, and with +/// `invisible_apart`, no show of invisible text joins the one before it. +fn text_lines_in(doc: &Document, p: &pdfcraft_model::Page, actual_text: bool, invisible_apart: bool) -> Result, EditError> { let res = p.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default(); let mut cache = HashMap::new(); let mut lines: Vec = Vec::new(); @@ -345,7 +352,7 @@ fn text_lines_in(doc: &Document, p: &pdfcraft_model::Page, actual_text: bool) -> && near // An OCR layer is one text object over the whole scan: its words continue a line // only across a word space, not across a table cell or a tab stop. - && (s.state.render_mode != 3 || gap < pen.size * 0.6) + && (s.state.render_mode != 3 || (gap < pen.size * 0.6 && !invisible_apart)) }); if joins && let Some(l) = lines.last_mut() { let gap = last.map_or(0.0, |(_, pen)| pen.continues(&s, 0.0).1); @@ -558,8 +565,9 @@ impl FormBudget { } } -/// A form XObject a page draws: its object, content, resources and its matrix composed with the -/// CTM at the `Do`. +/// A form XObject a page draws: its object, content, resources, its matrix composed with the +/// CTM at the `Do`, and the clip its `/BBox` makes (page space, under that same matrix, so a +/// turned form keeps its turned box; a box of no area hides everything) if it has a valid one. pub(crate) struct FormCall { pub(crate) obj: Option, pub(crate) data: Vec, @@ -568,6 +576,7 @@ pub(crate) struct FormCall { form: Arc, own: Option>, pub(crate) ctm: Matrix, + bbox: Option, } impl FormCall { @@ -637,6 +646,12 @@ pub(crate) fn form_call( <[f64; 6]>::try_from(v).ok().filter(|v| v.iter().all(|x| x.is_finite())) }); let own = s.dict.get(b"Resources").filter(|r| matches!(r, Object::Ref(_))).map(|r| doc.resolve(r)); + // Four finite numbers, or no bound at all: a member that isn't one makes the box unknown, and + // reading the numbers around it would invent a clip. + let bbox = s.dict.get(b"BBox").map(|b| doc.resolve(b)).and_then(|b| { + let values: Vec = b.as_array()?.iter().map(|x| doc.resolve(x).as_f64().filter(|v| v.is_finite())).collect::>()?; + <[f64; 4]>::try_from(values).ok() + }); let data = match budget.decode.decode(s) { Ok(data) => data, Err(_) if budget.decode.remaining() == 0 => { @@ -646,7 +661,8 @@ pub(crate) fn form_call( Err(_) => Vec::new(), }; budget.visits = budget.visits.saturating_add(1); - Some(FormCall { obj, data, form, own, ctm: m.map_or(ctm, |m| Matrix(m).then(&ctm)) }) + let ctm = m.map_or(ctm, |m| Matrix(m).then(&ctm)); + Some(FormCall { obj, data, form, own, ctm, bbox: bbox.and_then(|b| region(&ctm, b)).map(|r| if r.is_empty() { Vec::new() } else { vec![r] }) }) } /// The lines a page shows as it reads, including text drawn by form XObjects (which @@ -677,6 +693,12 @@ pub(super) fn reading_lines(doc: &Document, page: usize) -> Result<(Vec Result<(Vec Some(clip_within(outer, own)), + (outer, own) => outer.clone().or_else(|| own.clone()), + }; + let mut carry = Carry { + ts: Ts { ctm: form.ctm, clip, ..ts }, + stack: Vec::new(), + marked: Vec::new(), + layers: Vec::new(), + text_left: *text_left, + gap_space: false, + }; let stream = w.next_stream; w.next_stream = w.next_stream.saturating_add(1); if let Some(r) = form.obj { diff --git a/crates/edit/src/text/rewrite.rs b/crates/edit/src/text/rewrite.rs index 8741888c4e..c146ffcbc0 100644 --- a/crates/edit/src/text/rewrite.rs +++ b/crates/edit/src/text/rewrite.rs @@ -1,13 +1,13 @@ -use super::blocks::{BlockStyle, TextBlock, paragraph_separator, wrap}; +use super::blocks::{BlockStyle, TextBlock, paragraph_separator, row_break, wrap}; use super::fallback::{ - Encoder, MAX_TYPE3_GLYPHS, Type3Fallback, drop_unused_fallback_fonts, needs_type3, no_japanese_font, source_encodes, source_family, - source_program_completes, source_show, source_space, source_width, type3_encode, type3_font, + Encoder, MAX_TYPE3_GLYPHS, Type3Fallback, Type3Run, drop_unused_fallback_fonts, is_variation_selector, needs_type3, no_japanese_font, + source_encodes, source_family, source_program_completes, source_show, source_space, source_width, type3_encode, type3_font, }; use super::interp::{Carry, Origin, TextState, interpret}; use super::lines::{content_streams, joined, page_dict, rewritten_contents, splice, text_lines}; use super::{ Dict, Document, EditError, HashMap, Matrix, Metrics, Object, Op, PdfString, TextLine, TextSession, alignment_frame, group_blocks, is_centred, - parse, rewrite_styled_block, + parse, rewrite_changed_words, rewrite_styled_block, }; /// What replacing a line did. @@ -42,7 +42,7 @@ fn refuse_clipping_text(lines: &[TextLine], dropped: &std::collections::HashSet< /// [`replace_line`] with the most unique characters one generated font may hold (tests lower it). pub(crate) fn replace_line_with_limit(doc: &mut Document, page: usize, line: usize, text: &str, max_glyphs: usize) -> Result { - let text = text.replace(['\n', '\r'], " "); + let text = text.replace(row_break, " "); let lines = text_lines(doc, page)?; let target = lines.get(line).cloned().ok_or_else(|| EditError::Invalid(format!("page {} has no line {}", page + 1, line + 1)))?; target.ensure_editable()?; @@ -94,9 +94,18 @@ pub(crate) fn replace_line_with_limit(doc: &mut Document, page: usize, line: usi let runs = type3_encode(&fallback, &fallback_shown) .ok_or_else(|| EditError::Invalid(format!("\"{text}\" can't be shown by the {} fallback", fallback.family)))?; let size = font_size_before(&ops, first).unwrap_or(target.size); - for (name, bytes) in runs { - replacement.push(Op::new("Tf", vec![Object::name(&name), pdfcraft_content::num(size)])); - replacement.push(Op::new("Tj", vec![Object::String(PdfString::literal(bytes))])); + let spacing = target.origin.state.char_spacing; + for run in runs { + replacement.push(Op::new("Tf", vec![Object::name(&run.name), pdfcraft_content::num(size)])); + if let Some(tc) = run.spacing { + // A variation selector's empty glyph: shown at spacing zero, then the + // line's own spacing back, so it adds no advance. + replacement.push(Op::new("Tc", vec![pdfcraft_content::num(tc)])); + replacement.push(Op::new("Tj", vec![Object::String(PdfString::literal(run.bytes))])); + replacement.push(Op::new("Tc", vec![pdfcraft_content::num(spacing)])); + } else { + replacement.push(Op::new("Tj", vec![Object::String(PdfString::literal(run.bytes))])); + } } replacement.push(Op::new("Tf", vec![Object::name(&target.font), pdfcraft_content::num(size)])); substituted = Some(format!("{} Type3", fallback.family)); @@ -240,7 +249,7 @@ fn choose_rewrite_path( } let visual_indic = members.iter().any(|line| line.visual_order); let has_special_text = text.is_some_and(|draft| { - draft.contains(['\n', '\r']) + draft.contains(row_break) || pdfcraft_fonts::source_shaping::required(draft) || !pdfcraft_fonts::embedding::matching_faces(&block.base_font).is_empty() }); @@ -259,14 +268,14 @@ fn choose_rewrite_path( } /// The paragraph's text for writing back: explicit line breaks stay (#1443), and so does each typed -/// U+0020 space: a run of them is not folded into one (#4199, #3797). `\r\n` and a lone `\r` are -/// breaks too; other white space is written as one space. +/// U+0020 space: a run of them is not folded into one (#4199, #3797). Every [`row_break`] is a break +/// too; other white space is written as one space. fn paragraph_text(text: &str) -> String { text.replace("\r\n", "\n") .chars() .map(|c| match c { - '\r' => '\n', - c if c != '\n' && paragraph_separator(c) => ' ', + c if row_break(c) => '\n', + c if paragraph_separator(c) => ' ', c => c, }) .collect() @@ -351,7 +360,8 @@ struct Advance<'a> { impl Advance<'_> { fn of(&self, s: &str) -> f64 { - let extra = s.chars().count() as f64 * self.char_spacing; + // A variation selector draws nothing and adds no spacing: it takes no room. + let extra = s.chars().filter(|ch| !is_variation_selector(*ch)).count() as f64 * self.char_spacing; let t = match (self.source, self.type3) { (Some(m), _) => { let visual_s = if m.visual_order { crate::indic::to_visual(s) } else { s.to_owned() }; @@ -399,7 +409,7 @@ fn choose_font(inputs: &FontInputs<'_>, fonts_res: &mut Dict) -> Result, fonts_res: &mut Dict) -> Result { } // Each wrapped line in typing order; drawn left to right (right-to-left runs reversed). let visual = crate::bidi::to_visual(shown); - // Stretched without word spacing: shown up to and including each space, then moved on. - let pieces: Vec<&str> = if stretch > 0.0 { visual.split_inclusive(' ').collect() } else { vec![visual.as_str()] }; - for piece in pieces { - self.show_piece(piece, line, tw, cursor, ops)?; - if stretch > 0.0 && piece.ends_with(' ') { - ops.push(Op::new("TJ", vec![Object::Array(vec![n(-1000.0 * stretch / self.size)])])); + if let Some(metrics) = self.source.filter(|m| m.encode(" ").is_none()) { + let indic_line = if metrics.visual_order { crate::indic::to_visual(&visual) } else { visual }; + ops.extend(source_show(metrics, &indic_line, self.size, self.space, self.char_spacing, tw + stretch)?); + } else { + // Stretched without word spacing: shown up to and including each space, then moved on. + let pieces: Vec<&str> = if stretch > 0.0 { visual.split_inclusive(' ').collect() } else { vec![visual.as_str()] }; + for piece in pieces { + self.show_piece(piece, line, tw, cursor, ops)?; + if stretch > 0.0 && piece.ends_with(' ') { + ops.push(Op::new("TJ", vec![Object::Array(vec![n(-1000.0 * stretch / self.size)])])); + } } } let tw = tw + stretch; @@ -656,12 +675,20 @@ impl LineWriter<'_> { ops.extend(source_show(metrics, &indic_piece, self.size, self.space, self.char_spacing, tw)?); return Ok(()); } - for (font, bytes) in (self.encode)(piece).ok_or_else(|| EditError::Invalid(format!("\"{line}\" can't be shown")))? { - if font != cursor.shown_font { - ops.push(Op::new("Tf", vec![Object::name(&font), n(self.size)])); - cursor.shown_font = font; + for run in (self.encode)(piece).ok_or_else(|| EditError::Invalid(format!("\"{line}\" can't be shown")))? { + if run.name != cursor.shown_font { + ops.push(Op::new("Tf", vec![Object::name(&run.name), n(self.size)])); + cursor.shown_font = run.name; + } + if let Some(tc) = run.spacing { + // A variation selector's empty glyph: shown at spacing zero, then the paragraph's + // own spacing back, so it adds no advance. + ops.push(Op::new("Tc", vec![n(tc)])); + ops.push(Op::new("Tj", vec![Object::String(PdfString::literal(run.bytes))])); + ops.push(Op::new("Tc", vec![n(self.char_spacing)])); + } else { + ops.push(Op::new("Tj", vec![Object::String(PdfString::literal(run.bytes))])); } - ops.push(Op::new("Tj", vec![Object::String(PdfString::literal(bytes))])); } Ok(()) } @@ -829,6 +856,9 @@ pub(crate) fn rewrite_block_from( if b.invisible { crate::check_on_top(doc, &p)?; } + if let Some(edit) = text.and_then(|draft| rewrite_changed_words(doc, page, &b, &lines, draft, style, max_glyphs)) { + return Ok(edit); + } let mut res = p.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default(); let mut fonts_res = res.get(b"Font").map(|f| doc.resolve(f)).and_then(|f| f.as_dict().cloned()).unwrap_or_default(); let dispatch = choose_rewrite_path(doc, &fonts_res, &b, &members, first, text, style)?; @@ -996,6 +1026,14 @@ pub(crate) fn rewrite_block_from( /// rewritten invisible paragraph, drawn in the space whose CTM is `ctm`. Nothing for visible text. /// The paragraph's `q`/`Q` restores the rendering mode for the invisible text after it; one /// painted over where it was is drawn on top of the page instead ([`crate::draw_on_top`]). +/// What covering the word in `rect` paints: scanned ink reaches a little past the recognised box, +/// but not past `between`, the words on either side that stay. It always covers the word itself. +pub(crate) fn cover_box(rect: [f64; 4], between: Option<[f64; 2]>) -> [f64; 4] { + let pad = ((rect[3] - rect[1]).abs() * 0.1).clamp(0.5, 2.0); + let [left, right] = between.unwrap_or([f64::NEG_INFINITY, f64::INFINITY]); + [(rect[0] - pad).max(left.min(rect[0])), rect[1] - pad, (rect[2] + pad).min(right.max(rect[2])), rect[3] + pad] +} + pub(crate) fn reveal_ops(block: &TextBlock, lines: &[TextLine], style: &BlockStyle, ctm: [f64; 6]) -> Result, EditError> { if !block.invisible { return Ok(Vec::new()); @@ -1009,11 +1047,8 @@ pub(crate) fn reveal_ops(block: &TextBlock, lines: &[TextLine], style: &BlockSty let back = Matrix(ctm).invert().ok_or_else(|| EditError::Invalid("the paragraph is drawn in a space it can't be covered in".into()))?; ops.push(Op::new("q", vec![])); ops.push(Op::new("rg", colour.map(n).to_vec())); - for rect in block.lines.iter().filter_map(|i| lines.get(*i)).map(|l| l.rect) { - // Scanned ink reaches a little past the recognised box. - let pad = ((rect[3] - rect[1]).abs() * 0.1).clamp(0.5, 2.0); - let corners = - [[rect[0] - pad, rect[1] - pad], [rect[2] + pad, rect[1] - pad], [rect[2] + pad, rect[3] + pad], [rect[0] - pad, rect[3] + pad]]; + for rect in block.lines.iter().filter_map(|i| lines.get(*i)).map(|l| cover_box(l.rect, style.cover_between)) { + let corners = [[rect[0], rect[1]], [rect[2], rect[1]], [rect[2], rect[3]], [rect[0], rect[3]]]; for (i, [x, y]) in corners.into_iter().enumerate() { let (x, y) = back.apply(x, y); if !(x.is_finite() && y.is_finite()) { @@ -1117,6 +1152,36 @@ fn vertical_encoding(doc: &Document, font: &Dict) -> Result { Err(EditError::Invalid("the text encoding is nested too deeply".into())) } +/// The text-showing operators of paragraphs `indexes` (from [`group_blocks`] of the page's +/// lines), as indexes among the page's joined operators. `editable`: refuse a line that can't be +/// edited (moving rewrites its operators; copying takes them as they are). +pub(crate) fn paragraph_ops(doc: &Document, page: usize, indexes: &[usize], editable: bool) -> Result, EditError> { + let mut selected = std::collections::BTreeSet::new(); + if indexes.is_empty() { + return Ok(selected); + } + let lines = text_lines(doc, page)?; + let blocks = group_blocks(&lines); + for index in indexes { + let b = blocks.get(*index).ok_or_else(|| EditError::Invalid("a selected paragraph no longer exists".into()))?; + for line in &b.lines { + let l = lines.get(*line).ok_or_else(|| EditError::Invalid("a selected text line no longer exists".into()))?; + if editable { + l.ensure_editable()?; + } + selected.extend(l.ops.iter().copied()); + } + } + Ok(selected) +} + +/// Where each text-showing operator of `ops` (a page's joined operators, drawn with `resources`) +/// starts its glyphs: the text matrix there, and its rendering mode (`Tr`). +pub(crate) fn show_starts(doc: &Document, ops: &[Op], resources: &Dict) -> Result, EditError> { + let shown = interpret(doc, ops, resources, &mut HashMap::new(), &mut Carry::new(), None, None)?; + Ok(shown.into_iter().map(|s| (s.op, (s.tm, s.state.render_mode))).collect()) +} + /// Translate paragraphs together without re-encoding glyphs, changing fonts or reflowing. /// Numeric TJ offsets and text rise temporarily move each selected show operation, then /// restore both advances and rise. The PDF reader advances the original glyph bytes itself: @@ -1127,17 +1192,7 @@ pub(crate) fn translate_blocks(doc: &mut Document, page: usize, indexes: &[usize } let p = page_dict(doc, page)?; let streams = content_streams(doc, &p.dict)?; - let lines = text_lines(doc, page)?; - let blocks = group_blocks(&lines); - let mut selected = std::collections::BTreeSet::new(); - for index in indexes { - let b = blocks.get(*index).ok_or_else(|| EditError::Invalid("a selected paragraph no longer exists".into()))?; - for line in &b.lines { - let l = lines.get(*line).ok_or_else(|| EditError::Invalid("a selected text line no longer exists".into()))?; - l.ensure_editable()?; - selected.extend(l.ops.iter().copied()); - } - } + let selected = paragraph_ops(doc, page, indexes, true)?; let res = p.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default(); let fonts = res.get(b"Font").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default(); let pieces = joined(&streams); diff --git a/crates/edit/src/text_regions.rs b/crates/edit/src/text_regions.rs index 98de481444..ee808a8dec 100644 --- a/crates/edit/src/text_regions.rs +++ b/crates/edit/src/text_regions.rs @@ -3,15 +3,18 @@ use super::{ BlockStyle, CraftFont, Dict, Document, EditError, HashMap, LineEdit, MAX_TYPE3_GLYPHS, Matrix, Metrics, Object, Op, PdfString, Pieces, RangeStyle, TextBlock, TextLine, Type3Glyph, builtin_thai_choice, coincident_ops, content_streams, fallback_face, japanese_glyph_from, - needs_type3, no_fallback_font, no_glyph_for, page_dict, paragraph_separator, reveal_ops, rewritten_contents, same_base_font, same_font, - source_encodes, source_family, source_program, source_show, source_space, source_width, splice, strip_subset_tag, type3_faces, type3_glyph, - union, write_type3_glyphs, + kept_words, needs_type3, no_fallback_font, no_glyph_for, page_dict, paragraph_separator, reveal_ops, rewritten_contents, row_break, + same_base_font, same_font, source_encodes, source_family, source_program, source_show, source_space, source_width, splice, strip_subset_tag, + type3_faces, type3_glyph, union, word_lines, write_type3_glyphs, }; #[path = "text_regions/grouping.rs"] mod grouping; +#[path = "text_regions/ocr_words.rs"] +mod ocr_words; #[path = "text_regions/styled.rs"] mod styled; pub(super) use grouping::{alignment_frame, group_blocks, group_reading_blocks, is_centred}; +pub(super) use ocr_words::rewrite_changed_words; pub(super) use styled::rewrite_styled_block; diff --git a/crates/edit/src/text_regions/grouping.rs b/crates/edit/src/text_regions/grouping.rs index 69b3a16850..72a516292d 100644 --- a/crates/edit/src/text_regions/grouping.rs +++ b/crates/edit/src/text_regions/grouping.rs @@ -331,7 +331,19 @@ fn group_layout(lines: &[TextLine], mixed: bool) -> Vec { let scale = gap.abs().max(ngap.abs()); if same && ngap > tol * 0.8 && scale < tol * 4.0 && (ngap - gap).abs() <= scale * 0.08 + 0.5 { Some(gap) } else { None } }); - let max_gap = own_leading.filter(|lead| *lead > tol * 0.4).or(stated).or(repeated).map_or(tol * 2.05, |lead| lead * 1.5); + // A lone row with none of those: the step to the row after it is the leading of the + // paragraph it opens, and the gap above it is a heading's or a blank line's when it + // is much wider (#3577). + let following = rows.get(row_at.saturating_add(1)).and_then(|next| { + let nfirst = next.first().and_then(|i| lines.get(*i))?; + let ngap = first.frame_origin().1 - nfirst.frame_origin().1; + (same_plane(first, nfirst) && ngap > tol * 0.8 && ngap < tol * 2.05).then_some(ngap) + }); + let max_gap = own_leading + .filter(|lead| *lead > tol * 0.4) + .or(stated) + .or(repeated) + .map_or_else(|| following.map_or(tol * 2.05, |lead| (lead * 1.35).min(tol * 2.05)), |lead| lead * 1.5); // OCR lines stay apart: the scan fixes where each one is, so they never rewrap together. (!mixed || !(invisible(prev) || invisible(first))) && (mixed || same_style(prev, first)) @@ -366,10 +378,13 @@ fn group_layout(lines: &[TextLine], mixed: bool) -> Vec { if !block.rows.is_empty() { block.editor_text.push('\n'); } + // The run placed before this one in the row, tracked rather than searched for: rows + // don't share runs, so it is the last one this loop pushed. + let mut prev_in_row: Option<&TextLine> = None; for &index in row { let Some(line) = lines.get(index) else { continue }; // Some producers position separate words without encoding a space. - if let Some(prev) = block.lines.last().filter(|i| row.contains(i)).and_then(|i| lines.get(*i)) + if let Some(prev) = prev_in_row && !row_text.ends_with(char::is_whitespace) && !line.text.starts_with(char::is_whitespace) && line.frame_rect()[0] - prev.frame_rect()[2] > line.size * 0.12 @@ -383,6 +398,7 @@ fn group_layout(lines: &[TextLine], mixed: bool) -> Vec { row_text.push_str(&line.text); block.invisible &= line.origin.state.render_mode == 3; block.lines.push(index); + prev_in_row = Some(line); } if !block.text.is_empty() { if block.text.ends_with('-') { diff --git a/crates/edit/src/text_regions/ocr_words.rs b/crates/edit/src/text_regions/ocr_words.rs new file mode 100644 index 0000000000..d511ccdf47 --- /dev/null +++ b/crates/edit/src/text_regions/ocr_words.rs @@ -0,0 +1,436 @@ +//! Edit text on a scanned page, word by word: a text edit of an OCR phrase covers and redraws only +//! the words it changes, and the rest stay as the scan shows them (#3313). Each changed stretch is +//! rewritten as a phrase of its own, which is how a phrase beside others in one OCR layer already +//! rewrites. A deleted word leaves its gap: the words after it are pixels, and don't move. + +use std::collections::{BTreeMap, BTreeSet, HashMap}; +use std::ops::Range; + +use super::grouping::row_rect; +use super::styled::rewrite_styled_block; +use super::{BlockStyle, Document, EditError, LineEdit, TextBlock, TextLine, kept_words, word_lines}; + +/// How many times a replacement may be refitted to the room it has before the whole phrase is +/// redrawn instead. Each try rewrites a copy of the document. +const MAX_TRIES: usize = 16; + +/// The narrowest a replacement is squeezed to stay in its word's place. Most replacements are +/// only a little wider than the word they replace, and taking over the next word pushes that one +/// along too, and so on down the line. Past this it would be hard to read, so it takes the word over. +const MIN_SQUEEZE: f64 = 0.75; + +/// A stretch of the phrase's word runs that changes, the text that replaces it (empty when it is +/// deleted), and how much that text is squeezed across to fit. +#[derive(Debug, PartialEq)] +struct Change { + runs: Range, + /// Which of the draft's words replace them. + words: Range, + text: String, + squeeze: Option, +} + +/// The stretches `new` changes, given how many words each run holds and which word of `new` each +/// old word stays as ([`kept_words`]). A run in `redrawn` counts as changed, and new words with no +/// run of their own to replace add the run after them to it (or the last one, at the end). +fn changes(runs: &[usize], kept_as: &[Option], new: &[&str], redrawn: &mut BTreeSet) -> Vec { + // Each pass either returns or adds a run to `redrawn`, so this ends. + 'plan: loop { + let mut out = Vec::new(); + let (mut first_run, mut first_new, mut token) = (0usize, 0usize, 0usize); + for (run, count) in runs.iter().enumerate() { + let words = kept_as.get(token..token.saturating_add(*count)).unwrap_or_default(); + token = token.saturating_add(*count); + let at = words.first().copied().flatten(); + let kept = !redrawn.contains(&run) && !words.is_empty() && words.iter().enumerate().all(|(i, w)| *w == at.map(|a| a.saturating_add(i))); + let (true, Some(at)) = (kept, at) else { continue }; + if first_run == run && first_new < at { + redrawn.insert(run); + continue 'plan; + } + if first_run < run { + out.push(Change { + runs: first_run..run, + words: first_new..at, + text: new.get(first_new..at).unwrap_or_default().join(" "), + squeeze: None, + }); + } + first_run = run.saturating_add(1); + first_new = at.saturating_add(*count); + } + if first_run < runs.len() { + out.push(Change { + runs: first_run..runs.len(), + words: first_new..new.len(), + text: new.get(first_new..).unwrap_or_default().join(" "), + squeeze: None, + }); + } else if first_new < new.len() + && let Some(last) = runs.len().checked_sub(1) + { + redrawn.insert(last); + continue 'plan; + } + return out; + } +} + +/// Each word of `text` as a byte range. +fn word_ranges(text: &str) -> Vec> { + let mut out = Vec::new(); + let mut start = None; + for (at, c) in text.char_indices() { + match (c.is_whitespace(), start) { + (true, Some(from)) => { + out.push(from..at); + start = None; + } + (false, None) => start = Some(at), + _ => {} + } + } + if let Some(from) = start { + out.push(from..text.len()); + } + out +} + +/// Whether `plan` can stand for the spacing in `draft`, whose words are `words`. Spaces beside a +/// change are nothing to draw: Backspacing over a word leaves one either side of where it was, and +/// the words around it stay where the scan has them. Anything else that isn't one space has to be +/// drawn, inside the new words or between two words nothing else changed, which only the whole +/// rewrite can do. +fn spacing_fits(draft: &str, words: &[Range], plan: &[Change]) -> bool { + words.windows(2).enumerate().all(|(index, pair)| { + let [before, after] = pair else { return true }; + let gap = draft.get(before.end..after.start).unwrap_or(" "); + let boundary = index.saturating_add(1); + gap == " " || (gap.chars().all(|c| c == ' ') && plan.iter().any(|change| change.words.start == boundary || change.words.end == boundary)) + }) +} + +/// The phrase's stretch `indices` of `lines`, as a paragraph of its own. It keeps the whole +/// phrase's size, so what is redrawn is set alike wherever in the phrase it is. +fn stretch(phrase: &TextBlock, lines: &[TextLine], indices: &[usize]) -> Result { + let first = indices.first().and_then(|i| lines.get(*i)).ok_or_else(|| EditError::Invalid("the phrase has no such words".into()))?; + let rect = row_rect(indices, lines).ok_or_else(|| EditError::Invalid("the phrase has no such words".into()))?; + let mut text = String::new(); + let mut spans = Vec::with_capacity(indices.len()); + for index in indices { + let Some(line) = lines.get(*index) else { continue }; + if !text.is_empty() && !text.ends_with(char::is_whitespace) && !line.text.starts_with(char::is_whitespace) { + text.push(' '); + } + let start = text.len(); + text.push_str(&line.text); + spans.push((start..text.len(), *index)); + } + Ok(TextBlock { + reflow_text: text.clone(), + editor_text: text.clone(), + text, + spans, + rect, + base_font: first.base_font.clone(), + size: phrase.size, + bold: first.bold, + italic: first.italic, + color: first.color, + rotation: first.rotation(), + lines: indices.to_vec(), + rows: vec![indices.to_vec()], + invisible: true, + }) +} + +fn key(line: &TextLine) -> (&str, [i64; 4]) { + // Saturating casts: a non-finite edge only makes the comparison below fail, safely. + (line.text.as_str(), line.rect.map(|v| (v * 100.0).round() as i64)) +} + +/// How far right a rewrite of the runs `indices` drew (`None` when it drew nothing), or an error +/// when it moved or lost any other text on the page: text positioned after the end of the text +/// before it, say, depends on the words that went. +fn drawn_right(before: &[TextLine], indices: &[usize], after: &[TextLine]) -> Result, EditError> { + let mut unmatched: HashMap<_, usize> = HashMap::new(); + for line in before { + let count = unmatched.entry(key(line)).or_default(); + *count = count.saturating_add(1); + } + let mut drawn = Vec::new(); + for line in after { + match unmatched.get_mut(&key(line)).filter(|count| **count > 0) { + Some(count) => *count -= 1, + None => drawn.push(line), + } + } + // Visible text beside what was drawn reads as one line with it. It is still where it was. + let joined = |line: &TextLine| { + let [x0, y0, x1, y1] = line.rect; + drawn.iter().any(|d| { + d.text.contains(line.text.trim()) && d.rect[0] <= x0 + 0.01 && d.rect[1] <= y0 + 0.01 && d.rect[2] >= x1 - 0.01 && d.rect[3] >= y1 - 0.01 + }) + }; + let moved = before + .iter() + .enumerate() + .any(|(index, line)| !indices.contains(&index) && unmatched.get(&key(line)).is_some_and(|c| *c > 0) && !joined(line)); + if moved { + return Err(EditError::Invalid("rewriting these words alone would move the text around them".into())); + } + Ok(drawn.iter().map(|line| line.rect[2]).reduce(f64::max)) +} + +/// Rewrite each change of `plan` in `doc`, last first: a rewrite goes where its words were or at +/// the end of the page, so the words before it keep their indexes (an error if they don't). Gives +/// how far right each drew. +fn apply( + doc: &mut Document, + page: usize, + phrase: &TextBlock, + members: &[usize], + plan: &[Change], + style: &BlockStyle, + max_glyphs: usize, +) -> Result<(Vec>, Option), EditError> { + // The stretch starts where its words did. Its own box would centre a word that happens to + // sit in the middle of the page. + let mut style = BlockStyle { align: Some(crate::added::Align::Left), changed_words_only: false, ..style.clone() }; + let mut rights = vec![None; plan.len()]; + let mut substituted = None; + let original = word_lines(doc, page)?; + for (at, change) in plan.iter().enumerate().rev() { + let indices = members.get(change.runs.clone()).ok_or_else(|| EditError::Invalid("the phrase has no such words".into()))?; + let before = word_lines(doc, page)?; + // The words up to the next that stays must still be where they were. A rewrite also takes + // away copies of its words, and a copy earlier on the page moves these up the list. + let up_to = change.runs.end.saturating_add(1).min(members.len()); + let same = |index: &usize| before.get(*index).map(key) == original.get(*index).map(key); + if !members.get(..up_to).unwrap_or_default().iter().all(same) { + return Err(EditError::Invalid("rewriting these words alone would lose track of the words before them".into())); + } + let part = stretch(phrase, &before, indices)?; + // Relative to the scale the words have, which a layer may have narrowed already. + let own_scale = indices.first().and_then(|i| before.get(*i)).map_or(1.0, |line| line.origin.state.scale); + style.scale = change.squeeze.map(|squeeze| squeeze * own_scale * 100.0); + // The words on either side stay, with all their ink. + let edge = |run: Option| run.and_then(|run| members.get(run)).and_then(|i| before.get(*i)).map(|line| line.rect); + style.cover_between = Some([ + edge(change.runs.start.checked_sub(1)).map_or(f64::NEG_INFINITY, |rect| rect[2]), + edge(Some(change.runs.end)).map_or(f64::INFINITY, |rect| rect[0]), + ]); + let edit = rewrite_styled_block(doc, page, &part, &before, Some(&change.text), &style, max_glyphs)?; + substituted = substituted.or(edit.substituted); + let right = drawn_right(&before, indices, &word_lines(doc, page)?)?; + if right.is_none() && !change.text.is_empty() { + return Err(EditError::Invalid("the new words were not drawn".into())); + } + if let Some(slot) = rights.get_mut(at) { + *slot = right; + } + } + Ok((rights, substituted)) +} + +/// Whether a change that drew as far as `right` needs another try. It should end by the middle of +/// the gap before the word after it, so the two don't touch: within [`MIN_SQUEEZE`] it is squeezed +/// to, and otherwise it takes that word over. +fn make_room(change: &Change, right: f64, boxes: &[[f64; 4]], squeezes: &mut BTreeMap, redrawn: &mut BTreeSet) -> bool { + let (Some(first), Some(last), Some(next)) = + (boxes.get(change.runs.start), change.runs.end.checked_sub(1).and_then(|run| boxes.get(run)), boxes.get(change.runs.end)) + else { + return false; + }; + let limit = (last[2] + next[0]) / 2.0; + // Widths scale in proportion, so one squeeze lands within this; the rest is rounding. + if right <= limit + 0.25 { + return false; + } + let squeeze = change.squeeze.unwrap_or(1.0) * (limit - first[0]) / (right - first[0]); + if squeeze >= MIN_SQUEEZE { + squeezes.insert(change.runs.start, squeeze); + } else { + squeezes.remove(&change.runs.start); + redrawn.insert(change.runs.end); + } + true +} + +/// Rewrite only the words `draft` changes in the invisible phrase `phrase`, or `None` when this +/// edit is not one that can (it changes the formatting or the lines, the phrase is turned or +/// right-to-left, or its words can't be told apart or kept in place): the caller then rewrites the +/// whole phrase. Nothing changes in `doc` unless the whole edit works. +pub(crate) fn rewrite_changed_words( + doc: &mut Document, + page: usize, + phrase: &TextBlock, + joined: &[TextLine], + draft: &str, + style: &BlockStyle, + max_glyphs: usize, +) -> Option { + let right_to_left = |text: &str| text.chars().any(crate::bidi::is_rtl); + if !style.changed_words_only + || !phrase.invisible + || phrase.rows.len() != 1 + || draft.contains(['\n', '\r']) + || right_to_left(draft) + || right_to_left(&phrase.text) + { + return None; + } + let lines = word_lines(doc, page).ok()?; + let phrase_ops: BTreeSet = phrase.lines.iter().filter_map(|i| joined.get(*i)).flat_map(|line| line.ops.iter().copied()).collect(); + let members: Vec = + lines.iter().enumerate().filter(|(_, line)| line.ops.first().is_some_and(|op| phrase_ops.contains(op))).map(|(index, _)| index).collect(); + if members.iter().filter_map(|i| lines.get(*i)).any(|line| !line.upright() || line.visual_order) { + return None; + } + let runs: Vec> = members.iter().filter_map(|i| lines.get(*i)).map(|line| line.text.split_whitespace().collect()).collect(); + let old: Vec<&str> = runs.iter().flatten().copied().collect(); + let new: Vec<&str> = draft.split_whitespace().collect(); + // Words the layer doesn't show apart (a run per letter, say) can't be kept apart. + if !phrase.editor_text.split_whitespace().eq(old.iter().copied()) { + return None; + } + // A single word that changes has nothing to keep. With the same words, the plan is empty. + if old.len() < 2 && old != new { + return None; + } + let kept_as = kept_words(&old, &new)?; + let counts: Vec = runs.iter().map(Vec::len).collect(); + let boxes: Vec<[f64; 4]> = members.iter().filter_map(|i| lines.get(*i)).map(|line| line.rect).collect(); + let mut redrawn = BTreeSet::new(); + let mut squeezes = BTreeMap::new(); + for _ in 0..MAX_TRIES { + let mut plan = changes(&counts, &kept_as, &new, &mut redrawn); + if !spacing_fits(draft, &word_ranges(draft), &plan) { + return None; + } + for change in &mut plan { + change.squeeze = squeezes.get(&change.runs.start).copied(); + } + let mut trial = doc.clone(); + let (rights, substituted) = apply(&mut trial, page, phrase, &members, &plan, style, max_glyphs).ok()?; + let mut again = false; + for (change, right) in plan.iter().zip(&rights) { + if let Some(right) = right { + again |= make_room(change, *right, &boxes, &mut squeezes, &mut redrawn); + } + } + if !again { + *doc = trial; + return Some(LineEdit { substituted }); + } + } + None +} + +#[cfg(test)] +mod tests { + use super::*; + + fn plan(old: &[&[&str]], new: &str) -> Vec<(Range, String)> { + let flat: Vec<&str> = old.iter().flat_map(|run| run.iter().copied()).collect(); + let new: Vec<&str> = new.split_whitespace().collect(); + let counts: Vec = old.iter().map(|run| run.len()).collect(); + changes(&counts, &kept_words(&flat, &new).unwrap(), &new, &mut BTreeSet::new()).into_iter().map(|c| (c.runs, c.text)).collect() + } + + #[test] + fn only_the_runs_a_draft_changes_are_planned() { + let abc: &[&[&str]] = &[&["a"], &["b"], &["c"]]; + assert_eq!(plan(abc, "a b c"), []); + assert_eq!(plan(abc, "a c"), [(1..2, String::new())], "a deletion"); + assert_eq!(plan(abc, "a x c"), [(1..2, "x".to_owned())], "a replacement"); + assert_eq!(plan(abc, "x b y"), [(0..1, "x".to_owned()), (2..3, "y".to_owned())], "two apart"); + assert_eq!(plan(abc, "x y c"), [(0..2, "x y".to_owned())], "two together"); + assert_eq!(plan(abc, ""), [(0..3, String::new())], "everything"); + } + + #[test] + fn new_words_with_no_run_to_replace_take_a_neighbour() { + let abc: &[&[&str]] = &[&["a"], &["b"], &["c"]]; + assert_eq!(plan(abc, "x a b c"), [(0..1, "x a".to_owned())], "at the start"); + assert_eq!(plan(abc, "a x b c"), [(1..2, "x b".to_owned())], "in the middle"); + assert_eq!(plan(abc, "a b c x"), [(2..3, "c x".to_owned())], "at the end"); + } + + #[test] + fn a_run_of_several_words_is_kept_only_whole() { + let runs: &[&[&str]] = &[&["a", "b"], &["c"]]; + assert_eq!(plan(runs, "a b x"), [(1..2, "x".to_owned())]); + assert_eq!(plan(runs, "a x c"), [(0..1, "a x".to_owned())], "one of its words changed"); + assert_eq!(plan(runs, "a x b c"), [(0..1, "a x b".to_owned())], "a word went in between its words"); + } + + #[test] + fn a_drafts_words_are_found_by_their_bytes() { + assert!(word_ranges("").is_empty() && word_ranges(" ").is_empty()); + assert_eq!(word_ranges("ab é c "), [0..2, 4..6, 7..8]); + } + + /// The plan for `draft` against the words a, b and c, and whether it stands for its spacing. + fn fits(draft: &str) -> bool { + let (old, new): (Vec<&str>, Vec<&str>) = (vec!["a", "b", "c"], draft.split_whitespace().collect()); + let plan = changes(&[1, 1, 1], &kept_words(&old, &new).unwrap(), &new, &mut BTreeSet::new()); + spacing_fits(draft, &word_ranges(draft), &plan) + } + + #[test] + fn spaces_beside_a_change_are_nothing_to_draw() { + assert!(fits("a b c")); + assert!(fits("a b c "), "after the words"); + assert!(fits(" a b c"), "before them"); + assert!(fits("a c"), "either side of a deleted word"); + assert!(fits(" b c"), "where a first word went"); + assert!(fits("a x c"), "either side of a replacement"); + } + + #[test] + fn spacing_that_has_to_be_drawn_doesnt_fit() { + assert!(!fits("a b c"), "between words nothing else changed"); + assert!(!fits("x y c"), "inside the new words"); + assert!(!fits("a\u{3000}x c"), "another kind of space, even beside a change"); + assert!(!fits("a\u{a0}x c")); + } + + /// Word boxes 10 wide, with a gap of 4 between them. + const BOXES: [[f64; 4]; 3] = [[0.0, 0.0, 10.0, 10.0], [14.0, 0.0, 24.0, 10.0], [28.0, 0.0, 38.0, 10.0]]; + + fn room_for(right: f64) -> (bool, BTreeMap, BTreeSet) { + let (mut squeezes, mut redrawn) = (BTreeMap::new(), BTreeSet::new()); + let change = Change { runs: 0..1, words: 0..1, text: "x".to_owned(), squeeze: None }; + (make_room(&change, right, &BOXES, &mut squeezes, &mut redrawn), squeezes, redrawn) + } + + #[test] + fn a_word_that_fits_is_left_alone() { + assert_eq!(room_for(12.0), (false, BTreeMap::new(), BTreeSet::new()), "it ends in the gap's first half"); + let last = Change { runs: 2..3, words: 2..3, text: "x".to_owned(), squeeze: None }; + assert!(!make_room(&last, 900.0, &BOXES, &mut BTreeMap::new(), &mut BTreeSet::new()), "the last word has the page to grow into"); + } + + #[test] + fn a_word_a_little_too_wide_is_squeezed_to_the_middle_of_the_gap() { + let (again, squeezes, redrawn) = room_for(15.0); + assert!(again && redrawn.is_empty()); + assert_eq!(squeezes.get(&0).copied(), Some(12.0 / 15.0)); + } + + #[test] + fn a_word_far_too_wide_takes_over_the_next_one() { + let (again, squeezes, redrawn) = room_for(30.0); + assert!(again && squeezes.is_empty()); + assert_eq!(redrawn, BTreeSet::from([1])); + } + + #[test] + fn a_redrawn_run_joins_the_change_beside_it() { + let (counts, old, new) = ([1, 1, 1], ["a", "b", "c"], ["x", "b", "c"]); + let mut redrawn = BTreeSet::from([1]); + let planned = changes(&counts, &kept_words(&old, &new).unwrap(), &new, &mut redrawn); + assert_eq!(planned, [Change { runs: 0..2, words: 0..2, text: "x b".to_owned(), squeeze: None }]); + } +} diff --git a/crates/edit/src/text_regions/styled.rs b/crates/edit/src/text_regions/styled.rs index 3d38a594f2..487bb0e20e 100644 --- a/crates/edit/src/text_regions/styled.rs +++ b/crates/edit/src/text_regions/styled.rs @@ -3,7 +3,7 @@ use super::grouping::{across, is_centred, row_rect}; use super::{ BlockStyle, CraftFont, Dict, Document, EditError, HashMap, LineEdit, MAX_TYPE3_GLYPHS, Matrix, Metrics, Object, Op, PdfString, Pieces, RangeStyle, TextBlock, TextLine, Type3Glyph, builtin_thai_choice, coincident_ops, content_streams, fallback_face, japanese_glyph_from, - needs_type3, no_fallback_font, no_glyph_for, page_dict, reveal_ops, rewritten_contents, source_encodes, source_family, source_program, + needs_type3, no_fallback_font, no_glyph_for, page_dict, reveal_ops, rewritten_contents, row_break, source_encodes, source_family, source_program, source_show, source_space, source_width, splice, type3_faces, type3_glyph, write_type3_glyphs, }; @@ -181,8 +181,8 @@ impl CharacterPreparation<'_> { "the bundled Thai font ({thai_choice}) is not in this build, which has no craft-fonts Thai face; load a complete Thai font or choose one the page lists" )) })?; - if !pdfcraft_fonts::embedding::covers(face.bytes, span) { - let missing = span.chars().find(|c| !pdfcraft_fonts::embedding::covers(face.bytes, c.encode_utf8(&mut [0; 4]))); + if !covers_rows(face.bytes, span) { + let missing = span.chars().find(|&c| !row_break(c) && !pdfcraft_fonts::embedding::covers(face.bytes, c.encode_utf8(&mut [0; 4]))); return Err(EditError::Invalid(match missing { Some(c) => format!("{} has no glyph for {c} (U+{:04X}); choose a font that covers the text", face.family, u32::from(c)), None => format!("{} cannot show every character in this selection; choose a font that covers the text", face.family), @@ -251,12 +251,13 @@ impl CharacterPreparation<'_> { }; let start = range.start; for (offset, ch) in draft.get(range).unwrap_or_default().char_indices() { - if ch == '\n' { - chars.push((ch, index, source.font.clone())); - continue; - } - if ch == '\r' { - let crlf = start.checked_add(offset).and_then(|at| draft.get(at.saturating_add(1)..)).is_some_and(|rest| rest.starts_with('\n')); + // A row break moves to the next row instead of asking a font for a glyph. The CR + // of a CRLF stays a `\r` the rows skip, so the pair is one break; every other break + // is written as `\n`. Either way it keeps its place among the draft's characters, + // which rows are counted in. + if row_break(ch) { + let crlf = ch == '\r' + && start.checked_add(offset).and_then(|at| draft.get(at.saturating_add(1)..)).is_some_and(|rest| rest.starts_with('\n')); chars.push((if crlf { '\r' } else { '\n' }, index, source.font.clone())); continue; } @@ -353,6 +354,17 @@ struct StyledLines<'a> { } type StyledChar = (char, usize, String); + +/// End of a contiguous run sharing both its style and font resource. +fn styled_run_end(items: &[StyledChar], start: usize) -> usize { + let Some(first) = items.get(start) else { return start }; + let mut end = start + 1; + while items.get(end).is_some_and(|next| next.1 == first.1 && next.2 == first.2) { + end += 1; + } + end +} + type StyledRows = Vec>; type RowRanges = Vec>; enum ShapedFont { @@ -369,6 +381,7 @@ type ShapedRunCache = HashMap<(usize, usize), ShapedText>; struct ShapedRows { runs: ShapedRunCache, advances: HashMap<(usize, usize), f64>, + char_advances: HashMap<(usize, usize), Vec>, } fn prepare_shaped_rows( @@ -381,15 +394,15 @@ fn prepare_shaped_rows( ) -> Result { let mut runs = HashMap::new(); let mut advances = HashMap::new(); + let mut char_advances = HashMap::new(); for (row_index, row) in rows.iter().enumerate().filter(|(index, _)| !kept.get(*index).copied().unwrap_or(false)) { let mut at = 0; while let Some(item) = row.get(at) { - let mut end = at + 1; - while row.get(end).is_some_and(|next| next.1 == item.1 && next.2 == item.2) { - end += 1; - } + let end = styled_run_end(row, at); let segment = row.get(at..end).unwrap_or_default(); - advances.insert((row_index, at), segment.iter().map(&advance).sum::()); + let widths: Vec = segment.iter().map(&advance).collect(); + advances.insert((row_index, at), widths.iter().copied().sum()); + char_advances.insert((row_index, at), widths); let text: String = segment.iter().map(|character| character.0).collect(); if let Some(shaped) = shape_source(doc, fonts, thai, &item.2, &text)? { runs.insert((row_index, at), shaped); @@ -397,7 +410,7 @@ fn prepare_shaped_rows( at = end; } } - Ok(ShapedRows { runs, advances }) + Ok(ShapedRows { runs, advances, char_advances }) } type ShapedResources = HashMap<(usize, usize), Vec<(String, Vec)>>; @@ -685,18 +698,41 @@ fn row_step(style: &BlockStyle, size: f64, source_leading: f64) -> f64 { type WrappedStyledRows = (StyledRows, RowRanges, Vec, Vec, Vec); +/// For each of `chars`, whether a row may end just before it inside a run of non-space characters +/// (Chinese, Japanese and Thai), worked out once per run: Thai words need the whole run, not just +/// two neighbours. +fn styled_break_before(chars: &[StyledChar]) -> Vec { + let mut break_before = vec![false; chars.len()]; + let mut run_start = 0; + while run_start < chars.len() { + let mut run_end = run_start; + while chars.get(run_end).is_some_and(|c| !c.0.is_whitespace()) { + run_end += 1; + } + let run: String = chars.get(run_start..run_end).unwrap_or_default().iter().map(|c| c.0).collect(); + for (i, cut) in crate::linebreak::breaks_before_chars(&run).into_iter().enumerate() { + if let Some(slot) = break_before.get_mut(run_start.saturating_add(i)) { + *slot = cut; + } + } + run_start = run_end.max(run_start.saturating_add(1)); + } + break_before +} + fn wrap_styled_rows( chars: &[StyledChar], width: f64, measure: impl Fn(&[StyledChar]) -> Result, ) -> Result { + let break_before = styled_break_before(chars); let mut rows: Vec> = Vec::new(); let mut row_ranges = Vec::new(); // True when this row starts because the previous word did not fit, not because of a newline. let mut row_soft = Vec::new(); let mut next_soft = false; // How many separator characters a wrap trimmed off the row that just ended, for the row - // about to start (0 for a Chinese/Japanese word split, which trims none; meaningless unless + // about to start (0 for a Chinese, Japanese or Thai word split, which trims none; meaningless unless // the row it belongs to is soft). Mirrors `next_soft`. let mut row_gap = Vec::new(); let mut next_gap = 0usize; @@ -729,11 +765,8 @@ fn wrap_styled_rows( } let mut end = at + 1; if !item.0.is_whitespace() { - // A word ends at a space, or where Chinese or Japanese text may break between characters. - while let (Some(before), Some(current)) = (chars.get(end - 1), chars.get(end)) - && !current.0.is_whitespace() - && !crate::linebreak::between(before.0, current.0) - { + // A word ends at a space, or where Chinese, Japanese or Thai text may break inside it. + while chars.get(end).is_some_and(|c| !c.0.is_whitespace()) && !break_before.get(end).copied().unwrap_or(false) { end += 1; } } @@ -903,7 +936,7 @@ fn legacy_thai_font( if !has_thai_source_face(cache, &family) { return Ok(None); } - let rows = || span.split(['\n', '\r']).filter(|row| !row.is_empty()); + let rows = || span.split(row_break).filter(|row| !row.is_empty()); for row in rows() { if let Err(error) = shape_existing_thai(cache, &source.font, row) { // Not in the face's existing codes. A complete program of the face that shapes every @@ -1077,7 +1110,7 @@ fn matching_thai_outline(doc: &Document, fonts: &Dict, source: &TextLine, span: } if !span.chars().any(|c| matches!(u32::from(c), 0x0e00..=0x0e7f)) || !pdfcraft_fonts::embedding::matching_faces(&source.base_font).iter().any(|f| std::ptr::eq(*f, face)) - || !pdfcraft_fonts::embedding::covers(face.bytes, span) + || !covers_rows(face.bytes, span) { return false; } @@ -1092,16 +1125,22 @@ fn matching_thai_outline(doc: &Document, fonts: &Dict, source: &TextLine, span: && pdfcraft_fonts::pdf::embedded_sfnt(doc, &font).is_some_and(|bytes| { pdfcraft_fonts::embedding::validate(&bytes).is_ok() && pdfcraft_fonts::source_shaping::has_thai_layout(&bytes) - && pdfcraft_fonts::embedding::covers(&bytes, span) - && span.split(['\n', '\r']).all(|row| pdfcraft_fonts::source_shaping::shape(&bytes, row).is_ok()) + && covers_rows(&bytes, span) + && span.split(row_break).all(|row| pdfcraft_fonts::source_shaping::shape(&bytes, row).is_ok()) }) }) } +/// Whether a complete program draws every row of `span`. [`row_break`] characters move to the next +/// row and are never drawn, so they need no glyph (#6272); `embedding::covers` exempts only CR and LF. +fn covers_rows(bytes: &[u8], span: &str) -> bool { + span.split(row_break).all(|row| pdfcraft_fonts::embedding::covers(bytes, row)) +} + /// Use the same Thai script selection and mark/cluster positioning as Add Text, including /// rows beginning with Latin or digits (guessing from their first character would select Latin). fn shape_thai_row(text: &str) -> Result { - if text.len() > 262_144 || text.contains(['\n', '\r']) { + if text.len() > 262_144 || text.contains(row_break) { return Err(EditError::Invalid("shape one bounded text row at a time".into())); } let clusters = @@ -1270,6 +1309,70 @@ fn untouched_rows(block: &TextBlock, draft: &str, rows: &[std::ops::Range .collect() } +fn segment_underline_runs( + widths: &[f64], + segment_width: f64, + x: f64, + default_style: usize, + row_styles: Option<&[Option]>, + pos: usize, +) -> Vec<(usize, f64, f64)> { + let width_sum = widths.iter().copied().sum::(); + let width_scale = if width_sum > 1e-6 { segment_width / width_sum } else { 0.0 }; + let mut cursor = x; + let mut runs: Vec<(usize, f64, f64)> = Vec::new(); + for (offset, width) in widths.iter().enumerate() { + let style_index = row_styles.and_then(|styles| styles.get(pos.saturating_add(offset))).and_then(|style| *style).unwrap_or(default_style); + let next = cursor + *width * width_scale; + match runs.last_mut() { + Some((prev, _, to)) if *prev == style_index => *to = next, + _ => runs.push((style_index, cursor, next)), + } + cursor = next; + } + runs +} + +/// Keep spaces in a subset without a space glyph attached to an adjacent glyph-bearing segment on +/// the same row so `source_show` wraps them in one `/ActualText` span (#4229). +/// +/// The returned slot is the space's style index from before that reassignment, where the +/// grouping key changed. Underlines stay with that style; the row itself keeps the neighbour's +/// key so the space is not a group of its own. +fn assign_subset_space_fonts(row: &mut [(char, usize, String)], metrics: &std::collections::BTreeMap) -> Vec> { + let mut right_any = vec![None; row.len()]; + let mut next_any: Option<(usize, String)> = None; + for (i, item) in row.iter().enumerate().rev() { + if item.0 != ' ' { + next_any = Some((item.1, item.2.clone())); + } else if let Some(slot) = right_any.get_mut(i) { + *slot = next_any.clone(); + } + } + let mut prev_any: Option<(usize, String)> = None; + let mut underline_style = vec![None; row.len()]; + for (i, item) in row.iter_mut().enumerate() { + if item.0 != ' ' { + prev_any = Some((item.1, item.2.clone())); + continue; + } + if metrics.get(&item.2).is_none_or(|m| m.encode(" ").is_some()) { + continue; + } + if let Some((style, font)) = prev_any.as_ref().or_else(|| right_any.get(i).and_then(Option::as_ref)) { + let original = item.1; + item.1 = *style; + item.2 = font.clone(); + if let Some(slot) = underline_style.get_mut(i) + && item.1 != original + { + *slot = Some(original); + } + } + } + underline_style +} + /// Operator index of the paragraph's first text object, and of the next line's text object /// when one follows. fn paragraph_span(block: &TextBlock, lines: &[TextLine]) -> Option<(usize, Option)> { @@ -1411,7 +1514,7 @@ fn push_whole_paragraph_marker(block_ops: &mut Vec, marked: bool, op: &str) /// An empty marker between two rows of one rewrite. `/PCHard` is a newline that was already in /// the text. `/PCSoft` is a wrap and rejoins when the box widens, with `/N` the exact number of -/// separator characters it consumed (0 for a Chinese/Japanese word split), so a rejoin restores +/// separator characters it consumed (0 for a Chinese, Japanese or Thai word split), so a rejoin restores /// what was actually there instead of guessing a single space (#2529). Neither is a paragraph /// region, so the lines stay in one scope. fn push_row_break_marker(block_ops: &mut Vec, row_soft: &[bool], row_gap: &[usize], row_index: usize) { @@ -1557,10 +1660,7 @@ pub(crate) fn rewrite_styled_block( let mut width = 0.0; let mut at = 0; while let Some(item) = items.get(at) { - let mut end = at + 1; - while items.get(end).is_some_and(|v| v.1 == item.1 && v.2 == item.2) { - end += 1; - } + let end = styled_run_end(items, at); let segment = items.get(at..end).unwrap_or_default(); let string: String = segment.iter().map(|v| v.0).collect(); if let Some(shaped) = shape_source(doc, &fonts, &thai, &item.2, &string)? { @@ -1585,7 +1685,8 @@ pub(crate) fn rewrite_styled_block( } Ok(width) }; - let (rows, row_ranges, row_lines, row_soft, row_gap) = wrap_styled_rows(&chars, width, measure)?; + let (mut rows, row_ranges, row_lines, row_soft, row_gap) = wrap_styled_rows(&chars, width, measure)?; + let space_styles: Vec<_> = rows.iter_mut().map(|row| assign_subset_space_fonts(row, &metrics)).collect(); let streams = content_streams(doc, &p.dict)?; let pieces = crate::text::joined(&streams); let ops = pieces.parse().ops; @@ -1616,7 +1717,7 @@ pub(crate) fn rewrite_styled_block( validate_contextual_rows(&rows, &kept)?; let measured: Vec<_> = rows.iter().map(|row| measure(row)).collect::>()?; let shaped = prepare_shaped_rows(doc, &fonts, &thai, &rows, &kept, advance)?; - let ShapedRows { runs: shaped_runs, advances: segment_advances } = shaped; + let ShapedRows { runs: shaped_runs, advances: segment_advances, char_advances } = shaped; let glyph_resources = embed_shaped_resources(doc, &mut fonts, &shaped_runs)?; let n = pdfcraft_content::num; let mut block_ops = vec![Op::new("q", vec![])]; @@ -1693,10 +1794,7 @@ pub(crate) fn rewrite_styled_block( while pos < row.len() { let Some(item) = row.get(pos) else { break }; let source = lines.get(item.1).unwrap_or(first); - let mut end = pos + 1; - while row.get(end).is_some_and(|other| other.1 == item.1 && other.2 == item.2) { - end += 1; - } + let end = styled_run_end(row, pos); let segment = row.get(pos..end).unwrap_or_default(); let string: String = segment.iter().map(|c| c.0).collect(); let m = metrics.get(&item.2).ok_or_else(|| EditError::Invalid("the paragraph font is missing".into()))?; @@ -1752,11 +1850,20 @@ pub(crate) fn rewrite_styled_block( )?); block_ops.push(Op::new("ET", vec![])); } - if underlines.get(&item.1).copied().or(style.underline) == Some(true) { - let [r, g, b] = overrides.get(&item.1).and_then(|r| r.color).or(style.color).unwrap_or(source.color); - let underline_size = size_of(source, item.1) * source.origin.k; - let (x0, y0) = back.apply(x - o.ctm[4], baseline - o.ctm[5] - underline_size * 0.12); - let (x1, y1) = back.apply(x + segment_width - o.ctm[4], baseline - o.ctm[5] - underline_size * 0.12); + // A space reassigned onto a neighbour keeps that neighbour's grouping key, and its + // own style for the underline. Stroke each run of one underline style separately. + let widths = char_advances.get(&(row_index, pos)).map(Vec::as_slice).unwrap_or(&[]); + for (style_index, from, to) in + segment_underline_runs(widths, segment_width, x, item.1, space_styles.get(row_index).map(Vec::as_slice), pos) + { + let source = lines.get(style_index).unwrap_or(first); + if underlines.get(&style_index).copied().or(style.underline) != Some(true) || to <= from { + continue; + } + let [r, g, b] = overrides.get(&style_index).and_then(|r| r.color).or(style.color).unwrap_or(source.color); + let underline_size = size_of(source, style_index) * source.origin.k; + let (x0, y0) = back.apply(from - o.ctm[4], baseline - o.ctm[5] - underline_size * 0.12); + let (x1, y1) = back.apply(to - o.ctm[4], baseline - o.ctm[5] - underline_size * 0.12); block_ops.push(Op::new("RG", vec![n(r), n(g), n(b)])); block_ops.push(Op::new("w", vec![n(underline_size * 0.06 / o.ctm[2].hypot(o.ctm[3]).max(1e-6))])); block_ops.push(Op::new("m", vec![n(x0), n(y0)])); @@ -1792,3 +1899,19 @@ pub(crate) fn rewrite_styled_block( crate::draw_on_top(doc, &p, o.ctm, &on_top)?; Ok(LineEdit { substituted }) } + +#[cfg(test)] +mod tests { + use super::styled_run_end; + + #[test] + fn styled_runs_stop_at_style_or_font_changes() { + let items = vec![('a', 0, "F1".into()), (' ', 0, "F1".into()), ('b', 1, "F1".into()), ('c', 1, "F2".into())]; + assert_eq!(styled_run_end(&items, 0), 2); + assert_eq!(styled_run_end(&items, 1), 2); + assert_eq!(styled_run_end(&items, 2), 3); + assert_eq!(styled_run_end(&items, 3), 4); + assert_eq!(styled_run_end(&items, 4), 4); + assert_eq!(styled_run_end(&[], 0), 0); + } +} diff --git a/crates/edit/src/ungroup.rs b/crates/edit/src/ungroup.rs index 1b1916a1c8..c8b0d2f4d4 100644 --- a/crates/edit/src/ungroup.rs +++ b/crates/edit/src/ungroup.rs @@ -41,11 +41,11 @@ use crate::EditError; pub(crate) const MAX_FORM_BYTES: usize = 64 << 20; /// Resource categories a content stream refers to by name (ISO 32000-2 §7.8.3, Table 34). -const CATEGORIES: [&[u8]; 7] = [b"Font", b"XObject", b"ExtGState", b"ColorSpace", b"Pattern", b"Shading", b"Properties"]; +pub(crate) const CATEGORIES: [&[u8]; 7] = [b"Font", b"XObject", b"ExtGState", b"ColorSpace", b"Pattern", b"Shading", b"Properties"]; /// The `/ColorSpace` entries that remap device colours painted with the resources they are in /// (ISO 32000-2 §8.6.5.6). -const DEFAULT_SPACES: [&[u8]; 3] = [b"DefaultGray", b"DefaultRGB", b"DefaultCMYK"]; +pub(crate) const DEFAULT_SPACES: [&[u8]; 3] = [b"DefaultGray", b"DefaultRGB", b"DefaultCMYK"]; /// The most distinct patterns one ungrouped form may select; each becomes a new object. const MAX_MOVED_PATTERNS: usize = 256; @@ -258,7 +258,7 @@ pub(crate) fn ungroup_ops( // up in the form's (the renderer chains them to whatever draws it). Drawn from the page, it // would find the page's, which differ wherever a name was renamed (every form resource that // isn't already the page's, the nested item's own entry included). - if !renames.is_empty() && ops.iter().any(|op| inherits_resources(doc, &form_res, op)) { + if !renames.is_empty() && ops.iter().any(|op| inherits_resources(doc, &form_res, op, budget)) { return Err(Refusal { renaming: true, ..refuse("it draws a nested form, Type 3 font or pattern that uses its resources") }); } let mut depth = 0usize; @@ -505,7 +505,7 @@ fn shows_text(op: &Op) -> bool { } /// An operator whose operands the renderer can't read as it expects, so it skips the operator. -struct Unreadable; +pub(crate) struct Unreadable; /// Which operand of `op` names a resource, and from which category, read as the renderer reads /// it: operands are taken from the end, so a surplus operand before them is ignored. `Do`, `gs`, @@ -513,7 +513,7 @@ struct Unreadable; /// a name or an inline dictionary; `scn` and `SCN` numbers and the last name among them. /// `Ok(None)` when `op` names no resource; [`Unreadable`] when the renderer would skip it, which /// [`ungroup_ops`] refuses rather than guess at. -fn resource_operand(op: &Op) -> Result, Unreadable> { +pub(crate) fn resource_operand(op: &Op) -> Result, Unreadable> { let operands = &op.operands; let name_at = |i: Option| i.is_some_and(|i| matches!(operands.get(i), Some(Object::Name(_)))); let last = operands.len().checked_sub(1); @@ -553,25 +553,44 @@ fn resource_operand(op: &Op) -> Result, Unreadabl } /// Whether `op` draws a Form XObject (`Do`), Type 3 font (`Tf`) or tiling pattern (`scn`/`SCN`) from -/// `resources` that has no `/Resources` of its own, and so inherits the ones it is drawn with. -fn inherits_resources(doc: &Document, resources: &Dict, op: &Op) -> bool { +/// `resources` that has no `/Resources` of its own, and so inherits the ones it is drawn with. A +/// form or pattern whose content names no resource inherits nothing (a filled shape, say), so it +/// needs no renaming; its content is read within `budget`. A Type 3 font's glyphs are not read, +/// and always count. +fn inherits_resources(doc: &Document, resources: &Dict, op: &Op, budget: &mut DecodeBudget) -> bool { let Ok(Some((cat @ (b"XObject" | b"Font" | b"Pattern"), i))) = resource_operand(op) else { return false }; let Some(Object::Name(name)) = op.operands.get(i) else { return false }; let Some(entry) = resources.get(cat).map(|d| doc.resolve(d)).and_then(|d| d.as_dict().and_then(|d| d.get(name).cloned())) else { return false; }; - let dict = match &*doc.resolve(&entry) { - Object::Stream(s) => s.dict.clone(), - Object::Dict(d) => d.clone(), + let entry = doc.resolve(&entry); + let (dict, stream) = match &*entry { + Object::Stream(s) => (&s.dict, Some(s)), + Object::Dict(d) => (d, None), _ => return false, }; let drawn = match cat { - b"XObject" => is_name(doc, &dict, b"Subtype", b"Form"), - b"Font" => is_name(doc, &dict, b"Subtype", b"Type3"), + b"XObject" => is_name(doc, dict, b"Subtype", b"Form"), + b"Font" => is_name(doc, dict, b"Subtype", b"Type3"), // A shading pattern has no content of its own. - _ => int_of(doc, &dict, b"PatternType") == Some(1), + _ => int_of(doc, dict, b"PatternType") == Some(1), }; - drawn && dict.get(b"Resources").is_none() + drawn && dict.get(b"Resources").is_none() && (cat == b"Font" || stream.is_none_or(|s| names_resources(doc, s, budget))) +} + +/// Whether content stream `s` names a resource (or can't be read within `budget` to tell): device +/// colour spaces are not resources; an inline image counts, as its colour space may be one. +fn names_resources(doc: &Document, s: &pdfcraft_cos::Stream, budget: &mut DecodeBudget) -> bool { + let Ok(data) = budget.decode_strict(doc, s) else { return true }; + let parsed = parse(&data); + parsed.skipped > 0 + || parsed.ops.iter().any(|op| match resource_operand(op) { + Ok(None) => op.inline.is_some(), + Ok(Some((b"ColorSpace", i))) => { + !matches!(op.operands.get(i), Some(Object::Name(n)) if [&b"DeviceGray"[..], b"DeviceRGB", b"DeviceCMYK", b"Pattern"].contains(&n.as_slice())) + } + _ => true, + }) } /// Whether `ops` (drawn with `resources`) blend with something other than Normal, which counts diff --git a/crates/edit/tests/flatten_optional_content.rs b/crates/edit/tests/flatten_optional_content.rs new file mode 100644 index 0000000000..b6a6165149 --- /dev/null +++ b/crates/edit/tests/flatten_optional_content.rs @@ -0,0 +1,472 @@ +//! Optional-content regression fixtures for issue #3213. +use std::sync::Arc; + +use pdfcraft_cos::{Document, ObjRef, Object, SaveOptions, write_incremental}; + +/// Build a single-page PDF fixture with a hidden OCG (`4 0 R`), a shown OCG (`5 0 R`), +/// a form appearance (`7 0 R`), an annotation (`8 0 R`), and an OCMD (`9 0 R`). +fn fixture(membership: &str, flags: i64, appearance_membership: &str) -> Document { + let ink = "1 0 0 rg 0 0 100 50 re f"; + let objects = [ + "<< /Type /Catalog /Pages 2 0 R /OCProperties << /OCGs [4 0 R 5 0 R] /D << /OFF [4 0 R] /ON [5 0 R] /Order [4 0 R 5 0 R] >> >> >>".to_string(), + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>".into(), + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Resources << /Properties << /Existing 5 0 R /PCFlOC0 5 0 R /Private (keep) >> >> /Contents 6 0 R /Annots [8 0 R] >>".into(), + "<< /Type /OCG /Name (Hidden layer) /Private (keep layer) >>".into(), + "<< /Type /OCG /Name (Shown layer) >>".into(), + "<< /Length 0 >>\nstream\n\nendstream".into(), + format!("<< /Type /XObject /Subtype /Form /BBox [0 0 100 50] {appearance_membership} /Length {} >>\nstream\n{ink}\nendstream", ink.len()), + format!("<< /Type /Annot /Subtype /Square /Rect [50 50 150 100] /C [1 0 0] /AP << /N 7 0 R >> /F {flags} {membership} /Private (keep annotation) >>"), + "<< /Type /OCMD /OCGs [4 0 R] /P /AllOn /Private (keep membership) >>".into(), + ]; + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (i, object) in objects.iter().enumerate() { + offsets.push(out.len()); + out.extend_from_slice(format!("{} 0 obj\n{object}\nendobj\n", i + 1).as_bytes()); + } + let xref = out.len(); + out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objects.len() + 1).as_bytes()); + for offset in offsets { + out.extend_from_slice(format!("{offset:010} 00000 n \n").as_bytes()); + } + out.extend_from_slice(format!("trailer\n<< /Root 1 0 R /Size {} >>\nstartxref\n{xref}\n%%EOF\n", objects.len() + 1).as_bytes()); + Document::open(Arc::new(out)).unwrap() +} + +/// Serialize `doc` using an incremental save. +fn saved(doc: &Document) -> Vec { + write_incremental(doc, &SaveOptions::default()).unwrap() +} + +/// Optionally export test PDF bytes when `PDFCRAFT_3213_ARTIFACTS` is set. +fn export(name: &str, stage: &str, bytes: &[u8]) { + if let Some(directory) = std::env::var_os("PDFCRAFT_3213_ARTIFACTS") { + std::fs::write(std::path::Path::new(&directory).join(format!("{name}-{stage}.pdf")), bytes).unwrap(); + } +} + +/// Verify that transferable optional-content memberships are bound to flattened appearances +/// after saving and reopening, while hidden-flag annotations remain untouched. +#[test] +fn flatten_keeps_annotation_optional_content_after_save_and_reopen() { + let mut lost = Vec::new(); + for (name, membership, flags, ap_oc) in [ + ("off", "/OC 4 0 R", 4, ""), + ("on", "/OC 5 0 R", 4, ""), + ("ocmd", "/OC 9 0 R", 4, ""), + ("independent_ap", "/OC 4 0 R", 4, "/OC 5 0 R"), + ("direct_ocmd", "/OC << /Type /OCMD /OCGs [4 0 R] /P /AllOn /Private (keep inline) >>", 4, ""), + ("ocmd_ve", "/OC << /Type /OCMD /VE [/And 4 0 R [/Not 5 0 R]] /Private (keep expression) >>", 4, ""), + ("visible", "", 4, ""), + ("hidden_flag", "", 2, ""), + ] { + let mut doc = fixture(membership, flags, ap_oc); + let original_appearance = (*doc.get(ObjRef::new(7, 0))).clone(); + let expected_membership = doc.get(pdfcraft_cos::ObjRef::new(8, 0)).as_dict().unwrap().get(b"OC").cloned(); + export(name, "before", &saved(&doc)); + let count = pdfcraft_edit::flatten(&mut doc, &[0], true, false).unwrap(); + let bytes = saved(&doc); + export(name, "flat", &bytes); + let reopened = Document::open(Arc::new(bytes)).unwrap(); + assert_eq!(count, usize::from(flags != 2), "{name}"); + let paints = paint_memberships(&reopened); + if flags != 2 && paints != vec![(ObjRef::new(7, 0), expected_membership)] { + lost.push(name); + } + assert_eq!(*reopened.get(ObjRef::new(7, 0)), original_appearance, "{name}: preserve appearance"); + } + assert!(lost.is_empty(), "flattened paint loses its original membership: {lost:?}"); +} + +/// Verify that multiple annotations sharing one appearance stream keep separate memberships +/// without mutating the shared appearance stream itself. +#[test] +fn flatten_shared_appearance_keeps_separate_memberships_and_original_stream() { + let mut doc = fixture("/OC 4 0 R", 4, ""); + let first = ObjRef::new(8, 0); + let ap = ObjRef::new(7, 0); + let original_ap = (*doc.get(ap)).clone(); + let mut second = doc.get(first).as_dict().unwrap().clone(); + second.set(b"OC".to_vec(), Object::Ref(ObjRef::new(5, 0))); + second.set(b"Rect".to_vec(), Object::Array([50, 120, 150, 170].into_iter().map(Object::Int).collect())); + let second = doc.add(Object::Dict(second)); + let mut plain = doc.get(first).as_dict().unwrap().clone(); + plain.remove(b"OC"); + plain.set(b"Rect".to_vec(), Object::Array([0, 0, 20, 10].into_iter().map(Object::Int).collect())); + let plain = doc.add(Object::Dict(plain)); + doc.update_dict(ObjRef::new(3, 0), |page| { + page.set(b"Annots".to_vec(), Object::Array(vec![Object::Ref(first), Object::Ref(second), Object::Ref(plain)])) + }) + .unwrap(); + export("shared", "before", &saved(&doc)); + assert_eq!(pdfcraft_edit::flatten(&mut doc, &[0], true, false).unwrap(), 3); + let bytes = saved(&doc); + export("shared", "flat", &bytes); + let doc = Document::open(Arc::new(bytes)).unwrap(); + assert_eq!(*doc.get(ap), original_ap, "a shared appearance must never acquire either annotation's /OC"); + assert_eq!(paint_memberships(&doc), vec![(ap, Some(Object::Ref(ObjRef::new(4, 0)))), (ap, Some(Object::Ref(ObjRef::new(5, 0)))), (ap, None)]); +} + +/// Associate each `Do` invocation with its active `/OC` marked-content block. +fn paint_memberships(doc: &Document) -> Vec<(ObjRef, Option)> { + let page = pdfcraft_model::pages(doc).remove(0); + let resources = doc.resolve(page.dict.get(b"Resources").unwrap()); + let resources = resources.as_dict().unwrap(); + let properties = resources.get(b"Properties").map(|p| doc.resolve(p)); + let xobjects = resources.get(b"XObject").map(|p| doc.resolve(p)); + let content = doc.resolve(page.dict.get(b"Contents").unwrap()); + let streams = match &*content { + Object::Array(list) => list.clone(), + _ => vec![page.dict.get(b"Contents").unwrap().clone()], + }; + let mut contexts = Vec::new(); + let mut paints = Vec::new(); + for stream in streams { + let resolved = doc.resolve(&stream); + let Object::Stream(stream) = &*resolved else { panic!("content stream") }; + let parsed = pdfcraft_content::parse(&stream.decoded().unwrap()); + assert_eq!(parsed.skipped, 0); + for op in parsed.ops { + if op.is("BDC") { + let membership = if op.name(0) == Some(b"OC") { + Some(properties.as_ref().unwrap().as_dict().unwrap().get(op.name(1).unwrap()).unwrap().clone()) + } else { + None + }; + contexts.push(membership); + } else if op.is("BMC") { + contexts.push(None); + } else if op.is("EMC") { + assert!(contexts.pop().is_some(), "balanced marked content"); + } else if op.is("Do") { + let reference = xobjects.as_ref().unwrap().as_dict().unwrap().reference(op.name(0).unwrap()).unwrap(); + paints.push((reference, contexts.iter().rev().find_map(Clone::clone))); + } + } + } + assert!(contexts.is_empty(), "every optional-content block must close"); + paints +} + +/// Verify that unsupported optional-content memberships retain the annotation, its transitive +/// replies and pop-ups, and its AcroForm field links. +#[test] +fn unsupported_membership_keeps_annotations_replies_and_field_links() { + use pdfcraft_cos::Stream; + let direct_ocg = Object::Dict({ + let mut d = pdfcraft_cos::Dict::new(); + d.set(b"Type".to_vec(), Object::Name(b"OCG".to_vec())); + d + }); + let wrong_type = Object::Dict({ + let mut d = pdfcraft_cos::Dict::new(); + d.set(b"Type".to_vec(), Object::Name(b"Other".to_vec())); + d + }); + for membership in [Object::Null, Object::Int(7), Object::Ref(ObjRef::new(999, 0)), Object::Ref(ObjRef::new(7, 0)), direct_ocg, wrong_type] { + for widget in [false, true] { + let mut doc = fixture("", 4, ""); + let retained = ObjRef::new(8, 0); + let mut successful = doc.get(retained).as_dict().unwrap().clone(); + successful.remove(b"OC"); + let successful = doc.add(Object::Dict(successful)); + let empty_appearance = if widget { + let mut empty = doc.get(ObjRef::new(7, 0)).as_dict().unwrap().clone(); + empty.set(b"BBox".to_vec(), Object::Array(vec![Object::Int(0); 4])); + let reference = doc.add(Object::Stream(Stream::from_raw(empty, Vec::new()))); + let mut ap = pdfcraft_cos::Dict::new(); + ap.set(b"N".to_vec(), Object::Ref(reference)); + Some(Object::Dict(ap)) + } else { + None + }; + doc.update_dict(retained, |d| { + d.set(b"OC".to_vec(), membership.clone()); + d.set(b"IRT".to_vec(), Object::Ref(successful)); + if widget { + d.set(b"Subtype".to_vec(), Object::Name(b"Widget".to_vec())); + d.set(b"AP".to_vec(), empty_appearance.clone().unwrap()); + } else { + d.remove(b"AP"); + } + }) + .unwrap(); + let mut reply = doc.get(retained).as_dict().unwrap().clone(); + reply.remove(b"OC"); + reply.set(b"Subtype".to_vec(), Object::Name(b"Square".to_vec())); + reply.set(b"IRT".to_vec(), Object::Ref(retained)); + reply.set(b"AP".to_vec(), doc.get(successful).as_dict().unwrap().get(b"AP").unwrap().clone()); + let reply = doc.add(Object::Dict(reply)); + let mut grandchild = doc.get(reply).as_dict().unwrap().clone(); + grandchild.set(b"IRT".to_vec(), Object::Ref(reply)); + let grandchild = doc.add(Object::Dict(grandchild)); + let popup = doc.add(Object::Dict({ + let mut d = pdfcraft_cos::Dict::new(); + d.set(b"Subtype".to_vec(), Object::Name(b"Popup".to_vec())); + d.set(b"Parent".to_vec(), Object::Ref(retained)); + d + })); + doc.update_dict(ObjRef::new(3, 0), |d| { + d.set( + b"Annots".to_vec(), + Object::Array(vec![ + Object::Ref(successful), + Object::Ref(retained), + Object::Ref(reply), + Object::Ref(grandchild), + Object::Ref(popup), + ]), + ) + }) + .unwrap(); + if widget { + let fields = doc.add(Object::Dict({ + let mut d = pdfcraft_cos::Dict::new(); + d.set(b"Fields".to_vec(), Object::Array(vec![Object::Ref(retained)])); + d + })); + doc.update_dict(ObjRef::new(1, 0), |d| d.set(b"AcroForm".to_vec(), Object::Ref(fields))).unwrap(); + } + let originals = [retained, reply, grandchild, popup].map(|r| (*doc.get(r)).clone()); + let original_fields = doc.get(ObjRef::new(1, 0)).as_dict().unwrap().get(b"AcroForm").map(|o| (*doc.resolve(o)).clone()); + assert_eq!(pdfcraft_edit::flatten(&mut doc, &[0], true, true).unwrap(), 1, "{membership:?}"); + let doc = Document::open(Arc::new(saved(&doc))).unwrap(); + for (reference, original) in [retained, reply, grandchild, popup].into_iter().zip(originals) { + assert_eq!(*doc.get(reference), original); + } + let page = pdfcraft_model::pages(&doc).remove(0); + assert_eq!( + doc.resolve(page.dict.get(b"Annots").unwrap()).as_array().unwrap(), + &vec![Object::Ref(retained), Object::Ref(reply), Object::Ref(grandchild), Object::Ref(popup)] + ); + assert_eq!(doc.get(ObjRef::new(1, 0)).as_dict().unwrap().get(b"AcroForm").map(|o| (*doc.resolve(o)).clone()), original_fields); + } + } + // A stream that claims Type OCG must not qualify through Object::as_dict(). + let mut doc = fixture("/OC 4 0 R", 4, ""); + let mut d = pdfcraft_cos::Dict::new(); + d.set(b"Type".to_vec(), Object::Name(b"OCG".to_vec())); + doc.set(ObjRef::new(4, 0), Object::Stream(Stream::from_raw(d, Vec::new()))); + let before = saved(&doc); + assert_eq!(pdfcraft_edit::flatten(&mut doc, &[0], true, false).unwrap(), 0); + assert_eq!(saved(&doc), before); + + // Shared-subexpression /VE fan-out (10^16 paths within depth 16) stops at the node budget + // and retains the annotation unchanged instead of hanging. + let mut doc = fixture("/OC 9 0 R", 4, ""); + let mut target = ObjRef::new(4, 0); + for _ in 0..16 { + let mut expr = vec![Object::Name(b"Or".to_vec())]; + expr.extend(std::iter::repeat_n(Object::Ref(target), 10)); + target = doc.add(Object::Array(expr)); + } + let ve = (*doc.get(target)).clone(); + doc.update_dict(ObjRef::new(9, 0), |d| { + d.set(b"VE".to_vec(), ve.clone()); + }) + .unwrap(); + let before = saved(&doc); + assert_eq!(pdfcraft_edit::flatten(&mut doc, &[0], true, false).unwrap(), 0); + assert_eq!(saved(&doc), before); +} + +/// Preserve a malformed reply that targets a retained popup instead of a markup annotation. +#[test] +fn unsupported_membership_keeps_popup_replies_after_save_and_reopen() { + let mut doc = fixture("/OC null", 4, ""); + let root = ObjRef::new(8, 0); + let popup = doc.add(Object::Dict({ + let mut d = pdfcraft_cos::Dict::new(); + d.set(b"Subtype".to_vec(), Object::Name(b"Popup".to_vec())); + d.set(b"Parent".to_vec(), Object::Ref(root)); + d.set(b"Private".to_vec(), Object::String(pdfcraft_cos::PdfString::text("keep popup"))); + d + })); + let mut reply = doc.get(root).as_dict().unwrap().clone(); + reply.remove(b"OC"); + reply.set(b"IRT".to_vec(), Object::Ref(popup)); + let reply = doc.add(Object::Dict(reply)); + let mut drawable = doc.get(root).as_dict().unwrap().clone(); + drawable.remove(b"OC"); + let drawable = doc.add(Object::Dict(drawable)); + doc.update_dict(ObjRef::new(3, 0), |d| { + d.set(b"Annots".to_vec(), Object::Array([root, popup, reply, drawable].map(Object::Ref).to_vec())); + }) + .unwrap(); + let originals = [root, popup, reply].map(|r| (*doc.get(r)).clone()); + assert_eq!(pdfcraft_edit::flatten(&mut doc, &[0], true, false).unwrap(), 1, "only the unrelated annotation can flatten"); + let doc = Document::open(Arc::new(saved(&doc))).unwrap(); + for (reference, original) in [root, popup, reply].into_iter().zip(originals) { + assert_eq!(*doc.get(reference), original, "the retained popup graph stays unchanged"); + } + let page = pdfcraft_model::pages(&doc).remove(0); + assert_eq!(doc.resolve(page.dict.get(b"Annots").unwrap()).as_array().unwrap(), &[root, popup, reply].map(Object::Ref)); + assert_eq!(paint_memberships(&doc), vec![(ObjRef::new(7, 0), None)]); +} + +/// Undrawable text in a retained reply must not block an unrelated drawable annotation. +#[test] +fn unsupported_membership_keeps_undrawable_reply_without_refusing_flatten() { + for marked_appearance in [false, true] { + let mut doc = fixture("/OC null", 4, ""); + let root = ObjRef::new(8, 0); + let mut reply = doc.get(root).as_dict().unwrap().clone(); + reply.remove(b"OC"); + reply.set(b"Subtype".to_vec(), Object::Name(b"FreeText".to_vec())); + reply.set(b"IRT".to_vec(), Object::Ref(root)); + reply.set(b"Contents".to_vec(), Object::String(pdfcraft_cos::PdfString::text("漢"))); + if marked_appearance { + pdfcraft_fonts::set_undrawn_mark(&mut reply, Some('漢')); + } else { + reply.remove(b"AP"); + } + assert_eq!(pdfcraft_annot::appearance::undrawable_text(&reply).map(|(_, c)| c), Some('漢')); + let reply = doc.add(Object::Dict(reply)); + let mut drawable = doc.get(root).as_dict().unwrap().clone(); + drawable.remove(b"OC"); + let drawable = doc.add(Object::Dict(drawable)); + doc.update_dict(ObjRef::new(3, 0), |d| { + d.set(b"Annots".to_vec(), Object::Array([root, reply, drawable].map(Object::Ref).to_vec())); + }) + .unwrap(); + // Compare serialized dictionaries before and after flatten, including PDF string syntax. + let mut doc = Document::open(Arc::new(saved(&doc))).unwrap(); + let originals = [root, reply].map(|r| (*doc.get(r)).clone()); + assert_eq!(pdfcraft_edit::flatten(&mut doc, &[0], true, false).unwrap(), 1, "retained text is not baked into page content"); + let doc = Document::open(Arc::new(saved(&doc))).unwrap(); + for (reference, original) in [root, reply].into_iter().zip(originals) { + assert_eq!(*doc.get(reference), original); + } + let page = pdfcraft_model::pages(&doc).remove(0); + assert_eq!(doc.resolve(page.dict.get(b"Annots").unwrap()).as_array().unwrap(), &[root, reply].map(Object::Ref)); + assert_eq!(paint_memberships(&doc), vec![(ObjRef::new(7, 0), None)]); + } +} + +/// Verify that shared/inherited `/Resources` and `/Properties` dictionaries are never mutated in +/// place and that consecutive flatten runs allocate distinct `/PCFlOC{n}` property names. +#[test] +fn flatten_preserves_shared_resources_and_uses_new_names_on_each_call() { + for inherited in [false, true] { + for indirect in [false, true] { + let mut doc = fixture("/OC 4 0 R", 4, ""); + let mut resources = doc.get(ObjRef::new(3, 0)).as_dict().unwrap().get(b"Resources").unwrap().as_dict().unwrap().clone(); + let original_properties = resources.get(b"Properties").unwrap().clone(); + if indirect { + let r = doc.add(original_properties.clone()); + resources.set(b"Properties".to_vec(), Object::Ref(r)); + } + let resources = if indirect { Object::Ref(doc.add(Object::Dict(resources))) } else { Object::Dict(resources) }; + let owner = ObjRef::new(if inherited { 2 } else { 3 }, 0); + if inherited { + doc.update_dict(ObjRef::new(3, 0), |d| { + d.remove(b"Resources"); + }) + .unwrap(); + } + doc.update_dict(owner, |d| d.set(b"Resources".to_vec(), resources.clone())).unwrap(); + let original_resource_object = (*doc.resolve(&resources)).clone(); + assert_eq!(pdfcraft_edit::flatten(&mut doc, &[0], true, false).unwrap(), 1); + doc.update_dict(ObjRef::new(3, 0), |d| d.set(b"Annots".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(8, 0))]))).unwrap(); + assert_eq!(pdfcraft_edit::flatten(&mut doc, &[0], true, false).unwrap(), 1); + let doc = Document::open(Arc::new(saved(&doc))).unwrap(); + assert_eq!(paint_memberships(&doc), vec![(ObjRef::new(7, 0), Some(Object::Ref(ObjRef::new(4, 0)))); 2]); + let page = pdfcraft_model::pages(&doc).remove(0); + let effective = doc.resolve(page.dict.get(b"Resources").unwrap()); + let properties = doc.resolve(effective.as_dict().unwrap().get(b"Properties").unwrap()); + let properties = properties.as_dict().unwrap(); + for (key, value) in original_properties.as_dict().unwrap().iter() { + assert_eq!(properties.get(key), Some(value)); + } + assert_eq!(properties.len(), original_properties.as_dict().unwrap().len() + 2); + if inherited || indirect { + assert_eq!(*doc.resolve(&resources), original_resource_object); + } + } + } +} + +/// Verify that malformed `/Resources` or `/Properties` entries retain `/OC` annotations without +/// generating appearances, while inline annotations with valid `/OC` still flatten cleanly. +#[test] +fn malformed_resources_and_direct_ocg_stay_unchanged_before_appearance_generation() { + for malformed_resources in [false, true] { + let mut doc = fixture("/OC 4 0 R", 4, ""); + if malformed_resources { + doc.update_dict(ObjRef::new(3, 0), |d| d.set(b"Resources".to_vec(), Object::Null)).unwrap(); + } else { + let mut resources = doc.get(ObjRef::new(3, 0)).as_dict().unwrap().get(b"Resources").unwrap().as_dict().unwrap().clone(); + resources.set(b"Properties".to_vec(), Object::Int(1)); + doc.update_dict(ObjRef::new(3, 0), |d| d.set(b"Resources".to_vec(), Object::Dict(resources))).unwrap(); + } + doc.update_dict(ObjRef::new(8, 0), |d| { + d.remove(b"AP"); + }) + .unwrap(); + let before = saved(&doc); + assert_eq!(pdfcraft_edit::flatten(&mut doc, &[0], true, false).unwrap(), 0); + assert_eq!(saved(&doc), before); + } + let mut doc = fixture("/OC 4 0 R", 4, ""); + let annotation = doc.get(ObjRef::new(8, 0)).as_dict().unwrap().clone(); + doc.update_dict(ObjRef::new(3, 0), |d| d.set(b"Annots".to_vec(), Object::Array(vec![Object::Dict(annotation)]))).unwrap(); + assert_eq!(pdfcraft_edit::flatten(&mut doc, &[0], true, false).unwrap(), 1); + let doc = Document::open(Arc::new(saved(&doc))).unwrap(); + assert_eq!(paint_memberships(&doc), vec![(ObjRef::new(7, 0), Some(Object::Ref(ObjRef::new(4, 0))))]); +} + +/// Inline replies and inline popups pointing to a retained annotation are kept in /Annots. +#[test] +fn unsupported_membership_keeps_inline_replies_and_popups_after_save_and_reopen() { + let mut doc = fixture("/OC null", 4, ""); + let root = ObjRef::new(8, 0); + + let mut inline_reply = doc.get(root).as_dict().unwrap().clone(); + inline_reply.remove(b"OC"); + inline_reply.set(b"IRT".to_vec(), Object::Ref(root)); + + let mut indirect_reply = doc.get(root).as_dict().unwrap().clone(); + indirect_reply.remove(b"OC"); + indirect_reply.set(b"IRT".to_vec(), Object::Ref(root)); + let indirect_reply = doc.add(Object::Dict(indirect_reply)); + + let mut inline_grandchild = doc.get(root).as_dict().unwrap().clone(); + inline_grandchild.remove(b"OC"); + inline_grandchild.set(b"IRT".to_vec(), Object::Ref(indirect_reply)); + + let mut inline_popup = pdfcraft_cos::Dict::new(); + inline_popup.set(b"Subtype".to_vec(), Object::Name(b"Popup".to_vec())); + inline_popup.set(b"Parent".to_vec(), Object::Ref(root)); + + let mut drawable = doc.get(root).as_dict().unwrap().clone(); + drawable.remove(b"OC"); + let drawable = doc.add(Object::Dict(drawable)); + + doc.update_dict(ObjRef::new(3, 0), |d| { + d.set( + b"Annots".to_vec(), + Object::Array(vec![ + Object::Dict(inline_reply.clone()), + Object::Ref(root), + Object::Ref(indirect_reply), + Object::Dict(inline_grandchild.clone()), + Object::Dict(inline_popup.clone()), + Object::Ref(drawable), + ]), + ) + }) + .unwrap(); + + assert_eq!(pdfcraft_edit::flatten(&mut doc, &[0], true, false).unwrap(), 1); + + let doc = Document::open(Arc::new(saved(&doc))).unwrap(); + let page = pdfcraft_model::pages(&doc).remove(0); + let annots = doc.resolve(page.dict.get(b"Annots").unwrap()).as_array().unwrap().clone(); + + assert_eq!(annots.len(), 5); + assert_eq!(annots[0], Object::Dict(inline_reply)); + assert_eq!(annots[1], Object::Ref(root)); + assert_eq!(annots[2], Object::Ref(indirect_reply)); + assert_eq!(annots[3], Object::Dict(inline_grandchild)); + assert_eq!(annots[4], Object::Dict(inline_popup)); +} diff --git a/crates/edit/tests/hairlines_extgstate_lw.rs b/crates/edit/tests/hairlines_extgstate_lw.rs new file mode 100644 index 0000000000..a7e3cb4333 --- /dev/null +++ b/crates/edit/tests/hairlines_extgstate_lw.rs @@ -0,0 +1,88 @@ +//! Fix hairlines follows the line width an ExtGState's `/LW` sets through `gs`: a stroke it +//! leaves thin is found, and the width put back after a widened stroke is that one. + +use pdfcraft_cos::{Document, ObjRef, Object}; +use pdfcraft_edit::{Hairlines, fix_hairlines}; +use std::sync::Arc; + +/// A document of `objects` (object `i + 1` is the `i`th: a dictionary, and stream data when +/// it is a stream); object 1 is the catalog. +fn pdf(objects: &[(&str, Option<&[u8]>)]) -> Document { + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offs = Vec::new(); + for (i, (dict, data)) in objects.iter().enumerate() { + offs.push(out.len()); + out.extend_from_slice(format!("{} 0 obj\n", i + 1).as_bytes()); + match data { + Some(data) => { + out.extend_from_slice(format!("<< {dict} /Length {} >>\nstream\n", data.len()).as_bytes()); + out.extend_from_slice(data); + out.extend_from_slice(b"\nendstream"); + } + None => out.extend_from_slice(dict.as_bytes()), + } + out.extend_from_slice(b"\nendobj\n"); + } + let x = out.len(); + out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objects.len() + 1).as_bytes()); + for o in offs { + out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{x}\n%%EOF\n", objects.len() + 1).as_bytes()); + Document::open(Arc::new(out)).unwrap() +} + +/// A stream's data, its tokens joined by single spaces. +fn flat(doc: &Document, r: ObjRef) -> String { + match &*doc.get(r) { + Object::Stream(s) => String::from_utf8_lossy(&s.decoded().unwrap()).split_whitespace().collect::>().join(" "), + _ => panic!("not a stream"), + } +} + +/// Page `i`'s content (a single stream), flattened. +fn page_text(doc: &Document, i: usize) -> String { + let page = pdfcraft_model::pages(doc).swap_remove(i); + flat(doc, page.dict.get(b"Contents").and_then(Object::as_ref).unwrap()) +} + +/// A one-page document drawing `content`, with `/GS0` setting `/LW lw` and `/GS1` no width. +fn with_gstate(lw: &str, content: &[u8]) -> Document { + let page = format!( + "<< /Type /Page /Parent 2 0 R /Contents 4 0 R /Resources << /ExtGState << /GS0 << /Type /ExtGState /LW {lw} >> /GS1 << /Type /ExtGState /CA 1 >> >> >> >>" + ); + pdf(&[ + ("<< /Type /Catalog /Pages 2 0 R >>", None), + ("<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 800] >>", None), + (page.as_str(), None), + ("", Some(content)), + ]) +} + +#[test] +fn the_width_put_back_is_the_one_gs_set() { + // 2 pt under a tenth scale is 0.2 pt on the page: both strokes are widened, and each gets + // the 2 pt back after it. + let mut doc = with_gstate("2", b"/GS0 gs 0.1 0 0 0.1 0 0 cm 0 0 m 1 0 l S 0 1 m 1 1 l S"); + let fixed = fix_hairlines(&mut doc, &[0], &Hairlines::default()).unwrap(); + assert_eq!(fixed.lines, 2); + let t = page_text(&doc, 0); + assert_eq!(t.matches("l S 2 w").count(), 2, "{t}"); + assert!(!t.contains(" 1 w"), "no stroke is put back to the default width: {t}"); +} + +#[test] +fn a_zero_width_set_by_gs_is_a_hairline() { + let mut doc = with_gstate("0", b"/GS0 gs 0 0 m 1 0 l S"); + let fixed = fix_hairlines(&mut doc, &[0], &Hairlines::default()).unwrap(); + assert_eq!(fixed.lines, 1); + assert_eq!(page_text(&doc, 0), "/GS0 gs 0.72 w 0 0 m 1 0 l S 0 w"); +} + +#[test] +fn a_gs_without_a_width_keeps_the_one_in_effect() { + let mut doc = with_gstate("2", b"0.1 w /GS1 gs 0 0 m 1 0 l S"); + let fixed = fix_hairlines(&mut doc, &[0], &Hairlines::default()).unwrap(); + assert_eq!(fixed.lines, 1); + assert_eq!(page_text(&doc, 0), "0.1 w /GS1 gs 0.72 w 0 0 m 1 0 l S 0.1 w"); +} diff --git a/crates/edit/tests/hairlines_form_inherits_width.rs b/crates/edit/tests/hairlines_form_inherits_width.rs new file mode 100644 index 0000000000..a8ab4e456f --- /dev/null +++ b/crates/edit/tests/hairlines_form_inherits_width.rs @@ -0,0 +1,75 @@ +//! Fix hairlines measures a form's strokes with the line width in effect where the form is +//! drawn (a form inherits the graphics state of its `Do`), and puts that width back after a +//! stroke it widens, in forms drawn by forms too. + +use pdfcraft_cos::{Document, ObjRef, Object}; +use pdfcraft_edit::{Hairlines, fix_hairlines}; +use std::sync::Arc; + +/// A document of `objects` (object `i + 1` is the `i`th: a dictionary, and stream data when +/// it is a stream); object 1 is the catalog. +fn pdf(objects: &[(&str, Option<&[u8]>)]) -> Document { + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offs = Vec::new(); + for (i, (dict, data)) in objects.iter().enumerate() { + offs.push(out.len()); + out.extend_from_slice(format!("{} 0 obj\n", i + 1).as_bytes()); + match data { + Some(data) => { + out.extend_from_slice(format!("<< {dict} /Length {} >>\nstream\n", data.len()).as_bytes()); + out.extend_from_slice(data); + out.extend_from_slice(b"\nendstream"); + } + None => out.extend_from_slice(dict.as_bytes()), + } + out.extend_from_slice(b"\nendobj\n"); + } + let x = out.len(); + out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objects.len() + 1).as_bytes()); + for o in offs { + out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{x}\n%%EOF\n", objects.len() + 1).as_bytes()); + Document::open(Arc::new(out)).unwrap() +} + +/// A stream's data, its tokens joined by single spaces. +fn flat(doc: &Document, r: ObjRef) -> String { + match &*doc.get(r) { + Object::Stream(s) => String::from_utf8_lossy(&s.decoded().unwrap()).split_whitespace().collect::>().join(" "), + _ => panic!("not a stream"), + } +} + +const CATALOG: (&str, Option<&[u8]>) = ("<< /Type /Catalog /Pages 2 0 R >>", None); +const PAGES: (&str, Option<&[u8]>) = ("<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 800] >>", None); +const PAGE: (&str, Option<&[u8]>) = ("<< /Type /Page /Parent 2 0 R /Contents 4 0 R /Resources << /XObject << /Fm0 5 0 R >> >> >>", None); +const FORM: &str = "/Type /XObject /Subtype /Form /BBox [0 0 100 100]"; + +#[test] +fn a_form_without_its_own_width_is_measured_with_the_width_at_its_do() { + // 0.1 pt is in effect where the form is drawn; the form sets no width of its own. + let mut doc = pdf(&[CATALOG, PAGES, PAGE, ("", Some(b"0.1 w /Fm0 Do")), (FORM, Some(b"0 0 m 10 0 l S"))]); + let fixed = fix_hairlines(&mut doc, &[0], &Hairlines::default()).unwrap(); + assert_eq!(fixed.lines, 1); + assert_eq!(flat(&doc, ObjRef { num: 5, generation: 0 }), "0.72 w 0 0 m 10 0 l S 0.1 w"); +} + +#[test] +fn a_widened_stroke_in_a_form_gets_the_inherited_width_back() { + // 0.4 pt at half scale is 0.2 pt on the page; after the stroke the form's later strokes + // must draw with the 0.4 they inherited, not the 1 a fresh graphics state has. + let mut doc = pdf(&[CATALOG, PAGES, PAGE, ("", Some(b"0.5 0 0 0.5 0 0 cm 0.4 w /Fm0 Do")), (FORM, Some(b"0 0 m 1 0 l S"))]); + let fixed = fix_hairlines(&mut doc, &[0], &Hairlines::default()).unwrap(); + assert_eq!(fixed.lines, 1); + assert_eq!(flat(&doc, ObjRef { num: 5, generation: 0 }), "1.44 w 0 0 m 1 0 l S 0.4 w"); +} + +#[test] +fn a_form_drawn_by_a_form_inherits_the_width_too() { + let fm0 = "/Type /XObject /Subtype /Form /BBox [0 0 100 100] /Resources << /XObject << /Fm1 6 0 R >> >>"; + let mut doc = pdf(&[CATALOG, PAGES, PAGE, ("", Some(b"0.1 w /Fm0 Do")), (fm0, Some(b"/Fm1 Do")), (FORM, Some(b"0 0 m 10 0 l S"))]); + let fixed = fix_hairlines(&mut doc, &[0], &Hairlines::default()).unwrap(); + assert_eq!(fixed.lines, 1); + assert_eq!(flat(&doc, ObjRef { num: 6, generation: 0 }), "0.72 w 0 0 m 10 0 l S 0.1 w"); +} diff --git a/crates/edit/tests/hairlines_paths.rs b/crates/edit/tests/hairlines_paths.rs new file mode 100644 index 0000000000..a9f6720ce0 --- /dev/null +++ b/crates/edit/tests/hairlines_paths.rs @@ -0,0 +1,83 @@ +//! Fix hairlines gives a thin stroke a width of its own: set before its path begins (a `w` +//! inside a path is not allowed) and put back after it is painted, so the `w` other strokes +//! share, and strokes under another transformation, keep their width. + +use pdfcraft_cos::{Document, Object}; +use pdfcraft_edit::{Hairlines, fix_hairlines}; +use std::sync::Arc; + +/// A one-page document drawing `content`. +fn one_page(content: &[u8]) -> Document { + let mut objs: Vec> = vec![ + b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(), + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 800] >>".to_vec(), + b"<< /Type /Page /Parent 2 0 R /Contents 4 0 R >>".to_vec(), + ]; + let mut s = format!("<< /Length {} >>\nstream\n", content.len()).into_bytes(); + s.extend_from_slice(content); + s.extend_from_slice(b"\nendstream"); + objs.push(s); + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offs = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offs.push(out.len()); + out.extend_from_slice(format!("{} 0 obj\n", i + 1).as_bytes()); + out.extend_from_slice(o); + out.extend_from_slice(b"\nendobj\n"); + } + let x = out.len(); + out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offs { + out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{x}\n%%EOF\n", objs.len() + 1).as_bytes()); + Document::open(Arc::new(out)).unwrap() +} + +/// The page's content, its tokens joined by single spaces. +fn content_text(doc: &Document) -> String { + let page = pdfcraft_model::pages(doc).swap_remove(0); + let list: Vec = match &*doc.resolve(page.dict.get(b"Contents").unwrap()) { + Object::Array(a) => a.clone(), + _ => vec![page.dict.get(b"Contents").unwrap().clone()], + }; + let mut text = String::new(); + for e in list { + if let Object::Stream(s) = &*doc.resolve(&e) { + text.push_str(&String::from_utf8_lossy(&s.decoded().unwrap())); + text.push(' '); + } + } + text.split_whitespace().collect::>().join(" ") +} + +#[test] +fn a_thin_stroke_does_not_change_the_width_other_strokes_share() { + // The first line is 0.1 pt on the page; the second, under a tenfold scale, is 1 pt. + let mut doc = one_page(b"0.1 w 0 0 m 10 0 l S 10 0 0 10 0 0 cm 0 1 m 1 1 l S"); + let fixed = fix_hairlines(&mut doc, &[0], &Hairlines::default()).unwrap(); + assert_eq!(fixed.lines, 1); + let t = content_text(&doc); + assert!(t.starts_with("0.1 w 0.72 w 0 0 m 10 0 l S 0.1 w 10 0 0 10 0 0 cm"), "the shared w is kept: {t}"); + assert!(!t.contains("7.2"), "the scaled stroke keeps its 1 pt: {t}"); +} + +#[test] +fn a_stroke_inside_q_does_not_change_the_outer_width() { + let mut doc = one_page(b"0.1 w q 0 0 m 1 0 l S Q 0 1 m 1 1 l S"); + let fixed = fix_hairlines(&mut doc, &[0], &Hairlines::default()).unwrap(); + assert_eq!(fixed.lines, 2); + let t = content_text(&doc); + assert!(t.starts_with("0.1 w q 0.72 w 0 0 m 1 0 l S 0.1 w Q 0.72 w 0 1 m 1 1 l S 0.1 w"), "{t}"); +} + +#[test] +fn the_new_width_is_set_before_the_path_not_inside_it() { + // No `w` before the stroke: the default 1 pt at half scale is 0.5 pt on the page. + let mut doc = one_page(b"0.5 0 0 0.5 0 0 cm 0 0 m 10 0 l S 5 5 10 10 re f"); + let fixed = fix_hairlines(&mut doc, &[0], &Hairlines { narrower_than: 0.6, width: 1.0, ..Hairlines::default() }).unwrap(); + assert_eq!(fixed.lines, 1); + let t = content_text(&doc); + assert!(t.contains("cm 2 w 0 0 m 10 0 l S 1 w 5 5 10 10 re f"), "{t}"); + assert!(!t.contains("l 2 w S"), "no w inside the path: {t}"); +} diff --git a/crates/edit/tests/hairlines_shared_form.rs b/crates/edit/tests/hairlines_shared_form.rs new file mode 100644 index 0000000000..99f05499b6 --- /dev/null +++ b/crates/edit/tests/hairlines_shared_form.rs @@ -0,0 +1,165 @@ +//! Fix hairlines rewrites a form for the scale it is drawn under: a form a page not being +//! fixed also draws, or one drawn under two scales, is rewritten as a copy for the use that +//! needs it, so the other uses keep drawing it as it was. + +use pdfcraft_cos::{Document, ObjRef, Object}; +use pdfcraft_edit::{Hairlines, fix_hairlines}; +use std::sync::Arc; + +/// A document of `objects` (object `i + 1` is the `i`th: a dictionary, and stream data when +/// it is a stream); object 1 is the catalog. +fn pdf(objects: &[(&str, Option<&[u8]>)]) -> Document { + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offs = Vec::new(); + for (i, (dict, data)) in objects.iter().enumerate() { + offs.push(out.len()); + out.extend_from_slice(format!("{} 0 obj\n", i + 1).as_bytes()); + match data { + Some(data) => { + out.extend_from_slice(format!("<< {dict} /Length {} >>\nstream\n", data.len()).as_bytes()); + out.extend_from_slice(data); + out.extend_from_slice(b"\nendstream"); + } + None => out.extend_from_slice(dict.as_bytes()), + } + out.extend_from_slice(b"\nendobj\n"); + } + let x = out.len(); + out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objects.len() + 1).as_bytes()); + for o in offs { + out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{x}\n%%EOF\n", objects.len() + 1).as_bytes()); + Document::open(Arc::new(out)).unwrap() +} + +/// A stream's data, its tokens joined by single spaces. +fn flat(doc: &Document, r: ObjRef) -> String { + match &*doc.get(r) { + Object::Stream(s) => String::from_utf8_lossy(&s.decoded().unwrap()).split_whitespace().collect::>().join(" "), + _ => panic!("not a stream"), + } +} + +/// Page `i`'s content (a single stream), flattened. +fn page_text(doc: &Document, i: usize) -> String { + let page = pdfcraft_model::pages(doc).swap_remove(i); + flat(doc, page.dict.get(b"Contents").and_then(Object::as_ref).unwrap()) +} + +/// The object page `i`'s resources draw as `name`. +fn xobject(doc: &Document, i: usize, name: &[u8]) -> ObjRef { + let page = pdfcraft_model::pages(doc).swap_remove(i); + let res = doc.resolve(page.dict.get(b"Resources").unwrap()); + let xo = doc.resolve(res.as_dict().unwrap().get(b"XObject").unwrap()); + xo.as_dict().unwrap().get(name).and_then(Object::as_ref).unwrap() +} + +const FORM: &str = "/Type /XObject /Subtype /Form /BBox [0 0 100 100]"; +const ORIGINAL: &str = "0.4 w 0 0 m 1 0 l S"; + +#[test] +fn a_form_another_page_draws_is_fixed_as_a_copy() { + // Both pages share one resource dictionary; the first draws the form at half scale (its + // 0.4 pt line is 0.2 pt there), the second at full size. + let mut doc = pdf(&[ + ("<< /Type /Catalog /Pages 2 0 R >>", None), + ("<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 /MediaBox [0 0 600 800] >>", None), + ("<< /Type /Page /Parent 2 0 R /Contents 5 0 R /Resources 7 0 R >>", None), + ("<< /Type /Page /Parent 2 0 R /Contents 6 0 R /Resources 7 0 R >>", None), + ("", Some(b"0.5 0 0 0.5 0 0 cm /Fm0 Do")), + ("", Some(b"/Fm0 Do")), + ("<< /XObject << /Fm0 8 0 R >> >>", None), + (FORM, Some(ORIGINAL.as_bytes())), + ]); + let form = ObjRef { num: 8, generation: 0 }; + let fixed = fix_hairlines(&mut doc, &[0], &Hairlines::default()).unwrap(); + assert_eq!(fixed.lines, 1); + assert_eq!(flat(&doc, form), ORIGINAL, "the form the second page draws is unchanged"); + assert_eq!(xobject(&doc, 1, b"Fm0"), form, "the second page still draws it"); + let copy = xobject(&doc, 0, b"Fm0"); + assert_ne!(copy, form, "the first page draws a copy"); + assert_eq!(flat(&doc, copy), "0.4 w 1.44 w 0 0 m 1 0 l S 0.4 w"); +} + +#[test] +fn a_form_drawn_under_two_scales_gets_one_object_for_each() { + let mut doc = pdf(&[ + ("<< /Type /Catalog /Pages 2 0 R >>", None), + ("<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 800] >>", None), + ("<< /Type /Page /Parent 2 0 R /Contents 4 0 R /Resources << /XObject << /Fm0 5 0 R >> >> >>", None), + ("", Some(b"q 0.5 0 0 0.5 0 0 cm /Fm0 Do Q /Fm0 Do")), + (FORM, Some(ORIGINAL.as_bytes())), + ]); + let fixed = fix_hairlines(&mut doc, &[0], &Hairlines::default()).unwrap(); + assert_eq!(fixed.lines, 1); + let text = page_text(&doc, 0); + let words: Vec<&str> = text.split(' ').collect(); + let drawn: Vec<&str> = words.windows(2).filter(|w| w[1] == "Do").map(|w| w[0]).collect(); + assert_eq!(drawn.len(), 2, "{text}"); + assert_eq!(drawn[0], "/Fm0", "{text}"); + assert_ne!(drawn[1], "/Fm0", "the full-size use draws another name: {text}"); + let half = xobject(&doc, 0, b"Fm0"); + let full = xobject(&doc, 0, drawn[1].trim_start_matches('/').as_bytes()); + assert_ne!(half, full); + assert_eq!(flat(&doc, half), "0.4 w 1.44 w 0 0 m 1 0 l S 0.4 w"); + assert_eq!(flat(&doc, full), ORIGINAL, "at full size the 0.4 pt line is left as it was"); +} + +const SHADED: &str = "/Pattern cs /P0 scn 0 0 10 10 re f"; +const PATTERN: &str = "/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 9 9] /XStep 9 /YStep 9 /Resources << >>"; +const THIN: &str = "0 w 0 0 m 9 9 l S"; + +#[test] +fn a_pattern_another_page_uses_is_reported() { + let mut doc = pdf(&[ + ("<< /Type /Catalog /Pages 2 0 R >>", None), + ("<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 /MediaBox [0 0 600 800] >>", None), + ("<< /Type /Page /Parent 2 0 R /Contents 5 0 R /Resources 6 0 R >>", None), + ("<< /Type /Page /Parent 2 0 R /Contents 5 0 R /Resources 6 0 R >>", None), + ("", Some(SHADED.as_bytes())), + ("<< /Pattern << /P0 7 0 R >> >>", None), + (PATTERN, Some(THIN.as_bytes())), + ]); + let h = Hairlines { patterns: true, ..Hairlines::default() }; + let mut both = doc.clone(); + assert_eq!(fix_hairlines(&mut both, &[0, 1], &h).unwrap().elsewhere, 0, "every use is in the run"); + let fixed = fix_hairlines(&mut doc, &[0], &h).unwrap(); + assert_eq!((fixed.lines, fixed.elsewhere), (1, 1)); +} + +#[test] +fn a_pattern_a_form_uses_is_fixed_too() { + let mut doc = pdf(&[ + ("<< /Type /Catalog /Pages 2 0 R >>", None), + ("<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 800] >>", None), + ("<< /Type /Page /Parent 2 0 R /Contents 4 0 R /Resources << /XObject << /Fm0 5 0 R >> >> >>", None), + ("", Some(b"/Fm0 Do")), + // The form lists itself: walking the resources ends all the same. + ( + "/Type /XObject /Subtype /Form /BBox [0 0 100 100] /Resources << /Pattern << /P0 6 0 R >> /XObject << /Fm0 5 0 R >> >>", + Some(SHADED.as_bytes()), + ), + (PATTERN, Some(THIN.as_bytes())), + ]); + let fixed = fix_hairlines(&mut doc, &[0], &Hairlines { patterns: true, ..Hairlines::default() }).unwrap(); + assert_eq!(fixed.lines, 1); + assert_ne!(flat(&doc, ObjRef { num: 6, generation: 0 }), THIN); +} + +#[test] +fn widths_out_of_range_are_refused() { + let mut doc = pdf(&[ + ("<< /Type /Catalog /Pages 2 0 R >>", None), + ("<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 800] >>", None), + ("<< /Type /Page /Parent 2 0 R /Contents 4 0 R >>", None), + ("", Some(THIN.as_bytes())), + ]); + for (narrower_than, width) in [(0.2, -1.0), (0.2, f64::NAN), (0.2, f64::INFINITY), (0.2, 10_000.0), (-1.0, 0.7), (f64::NAN, 0.7), (10_000.0, 0.7)] + { + let h = Hairlines { narrower_than, width, ..Hairlines::default() }; + assert!(fix_hairlines(&mut doc, &[0], &h).is_err(), "{narrower_than} {width}"); + } + let most = Hairlines { narrower_than: Hairlines::MAX_POINTS, width: Hairlines::MAX_POINTS, ..Hairlines::default() }; + assert_eq!(fix_hairlines(&mut doc, &[0], &most).unwrap().lines, 1); +} diff --git a/crates/edit/tests/hairlines_streams.rs b/crates/edit/tests/hairlines_streams.rs new file mode 100644 index 0000000000..f6508822f5 --- /dev/null +++ b/crates/edit/tests/hairlines_streams.rs @@ -0,0 +1,92 @@ +//! Fix hairlines rewrites each of a page's content streams on its own: a stream keeps its +//! dictionary (its `/PCMark` among it), streams that did not change stay as they are, the +//! graphics state carries from one stream to the next, and a page with a stream that cannot +//! be read is left alone instead of losing that stream. + +use pdfcraft_cos::{Document, ObjRef, Object}; +use pdfcraft_edit::{Hairlines, fix_hairlines}; +use std::sync::Arc; + +/// A one-page document whose `/Contents` lists `streams` (dictionary entries, data). +fn one_page(streams: &[(&str, &[u8])]) -> Document { + let refs: Vec = (0..streams.len()).map(|i| format!("{} 0 R", i + 4)).collect(); + let mut objs: Vec> = vec![ + b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(), + b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 800] >>".to_vec(), + format!("<< /Type /Page /Parent 2 0 R /Contents [{}] >>", refs.join(" ")).into_bytes(), + ]; + for (dict, data) in streams { + let mut s = format!("<< {dict} /Length {} >>\nstream\n", data.len()).into_bytes(); + s.extend_from_slice(data); + s.extend_from_slice(b"\nendstream"); + objs.push(s); + } + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offs = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offs.push(out.len()); + out.extend_from_slice(format!("{} 0 obj\n", i + 1).as_bytes()); + out.extend_from_slice(o); + out.extend_from_slice(b"\nendobj\n"); + } + let x = out.len(); + out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offs { + out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{x}\n%%EOF\n", objs.len() + 1).as_bytes()); + Document::open(Arc::new(out)).unwrap() +} + +/// The page's `/Contents` references. +fn contents(doc: &Document) -> Vec { + let page = pdfcraft_model::pages(doc).swap_remove(0); + match &*doc.resolve(page.dict.get(b"Contents").unwrap()) { + Object::Array(a) => a.iter().filter_map(Object::as_ref).collect(), + _ => page.dict.get(b"Contents").and_then(Object::as_ref).into_iter().collect(), + } +} + +fn text_of(doc: &Document, r: ObjRef) -> String { + match &*doc.get(r) { + Object::Stream(s) => String::from_utf8_lossy(&s.decoded().unwrap()).split_whitespace().collect::>().join(" "), + _ => panic!("not a stream"), + } +} + +#[test] +fn each_stream_keeps_its_dictionary_and_only_changed_ones_are_replaced() { + let mut doc = one_page(&[("", b"2 w 0 0 m 1 1 l S"), ("/PCMark /Redaction", b"0 w 0 0 m 1 1 l S")]); + let before = contents(&doc); + let fixed = fix_hairlines(&mut doc, &[0], &Hairlines::default()).unwrap(); + assert_eq!(fixed.lines, 1); + let after = contents(&doc); + assert_eq!(after.len(), 2, "still two streams"); + assert_eq!(after[0], before[0], "the unchanged stream is the same object"); + assert_ne!(after[1], before[1]); + let mark = match &*doc.get(after[1]) { + Object::Stream(s) => s.dict.name(b"PCMark").map(<[u8]>::to_vec), + _ => None, + }; + assert_eq!(mark.as_deref(), Some(&b"Redaction"[..]), "the mark is kept"); + assert!(text_of(&doc, after[1]).contains("0.72 w 0 0 m 1 1 l S")); +} + +#[test] +fn the_graphics_state_carries_from_one_stream_to_the_next() { + // 0.3 pt under the first stream's half scale is 0.15 pt on the page. + let mut doc = one_page(&[("", b"0.5 0 0 0.5 0 0 cm"), ("", b"0.3 w 0 0 m 1 1 l S")]); + let fixed = fix_hairlines(&mut doc, &[0], &Hairlines::default()).unwrap(); + assert_eq!(fixed.lines, 1); + let after = contents(&doc); + assert!(text_of(&doc, after[1]).contains("1.44 w 0 0 m 1 1 l S")); +} + +#[test] +fn a_page_with_an_unreadable_stream_is_left_alone() { + let mut doc = one_page(&[("", b"0 w 0 0 m 1 1 l S"), ("/Filter /DCTDecode", b"not a picture")]); + let before = contents(&doc); + let fixed = fix_hairlines(&mut doc, &[0], &Hairlines::default()).unwrap(); + assert_eq!(fixed.lines, 0); + assert_eq!(contents(&doc), before, "no stream is dropped"); +} diff --git a/crates/edit/tests/it/convert_colours_cmyk_initial.rs b/crates/edit/tests/it/convert_colours_cmyk_initial.rs new file mode 100644 index 0000000000..277aaaaf20 --- /dev/null +++ b/crates/edit/tests/it/convert_colours_cmyk_initial.rs @@ -0,0 +1,71 @@ +//! Convert colours: `/DeviceCMYK cs` starts at black (0 0 0 1, ISO 32000-1 8.6.5.5), so the +//! converted operator is black too, not the white that four zeros would be. + +use pdfcraft_cos::{Document, ObjRef, Object, Stream}; +use pdfcraft_edit::{ColourTarget, ConvertColours, convert_colours}; +use std::sync::Arc; + +/// A document of `objs` (object k is `k + 1 0 obj`; object 1 is the catalog). A `{stream:...}` +/// entry is written as a stream holding the text after the colon, with its `/Length`. +fn pdf(objs: &[&str]) -> Document { + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offs = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offs.push(out.len()); + let body = match o.split_once(">>stream:") { + Some((dict, data)) => format!("{dict} /Length {} >>\nstream\n{data}\nendstream", data.len()), + None => (*o).to_string(), + }; + out.extend_from_slice(format!("{} 0 obj\n{body}\nendobj\n", i + 1).as_bytes()); + } + let x = out.len(); + out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offs { + out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{x}\n%%EOF\n", objs.len() + 1).as_bytes()); + Document::open(Arc::new(out)).unwrap() +} + +fn flat(b: &[u8]) -> String { + String::from_utf8_lossy(b).split_whitespace().collect::>().join(" ") +} + +fn stream_text(doc: &Document, r: ObjRef) -> String { + match &*doc.get(r) { + Object::Stream(s) => flat(&s.decoded().unwrap()), + _ => panic!("not a stream"), + } +} + +/// The page's `/Contents` references. +fn contents(doc: &Document, page: usize) -> Vec { + let p = pdfcraft_model::pages(doc).swap_remove(page); + match &*doc.resolve(p.dict.get(b"Contents").unwrap()) { + Object::Array(a) => a.iter().filter_map(Object::as_ref).collect(), + _ => p.dict.get(b"Contents").and_then(Object::as_ref).into_iter().collect(), + } +} + +fn no_images(_: &Document, _: &Stream, _: ColourTarget) -> Option { + None +} + +fn page_with(content: &str) -> Document { + pdf(&[ + "<< /Type /Catalog /Pages 2 0 R >>", + "<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 800] >>", + "<< /Type /Page /Parent 2 0 R /Contents 4 0 R >>", + &format!("<< >>stream:{content}"), + ]) +} + +#[test] +fn device_cmyk_starts_at_black() { + let mut doc = page_with("/DeviceCMYK cs 0 0 10 10 re f /DeviceCMYK CS 0 0 m 10 10 l S"); + let s = ConvertColours { preserve_black: false, ..ConvertColours::default() }; + let done = convert_colours(&mut doc, &[0], &s, &mut no_images).unwrap(); + assert_eq!(done.colours, 2); + let t = stream_text(&doc, contents(&doc, 0)[0]); + assert!(t.contains("0 g 0 0 10 10 re f") && t.contains("0 G 0 0 m 10 10 l S"), "black, not white: {t}"); +} diff --git a/crates/edit/tests/it/convert_colours_shared.rs b/crates/edit/tests/it/convert_colours_shared.rs new file mode 100644 index 0000000000..bd373011d6 --- /dev/null +++ b/crates/edit/tests/it/convert_colours_shared.rs @@ -0,0 +1,144 @@ +//! Convert colours on some pages leaves the others as they were: a form or picture they share +//! is converted as a copy for the pages being converted, and the resource dictionary they +//! share is not changed. + +use pdfcraft_cos::{Document, ObjRef, Object, Stream}; +use pdfcraft_edit::{ColourTarget, ConvertColours, convert_colours}; +use std::sync::Arc; + +/// A document of `objs` (object k is `k + 1 0 obj`; object 1 is the catalog). A `{stream:...}` +/// entry is written as a stream holding the text after the colon, with its `/Length`. +fn pdf(objs: &[&str]) -> Document { + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offs = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offs.push(out.len()); + let body = match o.split_once(">>stream:") { + Some((dict, data)) => format!("{dict} /Length {} >>\nstream\n{data}\nendstream", data.len()), + None => (*o).to_string(), + }; + out.extend_from_slice(format!("{} 0 obj\n{body}\nendobj\n", i + 1).as_bytes()); + } + let x = out.len(); + out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offs { + out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{x}\n%%EOF\n", objs.len() + 1).as_bytes()); + Document::open(Arc::new(out)).unwrap() +} + +fn flat(b: &[u8]) -> String { + String::from_utf8_lossy(b).split_whitespace().collect::>().join(" ") +} + +fn stream_text(doc: &Document, r: ObjRef) -> String { + match &*doc.get(r) { + Object::Stream(s) => flat(&s.decoded().unwrap()), + _ => panic!("not a stream"), + } +} + +fn no_images(_: &Document, _: &Stream, _: ColourTarget) -> Option { + None +} + +/// Two pages sharing one resource dictionary (inherited from the page tree) whose form draws +/// red, and a form inside it that draws blue. +fn two_pages() -> Document { + pdf(&[ + "<< /Type /Catalog /Pages 2 0 R >>", + "<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 /MediaBox [0 0 600 800] /Resources 5 0 R >>", + "<< /Type /Page /Parent 2 0 R /Contents 6 0 R >>", + "<< /Type /Page /Parent 2 0 R /Contents 6 0 R >>", + "<< /XObject << /Fm1 7 0 R >> >>", + "<< >>stream:/Fm1 Do", + "<< /Type /XObject /Subtype /Form /BBox [0 0 100 100] /Resources << /XObject << /Fm2 8 0 R >> >> >>stream:1 0 0 rg 0 0 10 10 re f /Fm2 Do", + "<< /Type /XObject /Subtype /Form /BBox [0 0 100 100] >>stream:0 0 1 rg 0 0 5 5 re f", + ]) +} + +fn form_of(doc: &Document, page: usize, name: &[u8]) -> ObjRef { + let p = pdfcraft_model::pages(doc).swap_remove(page); + let res = doc.resolve(p.dict.get(b"Resources").unwrap()); + let xo = doc.resolve(res.as_dict().unwrap().get(b"XObject").unwrap()); + xo.as_dict().unwrap().get(name).and_then(Object::as_ref).unwrap() +} + +fn inner_form(doc: &Document, outer: ObjRef) -> ObjRef { + match &*doc.get(outer) { + Object::Stream(s) => { + let res = doc.resolve(s.dict.get(b"Resources").unwrap()); + let xo = doc.resolve(res.as_dict().unwrap().get(b"XObject").unwrap()); + xo.as_dict().unwrap().get(b"Fm2").and_then(Object::as_ref).unwrap() + } + _ => panic!("not a stream"), + } +} + +#[test] +fn a_shared_form_is_converted_as_a_copy() { + let mut doc = two_pages(); + let done = convert_colours(&mut doc, &[0], &ConvertColours::default(), &mut no_images).unwrap(); + assert_eq!(done.colours, 2); + // The other page still draws the original forms, in their colours. + let kept = form_of(&doc, 1, b"Fm1"); + assert!(stream_text(&doc, kept).contains("1 0 0 rg 0 0 10 10 re f"), "{}", stream_text(&doc, kept)); + assert!(stream_text(&doc, inner_form(&doc, kept)).contains("0 0 1 rg 0 0 5 5 re f")); + // The converted page draws copies, the inner one too. + let copy = form_of(&doc, 0, b"Fm1"); + assert_ne!(copy, kept); + assert!(stream_text(&doc, copy).contains("0.299 g 0 0 10 10 re f"), "{}", stream_text(&doc, copy)); + assert!(stream_text(&doc, inner_form(&doc, copy)).contains("0.114 g 0 0 5 5 re f")); +} + +#[test] +fn converting_every_page_changes_the_forms_in_place() { + let mut doc = two_pages(); + let before = form_of(&doc, 0, b"Fm1"); + convert_colours(&mut doc, &[0, 1], &ConvertColours::default(), &mut no_images).unwrap(); + assert_eq!(form_of(&doc, 0, b"Fm1"), before); + assert_eq!(form_of(&doc, 1, b"Fm1"), before); + assert!(stream_text(&doc, before).contains("0.299 g 0 0 10 10 re f")); +} + +/// One page of `content` with `res` as its resources, followed by `more` objects from 5 up. +fn page_of(res: &str, content: &str, more: &[&str]) -> Document { + let page = format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 100 100] /Contents 4 0 R /Resources {res} >>"); + let content = format!("<< >>stream:{content}"); + let mut objs = vec!["<< /Type /Catalog /Pages 2 0 R >>", "<< /Type /Pages /Kids [3 0 R] /Count 1 >>", page.as_str(), content.as_str()]; + objs.extend_from_slice(more); + pdf(&objs) +} + +/// The `/None` colorant paints nothing; a colour of its own would make it appear. +#[test] +fn the_none_colorant_stays() { + let tint = "<< /FunctionType 2 /Domain [0 1] /C0 [0 0 0 0] /C1 [0 0 0 1] /N 1 >>"; + let res = format!( + "<< /ColorSpace << /S [/Separation /None /DeviceCMYK {tint}] /D [/DeviceN [/None /None] /DeviceCMYK {tint}] /Real [/Separation /Red /DeviceCMYK {tint}] >> >>" + ); + let content = "/S cs 1 scn 0 0 9 9 re f /D cs 1 1 scn 0 0 9 9 re f"; + let mut doc = page_of(&res, content, &[]); + assert_eq!(convert_colours(&mut doc, &[0], &ConvertColours::default(), &mut no_images).unwrap().colours, 0); + assert_eq!(stream_text(&doc, ObjRef { num: 4, generation: 0 }), content); + let mut doc = page_of(&res, "/Real cs 1 scn 0 0 9 9 re f", &[]); + assert_eq!(convert_colours(&mut doc, &[0], &ConvertColours::default(), &mut no_images).unwrap().colours, 2, "a real spot is converted"); +} + +/// Pictures the helper cannot re-encode are counted, unless they are in the target already or +/// have no colours of their own. +#[test] +fn pictures_left_alone_are_counted() { + const IMAGE: &str = "/Type /XObject /Subtype /Image /Width 1 /Height 1"; + let indexed = format!("<< {IMAGE} /BitsPerComponent 8 /ColorSpace [/Indexed /DeviceRGB 0 ] >>stream:x"); + let gray_table = format!("<< {IMAGE} /BitsPerComponent 8 /ColorSpace [/Indexed /DeviceGray 0 <80>] >>stream:x"); + let gray = format!("<< {IMAGE} /BitsPerComponent 8 /ColorSpace /DeviceGray >>stream:x"); + let stencil = format!("<< {IMAGE} /BitsPerComponent 1 /ImageMask true >>stream:x"); + let res = "<< /XObject << /A 5 0 R /B 6 0 R /C 7 0 R /D 8 0 R >> >>"; + let mut doc = page_of(res, "/A Do /B Do /C Do /D Do", &[&indexed, &gray_table, &gray, &stencil]); + let mut none = doc.clone(); + assert_eq!(convert_colours(&mut doc, &[0], &ConvertColours::default(), &mut no_images).unwrap().images_left, 1); + let s = ConvertColours { images: false, ..ConvertColours::default() }; + assert_eq!(convert_colours(&mut none, &[0], &s, &mut no_images).unwrap().images_left, 0, "pictures were not asked for"); +} diff --git a/crates/edit/tests/it/convert_colours_streams.rs b/crates/edit/tests/it/convert_colours_streams.rs new file mode 100644 index 0000000000..edad0a32e8 --- /dev/null +++ b/crates/edit/tests/it/convert_colours_streams.rs @@ -0,0 +1,100 @@ +//! Convert colours rewrites each of a page's content streams on its own: a stream keeps its +//! dictionary (its `/PCMark` among it), unchanged streams stay as they are, the colour space +//! set in one stream holds in the next, and a page with an unreadable stream is left alone. + +use pdfcraft_cos::{Document, ObjRef, Object, Stream}; +use pdfcraft_edit::{ColourTarget, ConvertColours, convert_colours}; +use std::sync::Arc; + +/// A document of `objs` (object k is `k + 1 0 obj`; object 1 is the catalog). A `{stream:...}` +/// entry is written as a stream holding the text after the colon, with its `/Length`. +fn pdf(objs: &[&str]) -> Document { + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offs = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offs.push(out.len()); + let body = match o.split_once(">>stream:") { + Some((dict, data)) => format!("{dict} /Length {} >>\nstream\n{data}\nendstream", data.len()), + None => (*o).to_string(), + }; + out.extend_from_slice(format!("{} 0 obj\n{body}\nendobj\n", i + 1).as_bytes()); + } + let x = out.len(); + out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offs { + out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{x}\n%%EOF\n", objs.len() + 1).as_bytes()); + Document::open(Arc::new(out)).unwrap() +} + +fn flat(b: &[u8]) -> String { + String::from_utf8_lossy(b).split_whitespace().collect::>().join(" ") +} + +fn stream_text(doc: &Document, r: ObjRef) -> String { + match &*doc.get(r) { + Object::Stream(s) => flat(&s.decoded().unwrap()), + _ => panic!("not a stream"), + } +} + +/// The page's `/Contents` references. +fn contents(doc: &Document, page: usize) -> Vec { + let p = pdfcraft_model::pages(doc).swap_remove(page); + match &*doc.resolve(p.dict.get(b"Contents").unwrap()) { + Object::Array(a) => a.iter().filter_map(Object::as_ref).collect(), + _ => p.dict.get(b"Contents").and_then(Object::as_ref).into_iter().collect(), + } +} + +fn no_images(_: &Document, _: &Stream, _: ColourTarget) -> Option { + None +} + +/// One page drawing `streams`, with an indexed colour space `/CS0` (0 = red, 1 = green). +fn page_with(streams: &[&str]) -> Document { + let refs: Vec = (0..streams.len()).map(|i| format!("{} 0 R", i + 4)).collect(); + let page = format!( + "<< /Type /Page /Parent 2 0 R /Resources << /ColorSpace << /CS0 [/Indexed /DeviceRGB 1 ] >> >> /Contents [{}] >>", + refs.join(" ") + ); + let mut objs: Vec<&str> = vec!["<< /Type /Catalog /Pages 2 0 R >>", "<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 800] >>", &page]; + objs.extend_from_slice(streams); + pdf(&objs) +} + +#[test] +fn each_stream_keeps_its_dictionary_and_only_changed_ones_are_replaced() { + let mut doc = page_with(&["<< >>stream:0.5 g 0 0 10 10 re f", "<< /PCMark /Redaction >>stream:1 0 0 rg 0 0 10 10 re f"]); + let before = contents(&doc, 0); + let done = convert_colours(&mut doc, &[0], &ConvertColours::default(), &mut no_images).unwrap(); + assert_eq!(done.colours, 1); + let after = contents(&doc, 0); + assert_eq!(after.len(), 2, "still two streams"); + assert_eq!(after[0], before[0], "the unchanged stream is the same object"); + let mark = match &*doc.get(after[1]) { + Object::Stream(s) => s.dict.name(b"PCMark").map(<[u8]>::to_vec), + _ => None, + }; + assert_eq!(mark.as_deref(), Some(&b"Redaction"[..]), "the mark is kept"); + assert!(stream_text(&doc, after[1]).contains("0.299 g 0 0 10 10 re f")); +} + +#[test] +fn the_colour_space_carries_from_one_stream_to_the_next() { + // Index 0 of the space set in the first stream is red; read as a gray, it would be black. + let mut doc = page_with(&["<< >>stream:/CS0 cs", "<< >>stream:0 sc 0 0 10 10 re f"]); + convert_colours(&mut doc, &[0], &ConvertColours::default(), &mut no_images).unwrap(); + let after = contents(&doc, 0); + assert!(stream_text(&doc, after[1]).contains("0.299 g 0 0 10 10 re f"), "{}", stream_text(&doc, after[1])); +} + +#[test] +fn a_page_with_an_unreadable_stream_is_left_alone() { + let mut doc = page_with(&["<< >>stream:1 0 0 rg 0 0 10 10 re f", "<< /Filter /DCTDecode >>stream:not a picture"]); + let before = contents(&doc, 0); + let done = convert_colours(&mut doc, &[0], &ConvertColours::default(), &mut no_images).unwrap(); + assert_eq!(done.colours, 0); + assert_eq!(contents(&doc, 0), before, "no stream is dropped"); +} diff --git a/crates/edit/tests/it/main.rs b/crates/edit/tests/it/main.rs new file mode 100644 index 0000000000..b64db49da1 --- /dev/null +++ b/crates/edit/tests/it/main.rs @@ -0,0 +1,5 @@ +//! The crate's integration tests, linked into one test binary. + +mod convert_colours_cmyk_initial; +mod convert_colours_shared; +mod convert_colours_streams; diff --git a/crates/edit/tests/printer_marks_accented_label.rs b/crates/edit/tests/printer_marks_accented_label.rs new file mode 100644 index 0000000000..ccb8eb5861 --- /dev/null +++ b/crates/edit/tests/printer_marks_accented_label.rs @@ -0,0 +1,12 @@ +//! The page information line keeps accented letters: WinAnsi bytes from 0x80 up must reach the +//! content stream as escapes, not be replaced with U+FFFD. + +use pdfcraft_edit::PrinterMarks; + +#[test] +fn accented_label_is_written_as_winansi_escapes() { + let marks = PrinterMarks { label: "Café.pdf".into(), ..PrinterMarks::default() }; + let content = marks.content([0.0, 0.0, 612.0, 792.0], [0.0, 0.0, 612.0, 792.0], 1, 1); + assert!(!content.contains('\u{FFFD}'), "no replacement character in the stream: {content:?}"); + assert!(content.contains("(Caf\\351.pdf 1 of 1)"), "é is the WinAnsi byte 0xE9 written as \\351: {content:?}"); +} diff --git a/crates/edit/tests/printer_marks_bleed_reach.rs b/crates/edit/tests/printer_marks_bleed_reach.rs new file mode 100644 index 0000000000..cb7590768c --- /dev/null +++ b/crates/edit/tests/printer_marks_bleed_reach.rs @@ -0,0 +1,50 @@ +//! Bleed marks start at the bleed box, so the room made for the marks counts the bleed too. + +use pdfcraft_cos::{Dict, Document, Object}; +use pdfcraft_edit::{PrinterMarks, add_printer_marks}; + +fn rect(r: [f64; 4]) -> Object { + Object::Array(r.iter().map(|v| Object::Real(*v)).collect()) +} + +fn page_with(boxes: &[(&[u8], [f64; 4])]) -> Document { + let mut doc = Document::new_empty(); + let pages = doc.get(doc.root().unwrap()).as_dict().unwrap().reference(b"Pages").unwrap(); + let mut page = Dict::new(); + page.set(b"Type".to_vec(), Object::name("Page")); + page.set(b"Parent".to_vec(), Object::Ref(pages)); + for (key, r) in boxes { + page.set(key.to_vec(), rect(*r)); + } + let page = doc.add(page); + doc.update_dict(pages, |d| { + d.set(b"Kids".to_vec(), Object::Array(vec![Object::Ref(page)])); + d.set(b"Count".to_vec(), Object::Int(1)); + }) + .unwrap(); + doc +} + +fn page_box(doc: &Document, key: &[u8]) -> Option> { + let page = pdfcraft_model::pages(doc).first().cloned()?; + let a = page.dict.get(key).map(|o| doc.resolve(o))?; + Some(a.as_array()?.iter().filter_map(|v| doc.resolve(v).as_f64()).collect()) +} + +/// A 9 pt bleed: the bleed marks run out to 9 + 6 + 18 = 33 pt from the trim box, further than the +/// usual 24, so the crop box grows by that much. +#[test] +fn crop_box_grows_to_hold_the_bleed_marks() { + let mut doc = page_with(&[ + (b"MediaBox", [0.0, 0.0, 1000.0, 1000.0]), + (b"CropBox", [200.0, 200.0, 800.0, 800.0]), + (b"TrimBox", [200.0, 200.0, 800.0, 800.0]), + (b"BleedBox", [191.0, 191.0, 809.0, 809.0]), + ]); + let marks = PrinterMarks::default(); + assert_eq!(marks.reach(), 24.0); + let widened = add_printer_marks(&mut doc, &[0], &marks, false).unwrap(); + assert_eq!(widened, vec![0]); + assert_eq!(page_box(&doc, b"CropBox"), Some(vec![167.0, 167.0, 833.0, 833.0])); + assert_eq!(page_box(&doc, b"TrimBox"), Some(vec![200.0, 200.0, 800.0, 800.0])); +} diff --git a/crates/edit/tests/printer_marks_boxes.rs b/crates/edit/tests/printer_marks_boxes.rs new file mode 100644 index 0000000000..d236c48197 --- /dev/null +++ b/crates/edit/tests/printer_marks_boxes.rs @@ -0,0 +1,108 @@ +//! Printer marks and the page boxes: the marks must show inside the crop box, and running +//! again must not move the trim box or grow the page. + +use pdfcraft_cos::{Dict, Document, Object}; +use pdfcraft_edit::{PrinterMarks, add_printer_marks}; + +fn rect(r: [f64; 4]) -> Object { + Object::Array(r.iter().map(|v| Object::Real(*v)).collect()) +} + +/// One empty page with these boxes set on it. +fn page_with(boxes: &[(&[u8], [f64; 4])]) -> Document { + let mut doc = Document::new_empty(); + let pages = doc.get(doc.root().unwrap()).as_dict().unwrap().reference(b"Pages").unwrap(); + let mut page = Dict::new(); + page.set(b"Type".to_vec(), Object::name("Page")); + page.set(b"Parent".to_vec(), Object::Ref(pages)); + for (key, r) in boxes { + page.set(key.to_vec(), rect(*r)); + } + let page = doc.add(page); + doc.update_dict(pages, |d| { + d.set(b"Kids".to_vec(), Object::Array(vec![Object::Ref(page)])); + d.set(b"Count".to_vec(), Object::Int(1)); + }) + .unwrap(); + doc +} + +fn page_box(doc: &Document, key: &[u8]) -> Option> { + let page = pdfcraft_model::pages(doc).first().cloned()?; + let a = page.dict.get(key).map(|o| doc.resolve(o))?; + Some(a.as_array()?.iter().filter_map(|v| doc.resolve(v).as_f64()).collect()) +} + +/// A crop box cut to the trim inside a large media box: the marks fit the media box but +/// not the crop box, so the crop box grows to show them and the trim box stays. +#[test] +fn printer_marks_widen_crop_inside_media() { + let mut doc = page_with(&[(b"MediaBox", [0.0, 0.0, 1000.0, 1000.0]), (b"CropBox", [200.0, 200.0, 800.0, 800.0])]); + let marks = PrinterMarks::default(); + let reach = marks.reach(); + let widened = add_printer_marks(&mut doc, &[0], &marks, false).unwrap(); + assert_eq!(widened, vec![0]); + assert_eq!(page_box(&doc, b"MediaBox"), Some(vec![0.0, 0.0, 1000.0, 1000.0]), "the media box already had room"); + let crop = page_box(&doc, b"CropBox").unwrap(); + assert_eq!(crop, vec![200.0 - reach, 200.0 - reach, 800.0 + reach, 800.0 + reach]); + assert_eq!(page_box(&doc, b"TrimBox"), Some(vec![200.0, 200.0, 800.0, 800.0])); +} + +/// No trim box given: it defaults to the crop box, which widening moves. Running again with +/// replace must keep the trim box and the page size from the first run. +#[test] +fn printer_marks_rerun_keeps_trim() { + let mut doc = page_with(&[(b"MediaBox", [0.0, 0.0, 612.0, 792.0])]); + let marks = PrinterMarks::default(); + add_printer_marks(&mut doc, &[0], &marks, false).unwrap(); + let media = page_box(&doc, b"MediaBox"); + let crop = page_box(&doc, b"CropBox"); + assert_eq!(page_box(&doc, b"TrimBox"), Some(vec![0.0, 0.0, 612.0, 792.0])); + assert_eq!(page_box(&doc, b"BleedBox"), Some(vec![0.0, 0.0, 612.0, 792.0])); + let widened = add_printer_marks(&mut doc, &[0], &marks, true).unwrap(); + assert!(widened.is_empty(), "the page already has room"); + assert_eq!(page_box(&doc, b"MediaBox"), media, "the page did not grow again"); + assert_eq!(page_box(&doc, b"CropBox"), crop); + assert_eq!(page_box(&doc, b"TrimBox"), Some(vec![0.0, 0.0, 612.0, 792.0])); +} + +/// A trim box past the media box is clipped to it. Widening the media box must not let it +/// grow to what was written. +#[test] +fn printer_marks_keep_the_trim_box_in_effect() { + let mut doc = page_with(&[(b"MediaBox", [0.0, 0.0, 100.0, 100.0]), (b"TrimBox", [-20.0, -20.0, 120.0, 120.0])]); + add_printer_marks(&mut doc, &[0], &PrinterMarks::default(), false).unwrap(); + assert_eq!(page_box(&doc, b"TrimBox"), Some(vec![0.0, 0.0, 100.0, 100.0])); +} + +/// Line weights and room out of range are refused, not clamped, and nothing past the largest +/// page a PDF can have is written. +#[test] +fn printer_marks_refuse_numbers_out_of_range() { + let mut doc = page_with(&[(b"MediaBox", [0.0, 0.0, 612.0, 792.0])]); + for line_weight in [-1.0, 0.0, 0.05, 5.0, 10_000.0, f64::NAN, f64::INFINITY] { + assert!(add_printer_marks(&mut doc, &[0], &PrinterMarks { line_weight, ..PrinterMarks::default() }, false).is_err(), "{line_weight}"); + } + for room in [-1.0, 289.0, 1e9, f64::NAN, f64::INFINITY] { + assert!(add_printer_marks(&mut doc, &[0], &PrinterMarks { make_room: Some(room), ..PrinterMarks::default() }, false).is_err(), "{room}"); + } + let mut big = page_with(&[(b"MediaBox", [0.0, 0.0, 14_400.0, 1000.0])]); + assert!(add_printer_marks(&mut big, &[0], &PrinterMarks::default(), false).is_err()); + assert!(add_printer_marks(&mut big, &[0], &PrinterMarks { make_room: None, ..PrinterMarks::default() }, false).is_ok(), "left as it is"); +} + +/// Registration colour is the /All Separation, in the page's resources beside what it had. +#[test] +fn printer_marks_use_the_all_separation() { + let mut doc = page_with(&[(b"MediaBox", [0.0, 0.0, 612.0, 792.0])]); + add_printer_marks(&mut doc, &[0], &PrinterMarks::default(), false).unwrap(); + let page = pdfcraft_model::pages(&doc).swap_remove(0); + let res = doc.resolve(page.dict.get(b"Resources").unwrap()); + let res = res.as_dict().unwrap(); + assert!(res.get(b"Font").is_some()); + let spaces = doc.resolve(res.get(b"ColorSpace").unwrap()); + let cs = doc.resolve(spaces.as_dict().unwrap().get(b"PCReg").unwrap()); + let cs = cs.as_array().unwrap(); + assert_eq!(cs[..3], [Object::name("Separation"), Object::name("All"), Object::name("DeviceCMYK")]); + assert!(cs[3].as_dict().unwrap().int(b"FunctionType") == Some(2)); +} diff --git a/crates/engine/src/catalog.rs b/crates/engine/src/catalog.rs index d6be8fd4eb..1f757f98e4 100644 --- a/crates/engine/src/catalog.rs +++ b/crates/engine/src/catalog.rs @@ -409,10 +409,10 @@ pub static TOOL_GROUPS: &[ToolGroup] = &[ items: &[ item("Output preview", "layers", "prepress.output_preview", Planned("M11")), item("Preflight", "file-check", "preflight.run", Planned("M11")), - item("Convert colors", "palette", "prepress.convert_colors", Planned("M11")), + item("Convert colours", "palette", "prepress.convert_colors", Ready), item("Set page boxes", "square-dashed-mouse-pointer", "page.boxes", Ready), - item("Add printer marks", "ruler", "prepress.marks", Planned("M11")), - item("Fix hairlines", "minus", "prepress.hairlines", Planned("M11")), + item("Add printer marks", "ruler", "prepress.marks", Ready), + item("Fix hairlines", "minus", "prepress.hairlines", Ready), ], }], }, diff --git a/crates/engine/src/commands.rs b/crates/engine/src/commands.rs index 7cee14657e..1996bcd793 100644 --- a/crates/engine/src/commands.rs +++ b/crates/engine/src/commands.rs @@ -228,6 +228,19 @@ pub const COMMANDS: &[CommandSpec] = &[ c("edit.snapshot", "Take a snapshot", EDIT, None, Copying, "camera"), c("view.full_screen", "Full screen mode", VIEW, Some(Shortcut::cmd("L")), Document, "maximize"), c("view.read_mode", "Read mode", VIEW, Some(Shortcut { command: true, shift: false, mac_ctrl: true, key: "H" }), Document, "book-open"), + // Split view (Acrobat's Window ▸ Split): the current document in two panes, or back to one; + // a tab's right-click menu also splits right or down with any tab. + c("view.split", "Split", VIEW, Some(Shortcut::cmd("\\")), Document, "columns-2"), + // A document can hide the toolbars, menu and panels (ISO 32000-2 Table 147); this brings them + // back for its tab, and with nothing hidden hides them. + c( + "view.toggle_interface", + "Show or hide toolbars and panels", + VIEW, + Some(Shortcut { command: false, shift: false, mac_ctrl: false, key: "F8" }), + Document, + "panel-left", + ), // Browsers' tab keys (#1542): Ctrl+Tab on every platform, ⌃Tab rather than ⌘Tab on macOS, // where the system takes ⌘Tab. They follow the tabs' order on screen and wrap around, and // also run from Combine files, which has no document open. @@ -403,6 +416,10 @@ pub const COMMANDS: &[CommandSpec] = &[ c("page.extract", "Extract pages…", PAGES, None, Assembly, "file-output"), c("page.split", "Split document…", PAGES, None, Assembly, "scissors"), c("page.number", "Number pages…", PAGES, None, Assembly, "hash"), + c("prepress.marks", "Add printer marks…", None, None, Modification, "ruler"), + c("prepress.marks.remove", "Remove printer marks", None, None, Marks(crate::MarkKind::PrinterMarks), "ruler"), + c("prepress.convert_colors", "Convert colours…", None, None, Modification, "palette"), + c("prepress.hairlines", "Fix hairlines…", None, None, Modification, "minus"), c("page.properties", "Page properties…", PAGES, None, Modification, "info"), c("help.shortcuts", "Keyboard shortcuts", HELP, None, Nothing, "circle-help"), c("help.discord", "Join the ArtCraft Discord", HELP, None, Nothing, "messages-square"), diff --git a/crates/engine/src/comments.rs b/crates/engine/src/comments.rs index c8262a7241..a700e38f17 100644 --- a/crates/engine/src/comments.rs +++ b/crates/engine/src/comments.rs @@ -24,9 +24,12 @@ pub(crate) fn comment_list(doc: &pdfcraft_cos::Document) -> Vec &str { } } -/// [`comment_type_name`], telling callouts, clouds and typewriter text apart by `/IT`. +/// [`comment_type_name`], telling replaced text, callouts, clouds and typewriter text apart by `/IT`. pub fn comment_kind(a: &pdfcraft_render::Annotation) -> &str { match a.intent.as_deref() { + Some("StrikeOutTextEdit" | "Replace") => "Replace Text", Some("FreehandHighlight") if a.subtype == "Ink" => "Freehand Highlight", Some("FreeTextCallout") => "Callout", Some("PolygonCloud") => "Cloud", @@ -106,19 +110,26 @@ pub fn comment_summary(name: &str, all: &[pdfcraft_render::Annotation], sort: Su } // Write a comment's replies, indenting each level. A reply to a reply keeps its `/IRT` on the // direct parent, so following one level would leave the deeper replies out of the summary. - fn write_replies(out: &mut String, children: &Children<'_>, parent: &pdfcraft_render::Annotation, depth: usize) { + // A group subordinate (`/RT /Group`, ISO 32000-2 §12.5.6.2) is part of the comment, not a + // reply: it is not written, and replies to it are written at its level, as the comment's. + // `depth` bounds the walk; `level` is the indent. + fn write_replies(out: &mut String, children: &Children<'_>, parent: &pdfcraft_render::Annotation, depth: usize, level: usize) { use std::fmt::Write; if depth > 64 { return; } for r in children.get(&(parent.page, parent.index)).into_iter().flatten() { - let indent = " ".repeat(depth + 1); + if r.is_group_member() { + write_replies(out, children, r, depth + 1, level); + continue; + } + let indent = " ".repeat(level + 1); let _ = writeln!(out, "{indent}Author: {} Subject: Reply Date: {}", r.author.as_deref().unwrap_or(""), r.modified.as_deref().unwrap_or("")); for line in r.contents.as_deref().unwrap_or("").lines() { let _ = writeln!(out, "{indent}{line}"); } - write_replies(out, children, r, depth + 1); + write_replies(out, children, r, depth + 1, level + 1); } } let mut numbered: Vec<(usize, &pdfcraft_render::Annotation)> = Vec::new(); @@ -157,7 +168,7 @@ pub fn comment_summary(name: &str, all: &[pdfcraft_render::Annotation], sort: Su if let Some(c) = a.contents.as_deref().filter(|c| !c.is_empty()) { let _ = writeln!(out, "{c}"); } - write_replies(&mut out, &children, a, 0); + write_replies(&mut out, &children, a, 0, 0); out.push('\n'); } out diff --git a/crates/engine/src/copy_image.rs b/crates/engine/src/copy_image.rs index 9d5668ccf5..6e8c2af8d4 100644 --- a/crates/engine/src/copy_image.rs +++ b/crates/engine/src/copy_image.rs @@ -2,7 +2,7 @@ //! file in its original format. A picture drawn through grouped //! Form artwork (as Word, InDesign and many scanners write them) is found inside the group. -use std::collections::HashSet; +use std::collections::{HashMap, HashSet}; use pdfcraft_cos::{Dict, Document as Cos, ObjRef, Object}; @@ -13,6 +13,31 @@ const MAX_COPY_PIXELS: u64 = 64_000_000; /// How deep Form artwork is searched for its picture. const MAX_FORM_DEPTH: usize = 12; +/// A picture on a page, with its index in [`Document::page_images`]. +pub type PagePicture = (usize, pdfcraft_edit::PageImage); + +/// The topmost of `pictures` under user-space point `at` (`[x, y]`). `pictures` is bottom first, +/// as [`Document::page_pictures_job`] lists them, so the last one that holds `at` wins. +pub fn picture_at(pictures: &[PagePicture], at: [f64; 2]) -> Option<&PagePicture> { + pictures.iter().rev().find(|(_, image)| image.contains(at)) +} + +/// The pictures on `page`, bottom first: raster images, and Form artwork that holds one. +fn pictures(cos: &Cos, page: usize) -> Vec { + let images = pdfcraft_edit::page_images(cos, page).unwrap_or_default(); + // Artwork drawn many times is searched once. + let mut holds_one: HashMap = HashMap::new(); + images + .into_iter() + .enumerate() + .filter(|(_, i)| match (i.is_form, i.object) { + (false, Some(_)) => true, + (true, Some(form)) => *holds_one.entry(form).or_insert_with(|| picture_in_form(cos, form).is_some()), + _ => false, + }) + .collect() +} + impl Document { /// The topmost picture on `page` under user-space point `at`, as its index in /// [`Document::page_images`]: a raster image, or Form artwork that holds one. @@ -28,6 +53,18 @@ impl Document { }) } + /// Work that lists the pictures on `page`, bottom first, for another thread: it reads every + /// drawing operator on the page, seconds on a dense one, so it must not run on the window + /// thread. Look a point up with [`picture_at`]. + /// + /// `None` when the document can't be read. The work reads the document as it is now: later + /// edits aren't seen (key its answer by [`Document::edit_generation`]). It returns no pictures + /// for a page that doesn't exist or can't be read. + pub fn page_pictures_job(&self, page: usize) -> Option Vec + Send + 'static> { + let cos = self.editor.as_ref()?.cos.clone(); + Some(move || crate::guard(|| pictures(&cos, page)).unwrap_or_default()) + } + /// Copy Image in Original Format: picture `index` on `page` as a file (extension, bytes): /// a JPEG as stored in the PDF, otherwise a lossless PNG. pub fn page_image_copy_file(&self, page: usize, index: usize) -> Result<(&'static str, Vec), String> { diff --git a/crates/engine/src/document.rs b/crates/engine/src/document.rs index 22667377c3..530f8787af 100644 --- a/crates/engine/src/document.rs +++ b/crates/engine/src/document.rs @@ -32,7 +32,8 @@ pub struct Document { pub dirty: bool, /// Bumped on every change to the working file (edit, undo, redo, save); autosave compares it. pub(crate) generation: u64, - /// The generation last handed out by `autosave_snapshots`. + /// The generation last handed out by `autosave_snapshots`, or another one when storing that + /// snapshot failed (`autosave_failed`), so it is due again. pub(crate) snapshot_generation: u64, /// Why the document cannot be edited (e.g. encryption), if so. pub read_only_reason: Option, @@ -58,6 +59,10 @@ pub struct Document { pub(crate) xfa_template: Option>, /// A script of the XFA form ran away, so its scripts are off (see [`Document::xfa_scripts_off`]). pub(crate) xfa_scripts_off: bool, + /// SOM paths of the XFA fields whose exit scripts ran with a value change ([`xfa::on_changes`]) + /// since they last gained focus: their exit on losing focus has run already + /// ([`Session::field_focus`]). + pub(crate) xfa_exited: std::collections::HashSet, /// A content-only refresh keeps `info` from before the edit; an edit may have added or /// embedded fonts, so the font list is read again on demand ([`Session::refresh_fonts`]). pub(crate) fonts_stale: bool, @@ -65,8 +70,9 @@ pub struct Document { /// (also in `info.warnings`, kept there when the document is re-read). pub xfa_warnings: Vec, /// What the last successful [`Session::apply`](crate::Session::apply) wants said (see - /// [`EditNotice`]): text its fonts can't draw, pages whose own backdrop stays off. The - /// interface shows it; tools return it. + /// [`EditNotice`]): text its fonts can't draw, pages whose own backdrop stays off, places + /// applied redactions left that still hold the removed text. The interface shows it; tools + /// return it. pub edit_notices: Vec, } @@ -81,6 +87,21 @@ pub enum EditNotice { /// turning it on could paint over something (`pdfcraft_edit::backdrops_left_off`). They look /// the same on white paper. BackdropsLeftOff { pages: Vec }, + /// Applying redactions left these places as they were, and they still hold the removed text + /// (bookmark titles, attachments, the comments that remain, XMP on other objects; see + /// [`RedactionElsewhere`](crate::RedactionElsewhere)). Only when one is counted, or the text + /// couldn't be searched for. + RedactionElsewhere(pdfcraft_redact::Elsewhere), + /// Applying redactions removed the document's XMP packet, which held the removed text and + /// said the file conforms to PDF/A or PDF/UA: each claim was written back in a packet of its + /// own, or lost (see [`RedactionIdentification`](crate::RedactionIdentification)). + RedactionIdentification(pdfcraft_redact::Identification), +} + +/// The standards whose identification `id` has as `claim`: "PDF/A", "PDF/UA" or both, joined. +pub(crate) fn standards_with(id: &pdfcraft_redact::Identification, claim: pdfcraft_redact::Claim) -> Option { + let names: Vec<&str> = [("PDF/A", id.pdfa), ("PDF/UA", id.pdfua)].into_iter().filter(|(_, c)| *c == claim).map(|(n, _)| n).collect(); + (!names.is_empty()).then(|| names.join(" and ")) } impl std::fmt::Display for EditNotice { @@ -95,10 +116,50 @@ impl std::fmt::Display for EditNotice { page_runs(pages) ) } + EditNotice::RedactionElsewhere(e) => elsewhere_text(e, f), + EditNotice::RedactionIdentification(id) => { + use pdfcraft_redact::Claim; + write!(f, "the document's XMP metadata held the redacted text and was removed")?; + if let Some(kept) = standards_with(id, Claim::Kept) { + write!(f, "; its {kept} identification was written back on its own")?; + } + match standards_with(id, Claim::Lost) { + Some(lost) => write!(f, "; its {lost} identification couldn't be kept, so the file no longer says it conforms to {lost}"), + None => Ok(()), + } + } } } } +/// [`EditNotice::RedactionElsewhere`] in words (English; the interface has its own wording). +fn elsewhere_text(e: &pdfcraft_redact::Elsewhere, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + let count = |n: usize, one: &str, many: &str| match n { + 0 => None, + 1 => Some(format!("1 {one}")), + n => Some(format!("{n} {many}")), + }; + let places: Vec = [ + count(e.bookmarks, "bookmark title", "bookmark titles"), + count(e.attachments, "attachment", "attachments"), + count(e.comments, "comment", "comments"), + count(e.xmp, "other metadata stream", "other metadata streams"), + ] + .into_iter() + .flatten() + .collect(); + let list = match places.split_last() { + None => "no place counted".to_string(), + Some((last, [])) => last.clone(), + Some((last, rest)) => format!("{} and {last}", rest.join(", ")), + }; + if e.unsearched { + write!(f, "the redacted text couldn't be searched for, so {list} may still hold it (Remove hidden information removes them)") + } else { + write!(f, "the redacted text also appears in {list}, which redaction doesn't change (Remove hidden information removes them)") + } +} + /// 0-based pages as 1-based runs: `[0, 1, 2, 5]` is "1-3, 6". pub(crate) fn page_runs(pages: &[usize]) -> String { let mut runs: Vec<(usize, usize)> = Vec::new(); @@ -185,6 +246,26 @@ pub(crate) fn undrawn_notices(prev: &pdfcraft_cos::Document, next: &pdfcraft_cos } impl Document { + /// The places the last edit's applied redactions left that still hold the removed text + /// ([`EditNotice::RedactionElsewhere`]), or `None` when there are none or the last edit + /// applied none. + pub fn redaction_elsewhere(&self) -> Option { + self.edit_notices.iter().find_map(|n| match n { + EditNotice::RedactionElsewhere(e) => Some(*e), + _ => None, + }) + } + + /// What became of the PDF/A and PDF/UA identification when the last edit's applied + /// redactions removed the document's XMP packet ([`EditNotice::RedactionIdentification`]), + /// or `None` when no packet that made such a claim was removed. + pub fn redaction_identification(&self) -> Option { + self.edit_notices.iter().find_map(|n| match n { + EditNotice::RedactionIdentification(id) => Some(*id), + _ => None, + }) + } + /// How the document opens (Document Properties ▸ Initial View). /// Accessibility ▸ Check for accessibility: the full check (`None` if the document can't be /// read for editing). @@ -303,6 +384,22 @@ impl Document { measure::snap::geometry(&e.cos, page).map_err(|e| e.to_string()) } + /// [`Self::measurement_paths`] within `limits`. + pub fn measurement_paths_within(&self, page: usize, limits: measure::snap::Limits) -> Result { + let e = self.editor.as_ref().ok_or("the document can't be read")?; + measure::snap::geometry_with_limits(&e.cos, page, limits).map_err(|e| e.to_string()) + } + + /// [`Self::measurement_paths`] as work to run away from the UI: a heavy drawing takes + /// seconds to read (millions of segments). It works on a copy of the document as it is now. + pub fn measurement_paths_job(&self, page: usize) -> impl FnOnce() -> Result + Send + 'static { + let cos = self.editor.as_ref().map(|e| e.cos.clone()); + move || { + let cos = cos.ok_or("the document can't be read")?; + measure::snap::geometry(&cos, page).map_err(|e| e.to_string()) + } + } + /// A counter that changes with every edit (for caches of derived data). pub fn edit_generation(&self) -> u64 { self.generation @@ -402,10 +499,16 @@ impl Document { self.undo_label().map(Label::as_str) } + /// What Redo would redo, in English, when there is something to redo. pub fn can_redo(&self) -> Option<&str> { self.redo_label().map(Label::as_str) } + /// How many steps Undo can take back: a step still at this depth is still the last one. + pub fn undo_depth(&self) -> usize { + self.editor.as_ref().map_or(0, |e| e.undo.len()) + } + /// What Undo would undo, in parts, for an interface that says it in another language. pub fn undo_label(&self) -> Option<&Label> { self.editor.as_ref().and_then(|e| e.undo.last()).map(|(l, ..)| l) @@ -425,16 +528,22 @@ impl Document { self.editor.as_ref().and_then(|e| e.cos.permissions()) } + /// The renderer opened an encrypted file that the editing parser did not, so its permission + /// bits are unknown. Unknown is not unrestricted: printing and copying are refused. + fn permissions_unknown(&self) -> bool { + self.editor.is_none() && self.info.encrypted + } + /// Printing is allowed (Table 22, bit 3). pub fn allows_printing(&self) -> bool { - self.permissions().is_none_or(|p| p.print()) + !self.permissions_unknown() && self.permissions().is_none_or(|p| p.print()) } /// Copying or otherwise extracting text and images is allowed (Table 22, bit 5): copying /// selected text, snapshots, Save Image As, exporting. Assistive technology may read the /// content regardless (§7.6.4.2), so reading aloud and accessibility tools don't ask. pub fn allows_copying(&self) -> bool { - self.permissions().is_none_or(|p| p.copy()) + !self.permissions_unknown() && self.permissions().is_none_or(|p| p.copy()) } /// [`Self::allows_copying`], as the error to report when it isn't. @@ -589,6 +698,12 @@ impl Document { self.editor.as_ref().map(|e| pdfcraft_redact::sanitize::scan(&e.cos)).unwrap_or_default() } + /// [`Self::hidden_info`] for the categories in `which` alone, without the reading the others + /// need (see [`pdfcraft_redact::sanitize::scan_only`]). + pub fn hidden_info_of(&self, which: &[Hidden]) -> Vec<(Hidden, usize)> { + self.editor.as_ref().map(|e| pdfcraft_redact::sanitize::scan_only(&e.cos, which)).unwrap_or_default() + } + /// Verification: faux redactions (opaque covers over extractable text) and redaction /// marks that were never applied. Fails closed: any read error is returned, never an /// empty list. @@ -727,6 +842,14 @@ pub(crate) fn write_display_copy( } } +/// `doc` as the reader shows it: comments without an appearance stream drawn +/// (`pdfcraft_annot::with_missing_appearances`) and form fields drawn from their values where the +/// file leaves that to the viewer. `None` when `doc` needs neither. The vector print draws from +/// this copy too, so what prints is what the page shows (#371, #3475 draw them for display only). +pub(crate) fn as_displayed(doc: &pdfcraft_cos::Document) -> Option { + with_field_appearances(pdfcraft_annot::with_missing_appearances(doc), doc) +} + /// Carry the marks across a revalidation that leaves the display copy alone, when it may: only /// when every verdict is the one the copy was drawn with. Returns whether it did; otherwise the /// copy has to be drawn again. @@ -778,7 +901,7 @@ pub fn display_bytes_for(bytes: Arc>, password: Option<&str>) -> Arc (display, true), None if layered => (cos, false), None => return None, diff --git a/crates/engine/src/edit.rs b/crates/engine/src/edit.rs index f1a99a2387..1d5181bf65 100644 --- a/crates/engine/src/edit.rs +++ b/crates/engine/src/edit.rs @@ -3,11 +3,11 @@ use std::sync::Arc; use crate::{ - AddedContent, AddedText, Background, BorderShape, BoxSpec, EditError, FieldProps, FieldValue, FillMark, HeaderFooter, Hidden, ImageEdit, - InitialView, Label, LineEnding, LinkAction, LinkStyle, MarkFile, MarkKind, Markup, NewAnnotation, NewField, NoteIcon, ObjectTarget, PageBox, - Protection, ReviewState, Rgb, Shape, Standard, Style, TabOrder, TransitionSpec, Watermark, a11y, copy_image, decode_data_file, embed_image, - field_table, fit_picture, image_content_type, is_xfa_data_file, js, mark_source, measure, model_poster, open_source, strip_xfa, unrestricted, - xfa, xfa_after_data_change, + AddedContent, AddedText, Background, BorderShape, BoxSpec, EditError, FieldProps, FieldValue, FillMark, Hairlines, HeaderFooter, Hidden, + ImageEdit, InitialView, Label, LineEnding, LinkAction, LinkStyle, MarkFile, MarkKind, Markup, NewAnnotation, NewField, NoteIcon, ObjectTarget, + PageBox, PrinterMarks, Protection, ReviewState, Rgb, Shape, Standard, Style, TabOrder, TransitionSpec, Watermark, a11y, copy_image, + decode_data_file, embed_image, field_table, fit_picture, image_content_type, is_xfa_data_file, js, mark_source, measure, model_poster, + open_source, strip_xfa, unrestricted, xfa, xfa_after_data_change, }; mod labels; @@ -160,12 +160,14 @@ pub enum Edit { /// `rect` is a point: the stamp is centred there at its natural size (at most 200 pt). /// Any other `rect` is a box: the picture is scaled to fit inside it, keeping its shape, /// and centred in it (#4077). + /// `line` is a text line under the picture, already filled in ([`Session::stamp_line`](crate::Session::stamp_line)). AddCustomStamp { page: usize, rect: [f64; 4], name: String, file: MarkFile, author: String, + line: Option, }, /// Rich Media ▸ Add 3D: a U3D or PRC `model` as a 3D annotation in `rect` on `page`, named /// `name` (its alternate description). Its poster is the picture in `poster` (a PDF page or an @@ -243,6 +245,14 @@ pub enum Edit { index: usize, points: Vec<[f64; 2]>, }, + /// Open, close or move a comment's pop-up window (`rect` in user space; a comment without a + /// pop-up gets one). + SetAnnotationPopup { + page: usize, + index: usize, + rect: Option<[f64; 4]>, + open: Option, + }, /// Resize a rectangle, oval or text box. ResizeAnnotation { page: usize, @@ -349,6 +359,20 @@ pub enum Edit { }, /// Document Properties ▸ Initial View (and the language and binding). SetInitialView(Box), + /// Print production ▸ Add printer marks: trim and bleed marks, registration targets, colour + /// bars and page information outside the trim box, as a page mark; the page may be widened + /// to make room (`settings.make_room`), so the page geometry is read again. + AddPrinterMarks { + pages: Vec, + settings: PrinterMarks, + replace: bool, + }, + /// Print production ▸ Fix hairlines: strokes narrower than `settings.narrower_than` points on + /// the page (after every transformation; zero-width lines always) get `settings.width`. + FixHairlines { + pages: Vec, + settings: Hairlines, + }, /// Edit ▸ Attach a file: a document-level attachment (`/Names /EmbeddedFiles`) under /// `name`, made unique; `mime` is the embedded file's `/Subtype` when known. AddAttachment { @@ -361,6 +385,16 @@ pub enum Edit { DeleteAttachment { key: String, }, + /// Print production ▸ Convert colours: every colour on the pages in grayscale, RGB or CMYK. + ConvertColours { + pages: Vec, + settings: pdfcraft_edit::ConvertColours, + }, + /// Change or clear only `/Desc` on a document-level attachment's file specification. + EditAttachmentDescription { + key: String, + description: Option, + }, /// Build a tagged structure tree for an untagged, text-only document. AutoTagText, /// Add alternate text: set (or clear, with `None`) a figure's alternate text. `figure` is @@ -391,6 +425,11 @@ pub enum Edit { MarkTagArtifact { object: pdfcraft_cos::ObjRef, }, + /// A layout table as plain blocks: the table, its row groups and rows become Div tags and its + /// cells P tags (Div when they hold tags), without their Table attributes. + LayoutTableToBlocks { + object: pdfcraft_cos::ObjRef, + }, /// Document JavaScripts: add, replace (`script`) or remove (`None`) the document-level /// script `name`. SetDocumentScript { @@ -457,6 +496,12 @@ pub enum Edit { objects: Vec, offset: [f64; 2], }, + /// Delete mixed existing and added objects together (references from one inventory of + /// `page`), as one step: what Delete and Cut do to objects selected together. + DeleteObjects { + page: usize, + objects: Vec, + }, /// Edit text in a paragraph box: replace paragraph `block` (from `Document::text_blocks`), /// rewrapped to the box's width. EditTextBlock { @@ -549,6 +594,13 @@ pub enum Edit { content: AddedContent, file: Option<(String, Arc>)>, }, + /// Edit ▸ Paste of objects copied together in Edit a PDF ([`Document::copy_objects`](crate::Document::copy_objects)), in this + /// document or another: drawn on top of `page` as they were, moved by `offset` (user space). + PasteObjects { + page: usize, + clip: Arc, + offset: [f64; 2], + }, /// Apply redaction marks (all, or those on `pages`): remove what they cover for good. ApplyRedactions { pages: Option>, diff --git a/crates/engine/src/edit/labels.rs b/crates/engine/src/edit/labels.rs index da7d718811..be1cb1a0a4 100644 --- a/crates/engine/src/edit/labels.rs +++ b/crates/engine/src/edit/labels.rs @@ -31,6 +31,7 @@ pub const EDIT_LABELS: &[&str] = &[ "Insert pages from {name} — fields renamed", "Attach {name}", "Delete attachment {key}", + "Edit attachment description", "Add bookmark", "Rename bookmark", "Delete bookmark", @@ -87,6 +88,9 @@ pub const EDIT_LABELS: &[&str] = &[ "Unlock comment", "Move comment", "Reshape comment", + "Open comment pop-up", + "Close comment pop-up", + "Move comment pop-up", "Resize comment", "Change comment properties", "Fill in {name}", @@ -232,6 +236,9 @@ impl Edit { Edit::LockAnnotation { .. } => Label::of("Unlock comment"), Edit::MoveAnnotation { .. } => Label::of("Move comment"), Edit::ReshapeAnnotation { .. } => Label::of("Reshape comment"), + Edit::SetAnnotationPopup { rect: Some(_), .. } => Label::of("Move comment pop-up"), + Edit::SetAnnotationPopup { open: Some(true), .. } => Label::of("Open comment pop-up"), + Edit::SetAnnotationPopup { .. } => Label::of("Close comment pop-up"), Edit::ResizeAnnotation { .. } => Label::of("Resize comment"), Edit::StyleAnnotation { .. } | Edit::FillAnnotation { .. } @@ -250,8 +257,11 @@ impl Edit { Edit::SetTabOrder { .. } | Edit::MoveInTabOrder { .. } => Label::of("Set tab order"), Edit::SetCalculationOrder { .. } => Label::of("Set calculation order"), Edit::SetInitialView(_) => Label::of("Change initial view"), + Edit::FixHairlines { .. } => Label::of("Fix hairlines"), Edit::AddAttachment { name, .. } => Label::with("Attach {name}", name.as_str()), Edit::DeleteAttachment { key } => Label::with("Delete attachment {key}", key.as_str()), + Edit::ConvertColours { .. } => Label::of("Convert colours"), + Edit::EditAttachmentDescription { .. } => Label::of("Edit attachment description"), Edit::AutoTagText => Label::of("Automatically tag text"), Edit::SetAltText { .. } => Label::of("Set alternate text"), Edit::AutoTagFormFields => Label::of("Tag form fields"), @@ -259,6 +269,7 @@ impl Edit { Edit::MoveTag { .. } => Label::of("Move object"), Edit::MarkDecorative { .. } => Label::of("Mark figure as decorative"), Edit::MarkTagArtifact { .. } => Label::of("Mark content as artifact"), + Edit::LayoutTableToBlocks { .. } => Label::of("Change tag properties"), Edit::AddOcrText { .. } => Label::of("Recognize text"), Edit::EnhanceScan { .. } => Label::of("Enhance scanned file"), Edit::ApplyScriptChanges { .. } => Label::of("Run JavaScript"), @@ -299,12 +310,14 @@ impl Edit { Edit::RemoveMarks { kind: MarkKind::HeaderFooter } => Label::of("Remove header & footer"), Edit::RemoveMarks { kind: MarkKind::Watermark } => Label::of("Remove watermark"), Edit::RemoveMarks { kind: MarkKind::Background } => Label::of("Remove background"), + Edit::RemoveMarks { kind: MarkKind::PrinterMarks } => Label::of("Remove printer marks"), + Edit::AddPrinterMarks { .. } => Label::of("Add printer marks"), Edit::AddText { .. } => Label::of("Add text"), Edit::AddImage { .. } => Label::of("Add image"), Edit::UpdateContent { .. } => Label::of("Edit content"), - Edit::DeleteContent { .. } => Label::of("Delete content"), + Edit::DeleteContent { .. } | Edit::DeleteObjects { .. } => Label::of("Delete content"), Edit::ReplaceImage { .. } => Label::of("Replace image"), - Edit::PasteContent { .. } => Label::of("Paste"), + Edit::PasteContent { .. } | Edit::PasteObjects { .. } => Label::of("Paste"), Edit::ApplyRedactions { .. } => Label::of("Apply redactions"), Edit::ClearRedactions => Label::of("Remove redaction marks"), Edit::RemoveHidden { .. } => Label::of("Remove hidden information"), diff --git a/crates/engine/src/edit/run.rs b/crates/engine/src/edit/run.rs index 82a302d769..eb213f3f64 100644 --- a/crates/engine/src/edit/run.rs +++ b/crates/engine/src/edit/run.rs @@ -62,6 +62,7 @@ pub(crate) fn check_permission(edit: &Edit, p: &pdfcraft_cos::Permissions) -> Re | Edit::LockAnnotation { .. } | Edit::MoveAnnotation { .. } | Edit::ReshapeAnnotation { .. } + | Edit::SetAnnotationPopup { .. } | Edit::ResizeAnnotation { .. } | Edit::StyleAnnotation { .. } | Edit::FillAnnotation { .. } @@ -91,6 +92,11 @@ pub(crate) fn check_permission(edit: &Edit, p: &pdfcraft_cos::Permissions) -> Re Err(EditError::NotPermitted("changing security")) } } + // Converting the appearances of comments and form fields changes them too, which + // content changes alone do not allow. + Edit::ConvertColours { settings, .. } if settings.annotations && !(p.modify() && p.annotate()) => { + Err(EditError::NotPermitted(if p.modify() { "comments" } else { "changes to the document" })) + } // Converting to PDF/A takes the encryption off (PDF/A forbids it), so on an encrypted // document it is a change of security (#3921). Edit::ConvertPdfA { .. } => { @@ -102,6 +108,7 @@ pub(crate) fn check_permission(edit: &Edit, p: &pdfcraft_cos::Permissions) -> Re } Edit::SetInfo { .. } | Edit::AddHeaderFooter { .. } + | Edit::AddPrinterMarks { .. } | Edit::AddWatermark { .. } | Edit::AddBackground { .. } | Edit::RemoveMarks { .. } @@ -111,9 +118,11 @@ pub(crate) fn check_permission(edit: &Edit, p: &pdfcraft_cos::Permissions) -> Re | Edit::AddImage { .. } | Edit::UpdateContent { .. } | Edit::MoveObjects { .. } + | Edit::DeleteObjects { .. } | Edit::DeleteContent { .. } | Edit::ReplaceImage { .. } | Edit::PasteContent { .. } + | Edit::PasteObjects { .. } | Edit::ClearRedactions | Edit::RemoveHidden { .. } | Edit::ImportData { .. } @@ -132,8 +141,11 @@ pub(crate) fn check_permission(edit: &Edit, p: &pdfcraft_cos::Permissions) -> Re | Edit::MoveInTabOrder { .. } | Edit::SetCalculationOrder { .. } | Edit::SetInitialView(_) + | Edit::FixHairlines { .. } | Edit::AddAttachment { .. } | Edit::DeleteAttachment { .. } + | Edit::ConvertColours { .. } + | Edit::EditAttachmentDescription { .. } | Edit::AutoTagText | Edit::SetAltText { .. } | Edit::AutoTagFormFields @@ -141,6 +153,7 @@ pub(crate) fn check_permission(edit: &Edit, p: &pdfcraft_cos::Permissions) -> Re | Edit::MoveTag { .. } | Edit::MarkDecorative { .. } | Edit::MarkTagArtifact { .. } + | Edit::LayoutTableToBlocks { .. } | Edit::AddOcrText { .. } | Edit::EnhanceScan { .. } | Edit::SetDocumentScript { .. } @@ -163,6 +176,47 @@ pub(crate) fn check_permission(edit: &Edit, p: &pdfcraft_cos::Permissions) -> Re } } +/// What a comment import left out of comments it did import (an appearance it could not read), +/// in the repair log as one line: said, not slipped in. The first five are named, each in at +/// most [`SKIPPED_CHARS`] characters with control characters escaped (a name from the file +/// holding a line break would otherwise read as a repair of its own). +fn note_skipped(doc: &mut pdfcraft_cos::Document, report: &pdfcraft_xfdf::Report) { + if report.skipped.is_empty() { + return; + } + let shown: Vec = report.skipped.iter().take(5).map(|s| skipped_text(s)).collect(); + let more = report.skipped.len().saturating_sub(shown.len()); + let more = if more > 0 { format!("; and {more} more") } else { String::new() }; + doc.note_repair(format!( + "imported {} comment{} without the appearance the file gave {} (PdfCraft draws it where it can; it may look different or not show): {}{more}", + report.skipped.len(), + if report.skipped.len() == 1 { "" } else { "s" }, + if report.skipped.len() == 1 { "it" } else { "them" }, + shown.join("; ") + )); +} + +/// At most this many characters of one skipped comment's description in the repair log. +const SKIPPED_CHARS: usize = 200; + +/// `s` for one line of the repair log: control characters and line separators escaped, cut to +/// [`SKIPPED_CHARS`] characters. +fn skipped_text(s: &str) -> String { + let mut out = String::new(); + for (n, c) in s.chars().enumerate() { + if n == SKIPPED_CHARS { + out.push('…'); + break; + } + if c.is_control() || matches!(c, '\u{2028}' | '\u{2029}') { + out.extend(c.escape_default()); + } else { + out.push(c); + } + } + out +} + /// Dates and unique ids stamped onto what an edit creates. pub(crate) struct EditCtx { /// Field scripts run (`None`: JavaScript is off) and what they produced. @@ -175,12 +229,16 @@ pub(crate) struct EditCtx { pub(crate) xfa_datasets: Option, /// A form script ran too long and was abandoned: the document's scripts go off. pub(crate) xfa_ran_away: bool, + /// An XFA enter or exit event whose scripts changed nothing: no edit to keep. + pub(crate) xfa_unchanged: bool, /// What the XFA scripts' time limits are multiplied by ([`Session::with_xfa_time_scale`]). pub(crate) xfa_time_scale: u32, /// The edit took the XFA out of the form: what went with it, for the document's warnings. pub(crate) xfa_removed: Vec, /// Pages in the document, for `xfa.layout.pageCount()`. pub(crate) pages: usize, + /// Whether the viewer highlights form fields, for `app.runtimeHighlight`. + pub(crate) highlight: bool, pub(crate) date: Option, /// Today in local time, for date tokens. pub(crate) today: (i64, u32, u32), @@ -194,12 +252,34 @@ pub(crate) struct EditCtx { /// The overlay streams the edit's applied redactions added, which alone the renderer's check /// leaves out. pub(crate) redaction_overlays: Vec, + /// What the edit's applied redactions left as it was that still holds the removed text. + pub(crate) redaction_elsewhere: Option, + /// What became of the PDF/A and PDF/UA identification when the edit's applied redactions + /// removed the catalog's XMP packet. + pub(crate) redaction_identification: pdfcraft_redact::Identification, /// What [`Edit::PrepareText`] prepared, repaired and left grouped, for /// [`Session::prepare_text`](crate::Session::prepare_text). pub(crate) prepared: Option, } impl EditCtx { + /// The field highlight (`app.runtimeHighlight`) the edit's next script reads: the field + /// scripts' runner holds it while there is one (its scripts change it there), else the edit. + pub(crate) fn highlight(&self) -> bool { + self.js.as_ref().map_or(self.highlight, |js| js.doc.highlight) + } + + /// A script changed the field highlight: both script runtimes of the edit read the new one, + /// so a later edit of a batch sees it whichever kind of script it runs. + pub(crate) fn set_highlight(&mut self, on: bool) { + self.highlight = on; + if let Some(js) = self.js.as_mut() { + js.doc.highlight = on; + } + } + + /// The context for one edit: `now` (the session's clock) dates it, and with `salt` seeds the + /// ids it makes. pub(crate) fn new(now: Option, salt: u64) -> Self { let mut seed = 0x9E37_79B9_7F4A_7C15u64 ^ salt; if let Some(t) = now { @@ -216,9 +296,11 @@ impl EditCtx { xfa_out: Default::default(), xfa_datasets: None, xfa_ran_away: false, + xfa_unchanged: false, xfa_time_scale: 1, xfa_removed: Vec::new(), pages: 0, + highlight: false, date: now.map(pdfcraft_cos::pdf_date), today: (1970, 1, 1), seed, @@ -226,6 +308,8 @@ impl EditCtx { rng: getrandom::fill, renamed_fields: Vec::new(), redaction_overlays: Vec::new(), + redaction_elsewhere: None, + redaction_identification: pdfcraft_redact::Identification::default(), prepared: None, } } @@ -512,10 +596,14 @@ pub(crate) fn run_edit(doc: &mut pdfcraft_cos::Document, edit: &Edit, cx: &mut E if report.comments == 0 { return Err(EditError::Invalid("the clipboard holds no comments".into())); } + note_skipped(doc, &report); } - Edit::AddCustomStamp { page, rect, name, file, author } => { + Edit::AddCustomStamp { page, rect, name, file, author, line } => { let src = mark_source(doc, file)?; - let (sw, sh) = (src.size.0.max(1.0), src.size.1.max(1.0)); + let line = line.as_deref().map(str::trim).filter(|l| !l.is_empty()).map(|l| l.chars().take(crate::MAX_STAMP_LINE).collect::()); + let picture = (src.size.0.max(1.0), src.size.1.max(1.0)); + // A text line adds its band under the picture, and widens the stamp to fit (#1741). + let (sw, sh) = pdfcraft_annot::custom_stamp_size(picture, line.as_deref()); let natural = rect[2] == rect[0] && rect[3] == rect[1]; let rotation = pdfcraft_organize::page_rotation(doc, *page)?; // The picture as displayed: a quarter turn swaps its sides in user space. @@ -527,7 +615,7 @@ pub(crate) fn run_edit(doc: &mut pdfcraft_cos::Document, edit: &Edit, cx: &mut E } else { fit_picture(*rect, dw, dh) }; - let shape = Shape::CustomStamp { rect, name: name.clone(), picture: src.xobject, image: src.image, size: (sw, sh) }; + let shape = Shape::CustomStamp { rect, name: name.clone(), picture: src.xobject, image: src.image, size: picture, line }; let style = pdfcraft_annot::Style::default_for(&shape); let new = NewAnnotation { page: *page, shape, style, contents: name.clone(), author: author.clone() }; let index = pdfcraft_annot::add_annotation(doc, &new, &cx.meta())?; @@ -554,13 +642,36 @@ pub(crate) fn run_edit(doc: &mut pdfcraft_cos::Document, edit: &Edit, cx: &mut E Edit::EraseInk { page, index, path, radius } => { pdfcraft_annot::erase_ink(doc, *page, *index, path, *radius, &cx.meta())?; } + Edit::FixHairlines { pages, settings } => { + let fixed = pdfcraft_edit::fix_hairlines(doc, pages, settings)?; + if fixed.lines == 0 { + return Err(EditError::Edit(pdfcraft_edit::EditError::Invalid("no hairlines found on those pages".into()))); + } + } Edit::AddAttachment { name, bytes, description, mime } => { pdfcraft_organize::add_attachment(doc, name, bytes, description.as_deref(), mime.as_deref(), cx.date.as_deref())?; } Edit::DeleteAttachment { key } => pdfcraft_organize::delete_attachment(doc, key)?, + Edit::ConvertColours { pages, settings } => { + let mut recolour = |doc: &pdfcraft_cos::Document, s: &pdfcraft_cos::Stream, t: pdfcraft_edit::ColourTarget| { + let target = match t { + pdfcraft_edit::ColourTarget::Gray => pdfcraft_optimize::ImageTarget::Gray, + pdfcraft_edit::ColourTarget::Rgb => pdfcraft_optimize::ImageTarget::Rgb, + pdfcraft_edit::ColourTarget::Cmyk => pdfcraft_optimize::ImageTarget::Cmyk, + }; + pdfcraft_optimize::recolour_image(doc, s, target) + }; + let done = pdfcraft_edit::convert_colours(doc, pages, settings, &mut recolour)?; + if done.colours == 0 && done.images == 0 { + return Err(EditError::Edit(pdfcraft_edit::EditError::Invalid("nothing on those pages needed converting".into()))); + } + } + Edit::EditAttachmentDescription { key, description } => { + pdfcraft_organize::set_attachment_description(doc, key, description.as_deref())?; + } Edit::ReplaceText { page, quads, text, author, strike, caret } => { - // One `/NM` each: the caret replies to the strikeout, and XFDF names that reply by - // the strikeout's `/NM` (#2666). + // One `/NM` each: the strikeout is grouped with the caret, and XFDF names that link by + // the caret's `/NM` (#2666). let (strike_meta, caret_meta) = (cx.meta(), cx.meta()); pdfcraft_annot::add_text_replacement( doc, @@ -582,6 +693,7 @@ pub(crate) fn run_edit(doc: &mut pdfcraft_cos::Document, edit: &Edit, cx: &mut E Edit::LockAnnotation { page, index, locked } => pdfcraft_annot::set_locked(doc, *page, *index, *locked)?, Edit::MoveAnnotation { page, index, dx, dy } => pdfcraft_annot::move_annotation(doc, *page, *index, *dx, *dy, &cx.meta())?, Edit::ReshapeAnnotation { page, index, points } => pdfcraft_annot::set_points(doc, *page, *index, points, &cx.meta())?, + Edit::SetAnnotationPopup { page, index, rect, open } => pdfcraft_annot::set_popup(doc, *page, *index, *rect, *open, &cx.meta())?, Edit::ResizeAnnotation { page, index, rect } => pdfcraft_annot::set_rect(doc, *page, *index, *rect, &cx.meta())?, Edit::StyleAnnotation { page, index, color, opacity, width, endings } => { pdfcraft_annot::set_style( @@ -605,17 +717,12 @@ pub(crate) fn run_edit(doc: &mut pdfcraft_cos::Document, edit: &Edit, cx: &mut E Some(js) => pdfcraft_forms::set_value_with(doc, name, value, js), None => pdfcraft_forms::set_value(doc, name, value), }; - match set(doc, value) { - Ok(()) => {} - // A date the field's format doesn't read may be typed by another of an XFA date - // field's patterns: it is kept in the first. Only a format refusal is tried - // again: it comes before any of the field's scripts run, so they run once. - Err(refused @ pdfcraft_forms::FormError::Format(_)) => match xfa::typed_date(doc, cx.xfa.as_deref(), name, value) { - Some(typed) => set(doc, &typed)?, - None => return Err(refused.into()), - }, - Err(refused) => return Err(refused.into()), - } + // A date typed by another of an XFA date field's patterns is kept in the first, before + // the field's format reads it more loosely (`18OCT16` in a `YYYYMMDD|DDMMMYY` field is + // the 18th of October 2016, not a day of 2018). + let laid_out = pdfcraft_xfa::xfa_shape(doc).is_some_and(|s| s.laid_out); + let typed = if laid_out { xfa::typed_date(doc, cx.xfa.as_deref(), name, value) } else { None }; + set(doc, typed.as_ref().unwrap_or(value))?; } Edit::ApplyScriptChanges { changes } => match cx.js.as_mut() { Some(js) => pdfcraft_forms::apply_script_changes(doc, changes, js)?, @@ -623,12 +730,26 @@ pub(crate) fn run_edit(doc: &mut pdfcraft_cos::Document, edit: &Edit, cx: &mut E }, Edit::XfaEvent { som, activity } => { let tpl = cx.xfa.clone().ok_or_else(|| EditError::Invalid("this is not a laid-out XFA form, or JavaScript is off".into()))?; - let mut run = - xfa::XfaRun { page_count: cx.pages, out: &mut cx.xfa_out, datasets: cx.xfa_datasets, ran_away: false, time_scale: cx.xfa_time_scale }; - let ran = xfa::on_event(doc, &tpl, som, activity, &mut run); + let highlight = cx.highlight(); + let mut run = xfa::XfaRun { + page_count: cx.pages, + out: &mut cx.xfa_out, + datasets: cx.xfa_datasets, + ran_away: false, + highlight, + time_scale: cx.xfa_time_scale, + }; + let ran = if matches!(activity.as_str(), "enter" | "exit") { + xfa::on_focus(doc, &tpl, som, activity, &mut run).map(|changes| changes.is_empty()) + } else { + xfa::on_event(doc, &tpl, som, activity, &mut run).map(|()| false) + }; cx.xfa_datasets = run.datasets; cx.xfa_ran_away |= run.ran_away; - ran.map_err(EditError::Invalid)?; + // The next edit of a batch reads the highlight this one's scripts set. + let on = run.highlight; + cx.set_highlight(on); + cx.xfa_unchanged = ran.map_err(EditError::Invalid)?; } Edit::SetFieldImage { name, image } => { let (img, _) = embed_image(doc, name, image)?; @@ -729,6 +850,8 @@ pub(crate) fn run_edit(doc: &mut pdfcraft_cos::Document, edit: &Edit, cx: &mut E Edit::EnhanceScan { images } => crate::scan_clean::write_back(doc, images)?, Edit::EditPageImage { page, index, change } => run_edit_page_image(doc, *page, *index, change)?, Edit::MoveObjects { page, objects, offset } => pdfcraft_edit::move_objects(doc, *page, objects, *offset)?, + Edit::DeleteObjects { page, objects } => crate::object_clip::delete(doc, *page, objects, |doc, edit| run_edit(doc, edit, cx))?, + Edit::PasteObjects { page, clip, offset } => crate::object_clip::paste(doc, *page, clip, *offset)?, Edit::EditTextBlock { page, block, text, style } => { pdfcraft_edit::rewrite_block(doc, *page, *block, Some(text), style)?; } @@ -751,6 +874,12 @@ pub(crate) fn run_edit(doc: &mut pdfcraft_cos::Document, edit: &Edit, cx: &mut E Edit::MarkTagArtifact { object } => { a11y::mark_tag_artifact(doc, *object).map_err(|e| EditError::Accessibility(e.to_string()))?; } + Edit::LayoutTableToBlocks { object } => { + a11y::layout_table_to_blocks(doc, *object).map_err(|e| EditError::Accessibility(e.to_string()))?; + } + Edit::AddPrinterMarks { pages, settings, replace } => { + pdfcraft_edit::add_printer_marks(doc, pages, settings, *replace)?; + } Edit::AddHeaderFooter { pages, settings, replace } => { let date = cx.today; pdfcraft_edit::add_header_footer(doc, pages, settings, *replace, &pdfcraft_edit::Context { date })?; @@ -825,7 +954,10 @@ pub(crate) fn run_edit(doc: &mut pdfcraft_cos::Document, edit: &Edit, cx: &mut E pdfcraft_edit::add_content(doc, *page, &content)?; } Edit::ApplyRedactions { pages } => { - cx.redaction_overlays = pdfcraft_redact::apply(doc, pages.as_deref())?.overlays; + let report = pdfcraft_redact::apply(doc, pages.as_deref())?; + cx.redaction_overlays = report.overlays; + cx.redaction_elsewhere = Some(report.still_holding); + cx.redaction_identification = report.identification; } Edit::ClearRedactions => { pdfcraft_redact::clear_marks(doc, None)?; @@ -870,7 +1002,10 @@ pub(crate) fn run_edit(doc: &mut pdfcraft_cos::Document, edit: &Edit, cx: &mut E // the data importer's error. let mut data = doc.clone(); match pdfcraft_xfdf::import(&mut data, bytes) { - Ok(_) => *doc = data, + Ok(report) => { + note_skipped(&mut data, &report); + *doc = data; + } Err(e) if bytes[..bytes.len().min(1024)].windows(5).any(|w| w == b"%PDF-") => match comment_source(bytes) { Ok(src) => import_pdf_comments(doc, &src)?, Err(SourceProblem::Unreadable(_)) => return Err(e.into()), @@ -939,6 +1074,11 @@ pub(crate) fn run_edit(doc: &mut pdfcraft_cos::Document, edit: &Edit, cx: &mut E doc.remove_encryption(); } Edit::Batch { edits, .. } => { + // A form-script event runs on its own: in a batch, its scripts and those the batch's + // value edits run afterwards (change, exit) would interleave, an exit running twice. + if edits.iter().any(|e| matches!(e, Edit::XfaEvent { .. })) { + return Err(EditError::Invalid("a form script event can't be part of a batch of edits".into())); + } for e in edits { run_edit(doc, e, cx)?; } diff --git a/crates/engine/src/history.rs b/crates/engine/src/history.rs index 4997f64efe..4259e6c721 100644 --- a/crates/engine/src/history.rs +++ b/crates/engine/src/history.rs @@ -47,8 +47,9 @@ pub(crate) fn uses_scripts(edit: &Edit) -> bool { pub(crate) fn scope_of(edit: &Edit) -> Scope { match edit { Edit::SetInfo { .. } => Scope::Metadata, + Edit::AddPrinterMarks { .. } => Scope::Full, // The Attachments list is read from the whole document. - Edit::AddAttachment { .. } | Edit::DeleteAttachment { .. } => Scope::Full, + Edit::AddAttachment { .. } | Edit::DeleteAttachment { .. } | Edit::EditAttachmentDescription { .. } => Scope::Full, // A file attachment also changes the Attachments list. Edit::AddAnnotation(a) if matches!(a.shape, Shape::Attachment { .. }) => Scope::Full, Edit::AddMeasurement(_) @@ -66,6 +67,7 @@ pub(crate) fn scope_of(edit: &Edit) -> Scope { | Edit::EraseInk { .. } | Edit::MoveAnnotation { .. } | Edit::ReshapeAnnotation { .. } + | Edit::SetAnnotationPopup { .. } | Edit::ResizeAnnotation { .. } | Edit::StyleAnnotation { .. } | Edit::FillAnnotation { .. } @@ -79,6 +81,7 @@ pub(crate) fn scope_of(edit: &Edit) -> Scope { // Page content, resources and added content/marks only: the page tree, annotations, // fields, links and document metadata are untouched (see `refresh_content`). Edit::MoveObjects { .. } + | Edit::DeleteObjects { .. } | Edit::EditTextLine { .. } | Edit::EditTextBlock { .. } | Edit::PrepareText { .. } @@ -90,10 +93,17 @@ pub(crate) fn scope_of(edit: &Edit) -> Scope { | Edit::DeleteContent { .. } | Edit::ReplaceImage { .. } | Edit::PasteContent { .. } + | Edit::PasteObjects { .. } | Edit::AddHeaderFooter { .. } | Edit::AddWatermark { .. } | Edit::AddBackground { .. } - | Edit::RemoveMarks { .. } => Scope::Content, + | Edit::RemoveMarks { .. } + | Edit::ConvertColours { .. } + | Edit::FixHairlines { .. } + // Recognized text is an invisible text layer stamped on its page, with its font in the + // page's resources (#6310): a full re-inspection made applying one word cost seconds on + // a large file. + | Edit::AddOcrText { .. } => Scope::Content, Edit::Batch { edits, .. } => { let mut scopes = edits.iter().map(scope_of); let first = scopes.next().unwrap_or(Scope::Full); diff --git a/crates/engine/src/js.rs b/crates/engine/src/js.rs index f581504204..31f1aed19e 100644 --- a/crates/engine/src/js.rs +++ b/crates/engine/src/js.rs @@ -11,6 +11,15 @@ pub use pdfcraft_js::{Limits, Message, MessageButtons, MessageIcon, Outcome, Pan use crate::{DocId, Edit, EditError, Session}; +/// A field gaining or losing focus ([`Session::field_focus`]). +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum FieldFocus { + /// The field gained focus: its `enter` scripts run. + Enter, + /// The field lost focus: its `exit` scripts run. + Exit, +} + /// What scripts produced for the viewer since it last looked. #[derive(Clone, Debug, Default, PartialEq)] pub struct JsOutput { @@ -26,29 +35,70 @@ pub struct JsOutput { pub doc_errors: Vec, } +/// Console lines one output keeps, from every script an edit runs together (a form's +/// calculations, say): each run's own cap ([`pdfcraft_js::xfa::MAX_CONSOLE`]) would otherwise +/// be multiplied by the number of fields a document declares. +pub const MAX_CONSOLE_LINES: usize = 10_000; +/// Bytes of console text one output keeps, for the same reason. +pub const MAX_CONSOLE_BYTES: usize = 1 << 20; +/// Alerts one output keeps. +pub const MAX_ALERTS: usize = 1_000; + impl JsOutput { /// Whether scripts produced nothing: no alerts, console lines, requests or errors. pub fn is_empty(&self) -> bool { self.alerts.is_empty() && self.console.is_empty() && self.requests.is_empty() && self.errors.is_empty() } + /// Adds `lines` to the console within [`MAX_CONSOLE_LINES`] and [`MAX_CONSOLE_BYTES`]; what + /// is past them is left out, which one closing line says. + fn say(&mut self, lines: impl IntoIterator) { + let note = format!("more console lines were left out: the scripts printed more than {MAX_CONSOLE_LINES} lines or {MAX_CONSOLE_BYTES} bytes"); + // Once the notice is written the console is closed: nothing may follow it, and a later + // rejection must not add a second copy. + if self.console.last() == Some(¬e) { + return; + } + let mut bytes: usize = self.console.iter().map(String::len).sum(); + let mut dropped = false; + for l in lines { + if self.console.len() >= MAX_CONSOLE_LINES || bytes.saturating_add(l.len()) > MAX_CONSOLE_BYTES { + dropped = true; + break; + } + bytes += l.len(); + self.console.push(l); + } + if dropped && !self.console.iter().any(|l| l == ¬e) { + self.console.push(note); + } + } + + /// Adds `alerts` within [`MAX_ALERTS`]. + fn alert(&mut self, alerts: impl IntoIterator) { + let room = MAX_ALERTS.saturating_sub(self.alerts.len()); + self.alerts.extend(alerts.into_iter().take(room)); + } + /// Adds what a field's own script produced: its alerts, console lines, errors and requests, /// except a request to show a side panel or open Save As, which is noted on the console /// and dropped. fn absorb(&mut self, o: &Outcome) { - self.alerts.extend(o.alerts.iter().cloned()); - self.console.extend(o.console.iter().cloned()); + self.alert(o.alerts.iter().cloned()); + self.say(o.console.iter().cloned()); // A field's own scripts (keystroke, format, validate, calculate) run unasked: only a // button's may show or hide a side panel, or open Save As. + // The warnings go through `say` like any other console line, so they count against the + // line and byte caps instead of growing the console past them. + let mut notes = Vec::new(); for r in &o.requests { match r { - Request::ShowPanel(_) => self.console.push("A field script asked to show a side panel; only a button may show a side panel".into()), - Request::SaveAs => { - self.console.push("A field script asked to save the document as a new file; only a button may open Save As".into()) - } + Request::ShowPanel(_) => notes.push("A field script asked to show a side panel; only a button may show a side panel".to_string()), + Request::SaveAs => notes.push("A field script asked to save the document as a new file; only a button may open Save As".to_string()), r => self.requests.push(r.clone()), } } + self.say(notes); self.errors.extend(o.error.iter().cloned()); // The document scripts run before every field script, so a failing one would repeat // once per calculation; it is reported once. @@ -61,8 +111,8 @@ impl JsOutput { } pub(crate) fn append(&mut self, other: JsOutput) { - self.alerts.extend(other.alerts); - self.console.extend(other.console); + self.alert(other.alerts); + self.say(other.console); self.requests.extend(other.requests); // As in `absorb`: a document script's error, already reported, is not repeated; a field // script's errors are each their own, even one worded like a document script's. @@ -78,6 +128,49 @@ impl JsOutput { self.errors.push(e); } } + + /// Takes out the alerts, console lines, requests and errors. The record of the document + /// scripts' errors stays, so they are still reported once. + fn set_aside(&mut self) -> JsOutput { + JsOutput { + alerts: std::mem::take(&mut self.alerts), + console: std::mem::take(&mut self.console), + requests: std::mem::take(&mut self.requests), + errors: std::mem::take(&mut self.errors), + doc_errors: Vec::new(), + } + } + + /// Puts back what [`JsOutput::set_aside`] took out and returns what was added since. + fn put_back(&mut self, waiting: JsOutput) -> JsOutput { + let own = self.set_aside(); + self.alerts = waiting.alerts; + self.console = waiting.console; + self.requests = waiting.requests; + self.errors = waiting.errors; + own + } +} + +impl Session { + /// Runs `f` and returns its value with the script output `f` added to document `id`. The + /// output limits count that output alone, and the output that was waiting stays for + /// [`Session::take_js_output`]. When `f` fails, its output joins the waiting output. + pub fn with_own_js_output(&mut self, id: DocId, f: impl FnOnce(&mut Session) -> Result) -> Result<(T, JsOutput), E> { + let waiting = self.docs.iter_mut().find(|d| d.id == id).map(|d| d.js_output.set_aside()); + let result = f(self); + let mut doc = self.docs.iter_mut().find(|d| d.id == id); + let own = doc.as_mut().zip(waiting).map(|(d, w)| d.js_output.put_back(w)).unwrap_or_default(); + match result { + Ok(value) => Ok((value, own)), + Err(e) => { + if let Some(d) = doc { + d.js_output.append(own); + } + Err(e) + } + } + } } fn kind(k: FieldKind) -> pdfcraft_js::FieldType { @@ -118,6 +211,7 @@ fn field_state_with_display(f: &Field, display: i32) -> pdfcraft_js::FieldState s.readonly = f.read_only(); s.required = f.has(pdfcraft_forms::flags::REQUIRED); s.display = display; + s.rect = f.widgets.first().map(|w| w.rect); s.char_limit = f.max_len; s.options = match f.kind { FieldKind::CheckBox | FieldKind::Radio => f.widgets.iter().filter_map(|w| w.on_state.clone()).map(|o| (o.clone(), o)).collect(), @@ -147,8 +241,9 @@ fn changes(before: &[pdfcraft_js::FieldState], after: &[pdfcraft_js::FieldState] read_only: (a.readonly != b.readonly).then_some(a.readonly), required: (a.required != b.required).then_some(a.required), display: (a.display != b.display).then_some(a.display), + rect: (a.rect != b.rect).then_some(a.rect).flatten(), }; - (c.value.is_some() || c.read_only.is_some() || c.required.is_some() || c.display.is_some()).then_some(c) + (c.value.is_some() || c.read_only.is_some() || c.required.is_some() || c.display.is_some() || c.rect.is_some()).then_some(c) }) .collect() } @@ -169,11 +264,18 @@ pub struct JsRunner { impl JsRunner { /// A runner for `cos`: what scripts read about the document (pages, info, whether it is - /// an XFA form) and its document-level scripts, read once. - pub fn new(cos: &pdfcraft_cos::Document, file_name: &str) -> JsRunner { + /// an XFA form) and its document-level scripts, read once. `highlight`: whether the viewer + /// highlights form fields (`app.runtimeHighlight`). + pub fn new(cos: &pdfcraft_cos::Document, file_name: &str, highlight: bool) -> JsRunner { let xfa = pdfcraft_xfa::xfa_shape(cos).is_some(); - let doc = - pdfcraft_js::DocInfo { file_name: file_name.to_string(), num_pages: pdfcraft_model::pages(cos).len(), page: 0, info: info(cos), xfa }; + let doc = pdfcraft_js::DocInfo { + file_name: file_name.to_string(), + num_pages: pdfcraft_model::pages(cos).len(), + page: 0, + info: info(cos), + highlight, + xfa, + }; // Once XFA owns the form, the PDF container's scripts belong to its fallback view. // Keep them in the file and in Document JavaScripts, but do not initialize an // AcroForm context with them: a viewer-compatibility alert would be misleading. @@ -184,6 +286,14 @@ impl JsRunner { } } +/// The field highlight the last of `requests` that sets it asks for (`app.runtimeHighlight`). +fn last_highlight(requests: &[Request]) -> Option { + requests.iter().rev().find_map(|r| match r { + Request::Highlight(on) => Some(*on), + _ => None, + }) +} + /// The Info dictionary as `this.info` keys (lower-case). fn info(cos: &pdfcraft_cos::Document) -> Vec<(String, String)> { let Some(d) = cos.trailer().get(b"Info").map(|i| cos.resolve(i)).and_then(|i| i.as_dict().cloned()) else { return Vec::new() }; @@ -203,6 +313,7 @@ impl Scripts for JsRunner { self.run_logged(event, script, target, value, fields, true).unwrap_or_else(|| ScriptResult { rc: true, value: value.to_string(), + value_set: false, changes: Vec::new(), message: None, }) @@ -233,12 +344,17 @@ impl JsRunner { if log { self.output.absorb(&o); } + // The edit's later scripts read the field highlight this one set. + if let Some(on) = last_highlight(&o.requests) { + self.doc.highlight = on; + } if o.error.is_some() { return None; } Some(ScriptResult { rc: o.rc, value: o.value.clone(), + value_set: o.value_set, changes: changes(&states, &o.changed), message: (!o.rc).then(|| o.alerts.last().map(|m| m.text.clone())).flatten(), }) @@ -267,21 +383,124 @@ impl crate::Document { } } +/// When a signature field's XFA signing scripts run ([`Session::run_sign_event`]). +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum SignEvent { + /// Just before the field is signed (`preSign`): what it changes is signed too. + Before, + /// Once the field is signed (`postSign`). + After, +} + +impl SignEvent { + /// The template's name for the event. + pub fn activity(self) -> &'static str { + match self { + SignEvent::Before => "preSign", + SignEvent::After => "postSign", + } + } +} + impl Session { /// Preferences ▸ JavaScript ▸ Enable Acrobat JavaScript (on by default). pub fn set_javascript(&mut self, on: bool) { self.js_off = !on; } + /// Whether JavaScript is on (Preferences ▸ JavaScript ▸ Enable Acrobat JavaScript). pub fn javascript(&self) -> bool { !self.js_off } + /// Whether the viewer highlights form fields (Highlight fields), which forms' scripts read as + /// `app.runtimeHighlight`. A script that changes it asks for [`Request::Highlight`]; the + /// viewer then calls this with the new setting. + pub fn set_field_highlight(&mut self, on: bool) { + self.field_highlight = on; + } + /// What scripts printed or asked for since the last call (alerts, console, print, …). pub fn take_js_output(&mut self, id: DocId) -> JsOutput { self.docs.iter_mut().find(|d| d.id == id).map(|d| std::mem::take(&mut d.js_output)).unwrap_or_default() } + /// The last field-highlight change (`app.runtimeHighlight`) scripts asked for in document + /// `id`, taken out of its waiting output; the rest stays for [`Session::take_js_output`]. + pub fn take_js_highlight(&mut self, id: DocId) -> Option { + let doc = self.docs.iter_mut().find(|d| d.id == id)?; + let mut last = None; + doc.js_output.requests.retain(|r| match r { + Request::Highlight(on) => { + last = Some(*on); + false + } + _ => true, + }); + last + } + + /// Widget `widget` of field `name` in document `id` gained or lost focus: on a laid-out XFA form, + /// run the field's `enter` or `exit` scripts, as one undoable edit when they change something + /// (and none when they don't). An exit that the field's new value already ran (the value's + /// change runs it) isn't run a second time. Returns whether the document changed; what the + /// scripts showed or asked for is in [`Session::take_js_output`]. + pub fn field_focus(&mut self, id: DocId, name: &str, widget: usize, focus: FieldFocus) -> Result { + if self.js_off { + return Ok(false); + } + let doc = self.doc_mut(id)?; + let Some(tpl) = doc.xfa_template.clone().filter(|t| pdfcraft_xfa::has_focus_scripts(t)) else { return Ok(false) }; + let Some(cos) = doc.editor.as_ref().map(|e| &e.cos) else { return Ok(false) }; + let Some(som) = doc.form.iter().find(|f| f.name == name).and_then(|f| crate::xfa::widget_som(cos, f, widget)) else { return Ok(false) }; + let activity = match focus { + FieldFocus::Enter => "enter", + FieldFocus::Exit => "exit", + }; + if !crate::xfa::has_event(cos, &tpl, &som, activity) { + doc.xfa_exited.remove(&som); + return Ok(false); + } + // Gaining focus starts afresh; losing it runs the exit unless the change just did. + if doc.xfa_exited.remove(&som) && focus == FieldFocus::Exit { + return Ok(false); + } + let before = doc.generation; + self.apply(id, Edit::XfaEvent { som, activity: activity.into() })?; + Ok(self.get(id).is_some_and(|d| d.generation != before)) + } + + /// Run signature field `field`'s XFA signing scripts for `event` in document `id`: `preSign` + /// before it is signed, `postSign` after, as one undoable step, as a form's signing events run + /// (DA 31's dates the signature and locks the date). `Ok(false)`, with nothing changed, when + /// there are none: not a laid-out XFA form, JavaScript off, no such field, or no such script. + /// What the scripts said or asked for is in [`Session::take_js_output`]. + pub fn run_sign_event(&mut self, id: DocId, field: &str, event: SignEvent) -> Result { + if self.get(id).is_none() { + return Err(EditError::NoDocument); + } + let Some(som) = self.sign_event_som(id, field, event) else { return Ok(false) }; + self.apply(id, Edit::XfaEvent { som, activity: event.activity().into() })?; + Ok(true) + } + + /// Whether signature field `field` of document `id` has XFA signing scripts for `event` that + /// [`Session::run_sign_event`] would run, without running them. + pub fn sign_event_scripted(&self, id: DocId, field: &str, event: SignEvent) -> bool { + self.sign_event_som(id, field, event).is_some() + } + + /// The SOM path of signature field `field` when it has scripts for `event` to run. + fn sign_event_som(&self, id: DocId, field: &str, event: SignEvent) -> Option { + if self.js_off { + return None; + } + let doc = self.get(id)?; + let (tpl, editor) = (doc.xfa_template.as_ref()?, doc.editor.as_ref()?); + let f = doc.form.iter().find(|f| f.name == field)?; + crate::xfa::scripted_som(&editor.cos, tpl, f, event.activity()) + } + /// Run `script` in document `id`, for a button (`target`: the field, event "Mouse Up") or /// the console (no target). Field changes and `resetForm` are applied as one undoable step; /// everything else the script asked for is returned. @@ -297,8 +516,8 @@ impl Session { && let Some(field) = doc.form.iter().find(|f| f.name == t) && let Some(som) = crate::xfa::clickable_som(&cos, &tpl, field) { - self.apply(id, Edit::XfaEvent { som, activity: "click".into() })?; - let out = self.take_js_output(id); + // The click returns only the output of its own edit; output from opening or from earlier edits stays. + let ((), out) = self.with_own_js_output(id, |s| s.apply(id, Edit::XfaEvent { som, activity: "click".into() }))?; return Ok(Outcome { rc: true, alerts: out.alerts, @@ -308,7 +527,7 @@ impl Session { ..Default::default() }); } - let runner = JsRunner::new(&cos, &doc.name); + let runner = JsRunner::new(&cos, &doc.name, self.field_highlight); let fields = pdfcraft_forms::fields(&cos); let states: Vec<_> = fields.iter().map(|f| field_state(&cos, f)).collect(); let event = match target { @@ -326,15 +545,17 @@ impl Session { }) .collect(); o.requests.retain(|r| !matches!(r, Request::Reset(_))); + // The calculations the script's changes set off read the field highlight it asked for. + if let Some(on) = last_highlight(&o.requests) { + self.field_highlight = on; + } let mut edits = Vec::new(); if !changes.is_empty() { edits.push(Edit::ApplyScriptChanges { changes }); } - for names in resets { - let all: Vec = fields.iter().map(|f| f.name.clone()).collect(); - let names = (!names.is_empty()) - .then(|| all.iter().filter(|n| names.iter().any(|x| *n == x || n.starts_with(&format!("{x}.")))).cloned().collect()); - edits.push(Edit::ResetForm { names }); + if !resets.is_empty() { + let all: Vec<&str> = fields.iter().map(|f| f.name.as_str()).collect(); + edits.push(Edit::ResetForm { names: reset_targets(&all, &resets) }); } if !edits.is_empty() { self.apply(id, Edit::Batch { label: "Run JavaScript".into(), edits })?; @@ -343,6 +564,21 @@ impl Session { } } +/// The fields every `resetForm` call of one script asked for, as one reset: a reset is to the +/// defaults, so resetting the union once does what each call in turn would, and recalculates +/// once. `None` is the whole form (a call with no names). A field is named by a selector that +/// is its name or the name of a field above it, looked up in a set of the selectors, so the +/// work is the fields times their depth, not the fields times the selectors, which a script +/// controls. +fn reset_targets(fields: &[&str], requests: &[Vec]) -> Option> { + if requests.iter().any(Vec::is_empty) { + return None; + } + let wanted: std::collections::HashSet<&str> = requests.iter().flatten().map(String::as_str).collect(); + let matched = |name: &str| name.match_indices('.').map(|(i, _)| &name[..i]).chain(std::iter::once(name)).any(|p| wanted.contains(p)); + Some(fields.iter().filter(|n| matched(n)).map(|n| n.to_string()).collect()) +} + /// A page's words for form-field detection, in user space (underscore runs split from the /// text around them). fn detection_words(text: &pdfcraft_render::PageText, info: &pdfcraft_render::PageInfo) -> Vec { @@ -492,6 +728,97 @@ impl Session { mod review_tests { use super::*; + /// Many `resetForm` calls with many selectors over a big form: one reset of the union, found + /// by looking each field's name and parents up, not by comparing every field with every + /// selector. + #[test] + fn reset_requests_are_coalesced_and_matched_through_an_index() { + let fields: Vec = (0..20_000).map(|i| format!("group{}.row{i}.cell", i % 100)).collect(); + let names: Vec<&str> = fields.iter().map(String::as_str).collect(); + let requests: Vec> = (0..1_000).map(|k| (0..20).map(|j| format!("nothing{}", k * 20 + j)).collect()).collect(); + let started = std::time::Instant::now(); + assert_eq!(reset_targets(&names, &requests), Some(Vec::new()), "no selector names a field"); + let mut requests = requests; + requests.push(vec!["group7".into(), "group7.row7.cell".into(), "group8.row8".into()]); + let hit = reset_targets(&names, &requests).unwrap(); + assert_eq!(hit.len(), 200 + 1, "group7 has 200 fields, one more by its exact name"); + assert!(hit.iter().all(|n| n.starts_with("group7.") || n == "group8.row8.cell")); + assert!(started.elapsed() < std::time::Duration::from_secs(5), "{:?}", started.elapsed()); + requests.push(Vec::new()); + assert_eq!(reset_targets(&names, &requests), None, "a call without names resets the whole form"); + } + + /// A click returns its own alert when the output waiting in the document is full, and the + /// waiting output stays. + #[test] + fn a_click_returns_its_alert_when_the_waiting_output_is_full() { + let mut s = Session::new(); + let pdf = pdfcraft_xfa::fixtures::shell(&pdfcraft_xfa::fixtures::scripted_template()); + let id = s.open("xfa.pdf", None, std::sync::Arc::new(pdf), None).unwrap(); + s.take_js_output(id); + s.docs.iter_mut().find(|d| d.id == id).unwrap().js_output.alert(vec![Message::plain("waiting"); MAX_ALERTS]); + let o = s.run_javascript(id, "", Some("hello")).unwrap(); + assert_eq!(o.alerts.iter().map(|a| a.text.as_str()).collect::>(), ["Hello 2"]); + let out = s.take_js_output(id); + assert_eq!(out.alerts, vec![Message::plain("waiting"); MAX_ALERTS]); + } + + /// With the console full and closed, an operation still gets its own lines. Output of an + /// operation that fails joins the waiting output. + #[test] + fn output_limits_count_only_the_operations_own_output() { + let mut s = Session::new(); + let pdf = pdfcraft_xfa::fixtures::shell(&pdfcraft_xfa::fixtures::scripted_template()); + let id = s.open("xfa.pdf", None, std::sync::Arc::new(pdf), None).unwrap(); + s.take_js_output(id); + fn output(s: &mut Session, id: DocId) -> &mut JsOutput { + &mut s.docs.iter_mut().find(|d| d.id == id).unwrap().js_output + } + output(&mut s, id).say(vec!["waiting".to_string(); MAX_CONSOLE_LINES + 1]); + let waiting = output(&mut s, id).console.clone(); + assert!(waiting.last().unwrap().starts_with("more console lines were left out")); + let ((), own) = s + .with_own_js_output::<_, ()>(id, |s| { + output(s, id).say(["mine".to_string()]); + output(s, id).errors.push("own error".into()); + Ok(()) + }) + .unwrap(); + assert_eq!(own.console, ["mine"]); + assert_eq!(own.errors, ["own error"]); + assert_eq!(output(&mut s, id).console, waiting); + assert!(output(&mut s, id).errors.is_empty()); + let failed = s.with_own_js_output::<(), _>(id, |s| { + output(s, id).errors.push("failed edit".into()); + Err("no") + }); + assert_eq!(failed.unwrap_err(), "no"); + let out = s.take_js_output(id); + assert_eq!(out.console, waiting); + assert_eq!(out.errors, ["failed edit"]); + } + + /// A form of many calculated fields, each printing its own cap of console lines, keeps a + /// bounded console: past the budget, lines are left out and one closing line says so. + #[test] + fn console_output_of_one_edit_is_bounded() { + let mut out = JsOutput::default(); + let line = "x".repeat(4_096); + let run = Outcome { console: vec![line; 1_000], alerts: vec![Message::plain("hi"); 100], ..Default::default() }; + for _ in 0..1_000 { + out.absorb(&run); + } + let bytes: usize = out.console.iter().map(String::len).sum(); + assert!(bytes <= MAX_CONSOLE_BYTES + 256, "{bytes} bytes kept"); + assert!(out.console.len() <= MAX_CONSOLE_LINES + 1, "{} lines kept", out.console.len()); + assert!(out.console.last().unwrap().starts_with("more console lines were left out")); + assert_eq!(out.console.iter().filter(|l| l.starts_with("more console lines")).count(), 1, "said once"); + assert_eq!(out.alerts.len(), MAX_ALERTS); + let mut merged = JsOutput::default(); + merged.append(out); + assert!(merged.console.len() <= MAX_CONSOLE_LINES + 1); + } + /// A long run of underscores (a form's blank) is read in one pass: each glyph is compared with /// the word so far without reading the word again. #[test] @@ -505,7 +832,8 @@ mod review_tests { }; let mut glyphs: Vec<_> = (0..n).map(|i| glyph(i, "_")).collect(); glyphs.push(glyph(n, "x")); - let text = pdfcraft_render::PageText { line_of: vec![0; glyphs.len()], space_before: vec![false; glyphs.len()], glyphs }; + // Built by hand, not read from a page the renderer cut short, so the text is complete. + let text = pdfcraft_render::PageText { line_of: vec![0; glyphs.len()], space_before: vec![false; glyphs.len()], glyphs, truncated: false }; UNDERSCORE_CHARS_READ.with(|c| c.set(0)); let words = words_with(&text, |x, y| [x, y]); assert_eq!(words.iter().map(|(w, _)| w.len()).collect::>(), [n, 1]); diff --git a/crates/engine/src/layers.rs b/crates/engine/src/layers.rs index f75967097c..b2fab68acb 100644 --- a/crates/engine/src/layers.rs +++ b/crates/engine/src/layers.rs @@ -6,10 +6,102 @@ use pdfcraft_render::{Layer, LayerOp, RenderPool}; use crate::{Document, display_config, render_threads}; +/// Most usage application dictionaries (`/AS` entries) that [`print_layer_states`] reads, and +/// most category names and groups it reads in all of them together (entries can share one +/// array, so the count is over every entry, not each). +const MAX_AUTO_STATES: usize = 1024; +const MAX_AUTO_ITEMS: usize = 65_536; + +/// The usage dictionary entry and its state key that a `/Category` name of a usage application +/// dictionary reads when printing (ISO 32000-2 Table 101): Print, View and Export each name a +/// state of the group's own. Zoom, User and Language depend on the magnification, the user and +/// the system language, which printing doesn't fix, so they give no recommendation here. +fn print_category(name: &[u8]) -> Option<(&'static [u8], &'static [u8])> { + match name { + b"Print" => Some((b"Print", b"PrintState")), + b"View" => Some((b"View", b"ViewState")), + b"Export" => Some((b"Export", b"ExportState")), + _ => None, + } +} + +/// The layers as they print: `layers` as shown, then the default configuration's auto state for +/// the Print event (ISO 32000-2 §8.11.4.4, Table 101). Each usage application dictionary +/// (`/AS` entry) whose `/Event` is `/Print` recommends, for each group in its `/OCGs` and each of +/// its categories ([`print_category`]), the state the group's usage dictionary gives (for the +/// Print category, `/Usage /Print /PrintState`). A group with recommendations is on only if all +/// of them, in all the entries naming it, are ON; other groups keep their state. `/Category` is +/// an array; a single name is read as one. `None` when no `/AS` entry is for printing. +pub(crate) fn print_layer_states(cos: &pdfcraft_cos::Document, layers: &[Layer]) -> Option> { + use pdfcraft_cos::Object; + let root = cos.trailer().get(b"Root").map(|r| cos.resolve(r))?; + let properties = root.as_dict()?.get(b"OCProperties").map(|p| cos.resolve(p))?; + let config = properties.as_dict()?.get(b"D").map(|d| cos.resolve(d))?; + let auto = config.as_dict()?.get(b"AS").map(|a| cos.resolve(a))?; + let mut printing = false; + let mut recommended: std::collections::HashMap<(u32, u16), bool> = std::collections::HashMap::new(); + let mut items_left = MAX_AUTO_ITEMS; + for entry in auto.as_array()?.iter().take(MAX_AUTO_STATES) { + let entry = cos.resolve(entry); + let Some(entry) = entry.as_dict() else { continue }; + if entry.get(b"Event").map(|e| cos.resolve(e)).as_deref().and_then(Object::as_name) != Some(b"Print") { + continue; + } + printing = true; + let categories = entry.get(b"Category").map(|c| cos.resolve(c)); + let mut read: Vec<(&[u8], &[u8])> = Vec::new(); + let mut consider = |name: Option<&[u8]>| { + if let Some(category) = name.and_then(print_category).filter(|c| !read.contains(c)) { + read.push(category); + } + }; + match categories.as_deref() { + Some(Object::Array(names)) => { + let count = names.len().min(items_left); + items_left -= count; + names.iter().take(count).for_each(|n| consider(cos.resolve(n).as_name())); + } + Some(Object::Name(n)) => consider(Some(n.as_slice())), + _ => {} + } + if read.is_empty() { + continue; + } + let Some(groups) = entry.get(b"OCGs").map(|g| cos.resolve(g)) else { continue }; + for group in groups.as_array().into_iter().flatten() { + let Some(left) = items_left.checked_sub(1) else { break }; + items_left = left; + let Some(id) = group.as_ref() else { continue }; + let ocg = cos.resolve(group); + let usage = ocg.as_dict().and_then(|g| g.get(b"Usage").map(|u| cos.resolve(u))); + for (entry_key, state_key) in &read { + // No state given (a Print category without `/PrintState`): no recommendation, so + // the group is left as it is. + let state = usage + .as_deref() + .and_then(|u| u.as_dict().and_then(|u| u.get(entry_key).map(|e| cos.resolve(e)))) + .and_then(|e| e.as_dict().and_then(|e| e.get(state_key).map(|s| cos.resolve(s)))) + .and_then(|s| match s.as_name() { + Some(b"ON") => Some(true), + Some(b"OFF") => Some(false), + _ => None, + }); + if let Some(on) = state { + let all_on = recommended.entry((id.num, id.generation)).or_insert(true); + *all_on = *all_on && on; + } + } + } + } + printing.then(|| layers.iter().map(|l| Layer { visible: recommended.get(&l.id).copied().unwrap_or(l.visible), ..l.clone() }).collect()) +} + /// Write `layers`' visibility into the default optional-content configuration of `cos` (a copy -/// made for printing): `/BaseState /ON`, `/ON` the visible layers, `/OFF` the hidden ones. Its -/// other entries (`/Order`, `/RBGroups`, `/AS`…) are kept. A document without `/OCProperties` -/// is left alone. +/// made for printing): `/BaseState /ON`, `/ON` the visible layers, `/OFF` the hidden ones. The +/// states written are final, so what would set them again goes: the auto state (`/AS`), and the +/// layers' own `/Usage`, which some readers apply when they draw (a `/ViewState /ON` turns a layer +/// that prints off back on). The configuration's other entries (`/Order`, `/RBGroups`…) are kept. +/// A document without `/OCProperties` is left alone. pub(crate) fn set_default_layer_state(cos: &mut pdfcraft_cos::Document, layers: &[Layer]) { use pdfcraft_cos::{Dict, ObjRef, Object}; let Some(root_ref) = cos.trailer().get(b"Root").and_then(Object::as_ref) else { return }; @@ -22,18 +114,35 @@ pub(crate) fn set_default_layer_state(cos: &mut pdfcraft_cos::Document, layers: config.set(b"BaseState".to_vec(), Object::name("ON")); config.set(b"ON".to_vec(), refs(true)); config.set(b"OFF".to_vec(), refs(false)); + config.remove(b"AS"); properties.set(b"D".to_vec(), Object::Dict(config)); root.set(b"OCProperties".to_vec(), Object::Dict(properties)); cos.set(root_ref, Object::Dict(root)); + for layer in layers { + let id = ObjRef::new(layer.id.0, layer.id.1); + let Some(mut group) = cos.get(id).as_dict().cloned() else { continue }; + if group.remove(b"Usage").is_some() { + cos.set(id, Object::Dict(group)); + } + } } /// Render `doc` with the layer visibility in its `info.layers`. pub(crate) fn use_layer_choices(doc: &mut Document) { let overrides: Vec<(i32, i32, bool)> = doc.info.layers.iter().map(|l| (l.id.0 as i32, l.id.1 as i32, l.visible)).collect(); doc.config.layers = Arc::new(overrides); + doc.config.print_layers = print_preview_layers(doc.editor.as_ref().map(|e| &e.cos), &doc.info.layers); doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); } +/// The layer overrides a print preview draws with (`RenderConfig::print_layers`): the layers as +/// they print ([`print_layer_states`]), so the preview shows the page as it prints. `None` when +/// the document has no auto state for printing (or no structure to read it from). +pub(crate) fn print_preview_layers(cos: Option<&pdfcraft_cos::Document>, layers: &[Layer]) -> Option>> { + let printed = print_layer_states(cos?, layers)?; + Some(Arc::new(printed.iter().map(|l| (l.id.0 as i32, l.id.1 as i32, l.visible)).collect())) +} + /// Apply a set-layer-visibility action to `layers`, one change at a time, so a toggle flips the /// state the changes before it left. Returns whether any layer ended up changed. pub(crate) fn apply_layer_state(layers: &mut [Layer], groups: &[Vec<(u32, u16)>], changes: &[(LayerOp, (u32, u16))], preserve_rb: bool) -> bool { diff --git a/crates/engine/src/lib.rs b/crates/engine/src/lib.rs index fbd1cacc1f..17f196cf6a 100644 --- a/crates/engine/src/lib.rs +++ b/crates/engine/src/lib.rs @@ -24,6 +24,7 @@ pub mod commands; mod comments; pub mod compare; mod copy_image; +pub use copy_image::{PagePicture, picture_at}; pub mod dates; mod document; mod edit; @@ -36,6 +37,7 @@ pub mod js; mod layers; pub mod links; pub mod model3d; +mod object_clip; pub mod ocr; mod open; pub mod optimizer; @@ -56,6 +58,7 @@ mod stamp_preview; #[cfg(test)] mod test_hooks; pub mod text_fonts; +pub mod text_links; mod text_preview; pub mod text_script; pub mod text_style_bookmarks; @@ -85,7 +88,7 @@ pub use pdfcraft_organize::Standard; pub use pdfcraft_organize::check_custom_key; pub use pdfcraft_organize::view::{InitialView, Layout as InitialLayout, Magnification, Navigation}; -pub use pdfcraft_cos::Algorithm; +pub use pdfcraft_cos::{Algorithm, civil_from_days}; pub use pdfcraft_create::{CONVERTIBLE, CreateError, ImageResolution, SourceKind, decode_text, source_kind}; pub use pdfcraft_edit::added::{ CHAR_SPACING_LIMIT as ADDED_CHAR_SPACING_LIMIT, paste_place as added_paste_place, paste_rect as added_paste_rect, text_rect as added_text_rect, @@ -93,8 +96,8 @@ pub use pdfcraft_edit::added::{ }; pub use pdfcraft_edit::paragraph_align; pub use pdfcraft_edit::{ - Added, AddedImage, AddedText, Align as TextAlign, Background, Content as AddedContent, Family as FontFamily, HeaderFooter, Ink as TextInk, - MarkKind, Watermark, + Added, AddedImage, AddedText, Align as TextAlign, Background, ColourTarget, ColoursConverted, Content as AddedContent, ConvertColours, + Family as FontFamily, Hairlines, HairlinesFixed, HeaderFooter, Ink as TextInk, MarkKind, PrinterMarks, Watermark, }; pub use pdfcraft_forms::{ BorderStyle, CalculatedField, CalculationIssue, CalculationOrder, CheckStyle, Field as FormField, FieldAction, FieldChange, FieldFont, @@ -108,28 +111,33 @@ pub use pdfcraft_edit::{Backdrop, BlockKey, BlockStyle, PageImage, RangeStyle, T pub use pdfcraft_measure as measure; pub use pdfcraft_xfa::Report as XfaLayout; +pub use object_clip::{ObjectClip, ObjectClipboard}; pub use pdfcraft_edit::{EditableObject, MAX_MOVE_OBJECTS, ObjectKind, ObjectTarget}; pub use pdfcraft_fonts::{MAX_SIGNATURE_CHARS, ScriptOutline, script_missing, script_outline, undrawable_message}; /// Comment geometry helpers (text-box line breaking) for frontends. pub use pdfcraft_annot::appearance as annot_text; -pub use pdfcraft_annot::links::{Highlight as LinkHighlight, LinkAction, LinkItem, LinkStyle}; +pub use pdfcraft_annot::links::{Highlight as LinkHighlight, LinkAction, LinkItem, LinkStyle, LinkZoom}; pub use pdfcraft_annot::{ - Activation3d, AppearanceCache, AttachIcon, BorderShape, FillMark, LineEnding, Markup, NOTE_SIZE, NewAnnotation, NoteIcon, OverlayFont, - OverlayLook, Props as CommentProps, ReviewState, Rgb, Shape, StampGroup, StampKind, Style, rect_quad, + Activation3d, AppearanceCache, AttachIcon, BorderShape, CALLOUT_KNEE_LINE, FillMark, LineEnding, Markup, NOTE_SIZE, NewAnnotation, NoteIcon, + OverlayFont, OverlayLook, Props as CommentProps, ReviewState, Rgb, Shape, Side as CalloutSide, StampGroup, StampKind, Style, callout_attach, + callout_knee, callout_side, rect_quad, }; pub use pdfcraft_annot::{MAX_MODEL_BYTES as MAX_3D_MODEL_BYTES, ModelFormat as Model3dFormat}; pub use pdfcraft_forms::detect; pub use pdfcraft_optimize as optimize; pub use pdfcraft_preflight as pdfa; pub use pdfcraft_print as print; +pub use pdfcraft_redact::Elsewhere as RedactionElsewhere; pub use pdfcraft_redact::codes::{CODE_SETS as REDACTION_CODE_SETS, CodeSet as RedactionCodeSet}; pub use pdfcraft_redact::patterns::{ MAX_WORDS as REDACT_MAX_WORDS, PATTERNS as REDACT_PATTERNS, Pattern as RedactPattern, find as find_pattern, word_list as redact_word_list, }; +pub use pdfcraft_redact::redacted_file_name; pub use pdfcraft_redact::sanitize::{HIDDEN, Hidden}; pub use pdfcraft_redact::user_regex::{UserRegex as RedactRegex, VALIDATORS as REDACT_VALIDATORS, Validator as RedactValidator}; +pub use pdfcraft_redact::{Claim as RedactionClaim, Identification as RedactionIdentification}; pub use pdfcraft_sign as sign; pub use pdfcraft_sign::{SignOptions, SignatureInfo, Status as SignatureStatus, StatusCaptions, TrustStore}; pub use pdfcraft_xfdf::Format as DataFormat; @@ -147,7 +155,7 @@ pub use edit::{BATCH_LABELS, EDIT_LABELS, Edit}; pub use error::{EditError, SourceProblem}; pub use fill_sign::{displayed_offset, text_box_at, typed_signature_graphic, typed_signature_shape}; pub use history::Label; -use layers::{apply_layer_state, set_default_layer_state, use_layer_choices}; +use layers::{apply_layer_state, print_layer_states, print_preview_layers, set_default_layer_state, use_layer_choices}; pub use open::{OpenContext, PreparedDocument}; pub use page_filter::{PageOrientation, PageParity, filter_pages}; pub use pictures::{MarkFile, image_dimensions}; @@ -158,7 +166,10 @@ pub use session::Session; pub use sources::{CombineSource, Combined, combine_source_check, source_page_count}; use sources::{open_source, open_source_with}; #[cfg(test)] -use test_hooks::{check_refresh_panic, panic_on_refresh, record_link_scan, record_marker_scan, reset_secondary_scan_counts, secondary_scan_counts}; +use test_hooks::{ + check_refresh_panic, panic_on_refresh, record_full_refresh, record_link_scan, record_marker_scan, reset_secondary_scan_counts, + secondary_scan_counts, take_full_refreshes, +}; use xfa_form::{ XFA_REMOVED_PREFIX, XFA_SCRIPTS_OFF, decode_data_file, image_content_type, is_xfa_data_file, note_warnings, strip_xfa, sync_xfa_state, undrawn_field_warnings, xfa_after_data_change, xfa_layout, xfa_redraw, xfa_sync_datasets, xfa_values_from_datasets, @@ -181,5 +192,8 @@ fn display_config(config: &pdfcraft_render::RenderConfig) -> pdfcraft_render::Re config.clone().on_screen() } +/// How long a custom stamp's text line may be, in characters (#1741). +pub const MAX_STAMP_LINE: usize = 200; + #[cfg(test)] mod tests; diff --git a/crates/engine/src/links.rs b/crates/engine/src/links.rs index e1d54d56ce..da3f0fc904 100644 --- a/crates/engine/src/links.rs +++ b/crates/engine/src/links.rs @@ -227,7 +227,25 @@ pub fn read_remote_document(path: &Path) -> Result, RemoteDocumentPathEr if meta.len() > MAX_REMOTE_DOCUMENT_BYTES { return Err(RemoteDocumentPathError::TooLarge); } - let file = std::fs::File::open(path).map_err(|e| RemoteDocumentPathError::Read(e.to_string()))?; + // The path can change between that check and the open (a file replaced by a FIFO): opened + // without blocking, and checked again on the handle, which is what is read. + let mut options = std::fs::OpenOptions::new(); + options.read(true); + #[cfg(unix)] + { + use std::os::unix::fs::OpenOptionsExt as _; + // O_NONBLOCK: Linux 0o4000, macOS and the BSDs 0x4. + options.custom_flags(if cfg!(target_os = "linux") { 0o4000 } else { 0x4 }); + } + let file = options.open(path).map_err(|e| RemoteDocumentPathError::Read(e.to_string()))?; + let meta = file.metadata().map_err(|e| RemoteDocumentPathError::Read(e.to_string()))?; + if !meta.is_file() { + return Err(RemoteDocumentPathError::NotAFile); + } + // the opened handle's own length: a file replaced after the path check must not size the buffer + if meta.len() > MAX_REMOTE_DOCUMENT_BYTES { + return Err(RemoteDocumentPathError::TooLarge); + } let mut bytes = Vec::with_capacity(usize::try_from(meta.len()).unwrap_or(0)); file.take(MAX_REMOTE_DOCUMENT_BYTES.saturating_add(1)).read_to_end(&mut bytes).map_err(|e| RemoteDocumentPathError::Read(e.to_string()))?; if u64::try_from(bytes.len()).map_or(true, |n| n > MAX_REMOTE_DOCUMENT_BYTES) { diff --git a/crates/engine/src/object_clip.rs b/crates/engine/src/object_clip.rs new file mode 100644 index 0000000000..3a2f0915a1 --- /dev/null +++ b/crates/engine/src/object_clip.rs @@ -0,0 +1,211 @@ +//! Edit ▸ Copy, Cut and Paste of objects selected together in Edit a PDF (#5989). +//! +//! A copy ([`ObjectClip`]) holds what the objects draw and the resources that drawing names +//! ([`pdfcraft_edit::copy_objects`]), with a snapshot of the document they came from, so it can +//! be pasted into any open document, also after the original changes or closes. Within the same +//! document, while the objects it refers to are unchanged, a paste draws them again from the same +//! fonts and images; otherwise it first copies them over ([`pdfcraft_organize::import_value`]): +//! each once, through cycles, sharing what the target already holds, and within a size limit. + +use std::sync::Arc; + +use pdfcraft_cos::Object; + +use crate::{DocId, Document, Edit, EditError, EditableObject, ObjectTarget}; + +/// The most objects a paste copies into another document. +const MAX_IMPORT_OBJECTS: usize = 20_000; +/// The most stream data (fonts, images…) a paste copies into another document. +const MAX_IMPORT_BYTES: usize = 256 << 20; +/// How far each further paste onto the same page steps down and to the right, in points, as +/// pasted added content steps ([`pdfcraft_edit::added::PASTE_STEP`]). +const STEP: f64 = pdfcraft_edit::added::PASTE_STEP; + +/// Objects copied from one page of a document. +#[derive(Clone)] +pub struct ObjectClip { + copied: pdfcraft_edit::CopiedObjects, + /// The document as it was when copied: what the copy's resources refer to. + source: pdfcraft_cos::Document, + doc: DocId, + page: usize, +} + +impl PartialEq for ObjectClip { + fn eq(&self, other: &Self) -> bool { + self.copied == other.copied + && self.doc == other.doc + && self.page == other.page + && self.source.source_identity() == other.source.source_identity() + } +} + +impl std::fmt::Debug for ObjectClip { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("ObjectClip").field("doc", &self.doc).field("page", &self.page).field("objects", &self.copied.objects.len()).finish() + } +} + +impl ObjectClip { + /// The objects as they were listed when copied ([`Document::editable_objects`]). + pub fn objects(&self) -> &[EditableObject] { + &self.copied.objects + } + + /// The text of the copied paragraphs and text boxes, one per line. + pub fn text(&self) -> String { + self.copied.text() + } + + /// The union of the objects' boxes, in the user space of the page they came from. + pub fn rect(&self) -> [f64; 4] { + self.copied.rect + } + + /// The document and page the objects were copied from. + pub fn source(&self) -> (DocId, usize) { + (self.doc, self.page) + } +} + +impl Document { + /// Edit ▸ Copy of `targets` (references from one [`Document::editable_objects`] inventory of + /// `page`). Refused when the document doesn't allow copying. Changes nothing. + pub fn copy_objects(&self, page: usize, targets: &[ObjectTarget]) -> Result { + self.check_copying()?; + let editor = self.editor.as_ref().ok_or_else(|| EditError::ReadOnly("the document can't be read".into()))?; + let copied = pdfcraft_edit::copy_objects(&editor.cos, page, targets)?; + Ok(ObjectClip { copied, source: editor.cos.clone(), doc: self.id, page }) + } + + /// Where Paste puts `clip` on `page`: the user-space offset that moves it `steps` × 18 pt down + /// and to the right as displayed, or, when that leaves nothing of it on the page's crop box, + /// the offset that centres it there. + pub fn object_paste_offset(&self, page: usize, clip: &ObjectClip, steps: u32) -> Result<[f64; 2], String> { + let step = STEP * f64::from(steps); + let offset = self.object_move_offset(page, [step, step])?; + let p = self.info.pages.get(page).ok_or("the page no longer exists")?; + let crop = p.crop.map(f64::from); + let r = clip.rect(); + let moved = [r[0] + offset[0], r[1] + offset[1], r[2] + offset[0], r[3] + offset[1]]; + if moved[0] < crop[2] && moved[2] > crop[0] && moved[1] < crop[3] && moved[3] > crop[1] { + return Ok(offset); + } + let centre = |a: f64, b: f64| (a + b) / 2.0; + Ok([centre(crop[0], crop[2]) - centre(r[0], r[2]), centre(crop[1], crop[3]) - centre(r[1], r[3])]) + } +} + +/// The objects copied in an application or a script, and how many times each page has had them +/// pasted, which sets how far the next paste there steps ([`Document::object_paste_offset`]). +#[derive(Clone, Debug)] +pub struct ObjectClipboard { + clip: Arc, + pastes: Vec<(DocId, usize, u32)>, +} + +impl ObjectClipboard { + pub fn new(clip: ObjectClip) -> Self { + ObjectClipboard { clip: Arc::new(clip), pastes: Vec::new() } + } + + pub fn clip(&self) -> &ObjectClip { + &self.clip + } + + /// The edit that pastes the copy onto `page` of `doc`: where it was, or on the page it came + /// from one step down and to the right (as Acrobat offsets a paste onto the same page), and + /// one step further for each earlier paste onto that page. + pub fn paste(&mut self, doc: &Document, page: usize) -> Result { + let at = self.pastes.iter().position(|(d, p, _)| *d == doc.id && *p == page); + let earlier = at.and_then(|i| self.pastes.get(i)).map_or(0, |(_, _, n)| *n); + let own = u32::from(self.clip.source() == (doc.id, page)); + let offset = doc.object_paste_offset(page, &self.clip, earlier.saturating_add(own))?; + match at.and_then(|i| self.pastes.get_mut(i)) { + Some(entry) => entry.2 = entry.2.saturating_add(1), + None => self.pastes.push((doc.id, page, 1)), + } + Ok(Edit::PasteObjects { page, clip: Arc::clone(&self.clip), offset }) + } +} + +impl Edit { + /// The edit that deletes `targets` (indexes into one inventory of `page`) as one undo step: + /// what Delete and Cut do to objects selected together. One object is deleted by its own + /// edit, several by [`Edit::DeleteObjects`]; none is no edit. + pub fn delete_objects(page: usize, mut targets: Vec) -> Option { + targets.sort(); + targets.dedup(); + match targets.as_slice() { + [] => None, + [t] => Some(delete_one(page, *t)), + _ => Some(Edit::DeleteObjects { page, objects: targets }), + } + } +} + +fn delete_one(page: usize, t: ObjectTarget) -> Edit { + use crate::ObjectKind; + match t.kind { + ObjectKind::Text => Edit::EditTextBlock { page, block: t.index, text: String::new(), style: crate::BlockStyle::default() }, + ObjectKind::Image => Edit::EditPageImage { page, index: t.index, change: crate::ImageEdit::Delete }, + ObjectKind::Added => Edit::DeleteContent { page, index: t.index }, + } +} + +/// [`Edit::DeleteObjects`], each object by its own edit (`run`). A deletion renumbers what follows +/// it in its own inventory, so pictures go from the last to the first, and added content, which +/// the other inventories leave out, after them. Deleting a paragraph also removes a copy drawn +/// exactly over it (fake bold, a paste onto itself), which renumbers paragraphs before it too: so +/// each paragraph is found again by its text and box, and one already gone with another is done. +pub(crate) fn delete( + doc: &mut pdfcraft_cos::Document, + page: usize, + objects: &[ObjectTarget], + mut run: impl FnMut(&mut pdfcraft_cos::Document, &Edit) -> Result<(), EditError>, +) -> Result<(), EditError> { + use crate::ObjectKind; + let mut targets = objects.to_vec(); + targets.sort_by_key(|t| (t.kind == ObjectKind::Added, std::cmp::Reverse(t.index))); + targets.dedup(); + let blocks = pdfcraft_edit::text_blocks(doc, page)?; + for t in targets { + let edit = match t.kind { + ObjectKind::Text => { + let key = blocks.get(t.index).map(crate::BlockKey::of).ok_or_else(|| { + EditError::Edit(pdfcraft_edit::EditError::Invalid("a selected paragraph no longer exists; select the objects again".into())) + })?; + let Some(block) = key.find(&pdfcraft_edit::text_blocks(doc, page)?, t.index) else { continue }; + delete_one(page, ObjectTarget { kind: t.kind, index: block }) + } + _ => delete_one(page, t), + }; + run(doc, &edit)?; + } + Ok(()) +} + +/// [`Edit::PasteObjects`]: the copy's resources as `doc` holds them, then the paste. +pub(crate) fn paste(doc: &mut pdfcraft_cos::Document, page: usize, clip: &ObjectClip, offset: [f64; 2]) -> Result<(), EditError> { + let resources = Object::Dict(clip.copied.resources.clone()); + let refused = + |e: pdfcraft_organize::OrganizeError| EditError::Edit(pdfcraft_edit::EditError::Invalid(format!("the copied objects can't be pasted: {e}"))); + let used = pdfcraft_organize::reachable(&clip.source, &resources, MAX_IMPORT_OBJECTS, MAX_IMPORT_BYTES).map_err(refused)?; + // Still the objects the copy was made from: the same document, and each of them unchanged. + let same = doc.source_identity() == clip.source.source_identity() + && used.iter().all(|r| { + let (here, there) = (doc.get(*r), clip.source.get(*r)); + Arc::ptr_eq(&here, &there) || here == there + }); + let resources = if same { resources } else { pdfcraft_organize::import_value(doc, &clip.source, &resources).map_err(refused)? }; + let Object::Dict(resources) = resources else { + return Err(EditError::Edit(pdfcraft_edit::EditError::Invalid("the copied objects' resources could not be copied".into()))); + }; + let copied = pdfcraft_edit::CopiedObjects { resources, ..clip.copied.clone() }; + pdfcraft_edit::paste_objects(doc, page, &copied, offset)?; + Ok(()) +} + +#[cfg(test)] +#[path = "object_clip_tests.rs"] +mod tests; diff --git a/crates/engine/src/object_clip_tests.rs b/crates/engine/src/object_clip_tests.rs new file mode 100644 index 0000000000..49baae69b8 --- /dev/null +++ b/crates/engine/src/object_clip_tests.rs @@ -0,0 +1,257 @@ +use super::*; +use crate::{ObjectKind, Session}; + +/// A PDF of 300 × 300 pages, each drawing its `content` with `resources` and objects `extra` +/// (numbered after the pages' content streams). +fn pdf(pages: &[(&str, &str)], extra: &[String]) -> Arc> { + let n = pages.len(); + let kids: Vec = (0..n).map(|i| format!("{} 0 R", 3 + i)).collect(); + let mut objs = vec![ + "<< /Type /Catalog /Pages 2 0 R >>".to_string(), + format!("<< /Type /Pages /Kids [{}] /Count {n} /MediaBox [0 0 300 300] >>", kids.join(" ")), + ]; + for (i, (_, resources)) in pages.iter().enumerate() { + objs.push(format!("<< /Type /Page /Parent 2 0 R /Contents {} 0 R /Resources {resources} >>", 3 + n + i)); + } + for (content, _) in pages { + objs.push(format!("<< /Length {} >>\nstream\n{content}\nendstream", content.len())); + } + objs.extend(extra.iter().cloned()); + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offs = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offs.push(out.len()); + out.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes()); + } + let x = out.len(); + out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offs { + out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{x}\n%%EOF\n", objs.len() + 1).as_bytes()); + Arc::new(out) +} + +const HELVETICA: &str = "<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>"; +const COURIER: &str = "<< /Type /Font /Subtype /Type1 /BaseFont /Courier /Encoding /WinAnsiEncoding >>"; +/// A 2 × 2 RGB picture of the twelve bytes `data` (ASCII, so the file's lengths are its bytes). +fn picture(data: &str) -> String { + assert!(data.is_ascii() && data.len() == 12); + format!( + "<< /Type /XObject /Subtype /Image /Width 2 /Height 2 /ColorSpace /DeviceRGB /BitsPerComponent 8 /Length 12 >>\nstream\n{data}\nendstream" + ) +} + +/// A one-page document whose paragraph "Copied" (blue, half opaque, Helvetica /F1 at objects 5) +/// and picture /Im0 (object 6) are copied; "Left" is not. +fn source() -> Arc> { + let content = "0 0 1 rg /GS0 gs BT /F1 24 Tf 20 200 Td (Copied) Tj ET q 60 0 0 40 20 100 cm /Im0 Do Q BT /F1 12 Tf 200 40 Td (Left) Tj ET"; + pdf( + &[(content, "<< /Font << /F1 5 0 R >> /XObject << /Im0 6 0 R >> /ExtGState << /GS0 << /ca 0.5 >> >> >>")], + &[HELVETICA.into(), picture("\u{7f}\u{0}\u{0}\u{0}\u{7f}\u{0}\u{0}\u{0}\u{7f}\u{7f}\u{7f}\u{0}")], + ) +} + +/// A one-page document that already draws with Courier as /F1 and has a picture of its own as /Im0. +fn target() -> Arc> { + pdf( + &[("BT /F1 10 Tf 250 280 Td (x) Tj ET q 10 0 0 10 280 5 cm /Im0 Do Q", "<< /Font << /F1 5 0 R >> /XObject << /Im0 6 0 R >> >>")], + &[COURIER.into(), picture("@@@@@@@@@@@@")], + ) +} + +fn copy_all_but_left(s: &Session, id: DocId) -> ObjectClip { + let doc = s.get(id).unwrap(); + let targets: Vec<_> = doc.editable_objects(0).unwrap().into_iter().filter(|o| o.text.as_deref() != Some("Left")).map(|o| o.target).collect(); + assert_eq!(targets.len(), 2); + doc.copy_objects(0, &targets).unwrap() +} + +fn render(s: &Session, id: DocId) -> pdfcraft_render::RenderedPage { + let doc = s.get(id).unwrap(); + let mut renderer = pdfcraft_render::PageRenderer::new(doc.bytes.clone(), doc.config.clone()); + renderer.render(pdfcraft_render::RenderRequest { page: 0, scale: 1.0, ..Default::default() }) +} + +/// The pixels in user-space box `r` of a 300 pt page rendered at scale 1. +fn pixels(page: &pdfcraft_render::RenderedPage, r: [f64; 4]) -> Vec { + let mut out = Vec::new(); + for y in (300.0 - r[3]) as u32..(300.0 - r[1]) as u32 { + for x in r[0] as u32..r[2] as u32 { + let i = ((y * page.width + x) * 4) as usize; + out.extend_from_slice(&page.rgba[i..i + 4]); + } + } + out +} + +fn images(s: &Session, id: DocId) -> Vec> { + s.get(id).unwrap().page_images(0).into_iter().map(|i| i.object).collect() +} + +#[test] +fn objects_copied_from_one_document_paste_into_another_and_draw_the_same() { + let mut s = Session::new().with_clock(|| 1_700_000_000); + let a = s.open("a.pdf", None, source(), None).unwrap(); + let b = s.open("b.pdf", None, target(), None).unwrap(); + let clip = copy_all_but_left(&s, a); + assert_eq!(clip.text(), "Copied"); + let mut clipboard = ObjectClipboard::new(clip); + // The source can change, or close: the copy is a snapshot. + s.apply(a, Edit::delete_objects(0, vec![ObjectTarget { kind: ObjectKind::Image, index: 0 }]).unwrap()).unwrap(); + s.close(a); + let a = s.open("a.pdf", None, source(), None).unwrap(); + + let paste = clipboard.paste(s.get(b).unwrap(), 0).unwrap(); + assert!(matches!(&paste, Edit::PasteObjects { offset, .. } if *offset == [0.0, 0.0]), "another document: where it was"); + s.apply(b, paste).unwrap(); + assert_eq!(s.get(b).unwrap().can_undo(), Some("Paste")); + let area = clipboard.clip().rect().map(|v| v.round()); + let area = [area[0] - 3.0, area[1] - 3.0, area[2] + 3.0, area[3] + 3.0]; + let (original, pasted) = (pixels(&render(&s, a), area), pixels(&render(&s, b), area)); + assert!(original.chunks(4).any(|p| p[2] > 200 && p[0] < 200), "the blue paragraph is drawn"); + assert_eq!(original, pasted, "the pasted paragraph and picture draw the same pixels"); + + // Its own /F1 and /Im0 are left alone; what the copy uses came over under other names. + let doc = s.get(b).unwrap(); + let texts: Vec<_> = doc.editable_objects(0).unwrap().into_iter().filter_map(|o| o.text).collect(); + assert!(texts.contains(&"Copied".to_string()) && !texts.contains(&"Left".to_string()), "{texts:?}"); + let pictures = images(&s, b); + assert_eq!(pictures.len(), 2); + assert_ne!(pictures[0], pictures[1]); + + // A second paste steps down and to the right, and shares the fonts and pictures the first + // brought instead of copying them again. + let paste = clipboard.paste(s.get(b).unwrap(), 0).unwrap(); + assert!(matches!(&paste, Edit::PasteObjects { offset, .. } if *offset == [18.0, -18.0]), "{paste:?}"); + s.apply(b, paste).unwrap(); + let pictures = images(&s, b); + assert_eq!(pictures.len(), 3); + assert_eq!(pictures[1], pictures[2], "one picture object for both pastes"); + + // Saved and reopened, the pastes are there; undone, the page is as it was. + let saved = s.save_bytes(b).unwrap(); + let mut again = Session::new(); + let reopened = again.open("b.pdf", None, saved, None).unwrap(); + assert_eq!(again.get(reopened).unwrap().page_images(0).len(), 3); + s.undo(b).unwrap(); + s.undo(b).unwrap(); + assert_eq!(images(&s, b).len(), 1); + assert_eq!(s.get(b).unwrap().can_undo(), None); +} + +#[test] +fn a_paste_onto_the_page_it_came_from_steps_and_draws_from_the_same_objects() { + let mut s = Session::new().with_clock(|| 1_700_000_000); + let a = s.open("a.pdf", None, source(), None).unwrap(); + let mut clipboard = ObjectClipboard::new(copy_all_but_left(&s, a)); + let paste = clipboard.paste(s.get(a).unwrap(), 0).unwrap(); + assert!(matches!(&paste, Edit::PasteObjects { offset, .. } if *offset == [18.0, -18.0]), "one step down and right: {paste:?}"); + s.apply(a, paste).unwrap(); + let pictures = images(&s, a); + assert_eq!(pictures.len(), 2); + assert_eq!(pictures[0], pictures[1], "the same picture object, not a copy"); + // Far off the page (a copy from a larger page), a paste is centred instead. + let doc = s.get(a).unwrap(); + let mut far = clipboard.clip().clone(); + far.copied.rect = [1000.0, 1000.0, 1100.0, 1050.0]; + assert_eq!(doc.object_paste_offset(0, &far, 0).unwrap(), [150.0 - 1050.0, 150.0 - 1025.0]); +} + +#[test] +fn pasting_into_another_document_follows_cycles_and_refuses_past_its_limit() { + let mut s = Session::new().with_clock(|| 1_700_000_000); + // A font that refers to itself and to an object that refers back. + let cyclic = pdf( + &[("BT /F1 20 Tf 20 200 Td (Loop) Tj ET", "<< /Font << /F1 5 0 R >> >>")], + &[ + "<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding /Self 5 0 R /Next [6 0 R] >>".into(), + "<< /Back 5 0 R /Again 6 0 R >>".into(), + ], + ); + let a = s.open("cyclic.pdf", None, cyclic, None).unwrap(); + let b = s.open("b.pdf", None, target(), None).unwrap(); + let all: Vec<_> = s.get(a).unwrap().editable_objects(0).unwrap().into_iter().map(|o| o.target).collect(); + let mut clipboard = ObjectClipboard::new(s.get(a).unwrap().copy_objects(0, &all).unwrap()); + s.apply(b, clipboard.paste(s.get(b).unwrap(), 0).unwrap()).unwrap(); + assert!(s.get(b).unwrap().editable_objects(0).unwrap().iter().any(|o| o.text.as_deref() == Some("Loop"))); + + // An ExtGState that drags in more objects than a paste copies is refused with a reason. + let many = 20_001; + let refs: Vec = (0..many).map(|i| format!("{} 0 R", 6 + i)).collect(); + let mut extra = vec![format!("<< /ca 0.5 /Junk [{}] >>", refs.join(" "))]; + extra.extend((0..many).map(|i| format!("<< /N {i} >>"))); + let resources = format!("<< /Font << /F1 {HELVETICA} >> /ExtGState << /GS0 5 0 R >> >>"); + let big = pdf(&[("/GS0 gs BT /F1 20 Tf 20 200 Td (Big) Tj ET", &resources)], &extra); + let c = s.open("big.pdf", None, big, None).unwrap(); + let all: Vec<_> = s.get(c).unwrap().editable_objects(0).unwrap().into_iter().map(|o| o.target).collect(); + assert_eq!(all.len(), 1); + let mut clipboard = ObjectClipboard::new(s.get(c).unwrap().copy_objects(0, &all).unwrap()); + let before = s.get(b).unwrap().can_undo().map(str::to_string); + let err = s.apply(b, clipboard.paste(s.get(b).unwrap(), 0).unwrap()).unwrap_err().to_string(); + assert!(err.contains("copy fewer objects"), "{err}"); + assert_eq!(s.get(b).unwrap().can_undo().map(str::to_string), before, "a refused paste changes nothing"); +} + +#[test] +fn copying_refuses_a_stale_selection_and_deleting_several_objects_is_one_step() { + let mut s = Session::new().with_clock(|| 1_700_000_000); + let a = s.open("a.pdf", None, source(), None).unwrap(); + let doc = s.get(a).unwrap(); + assert!(doc.copy_objects(0, &[ObjectTarget { kind: ObjectKind::Text, index: 7 }]).is_err()); + assert!(doc.copy_objects(3, &[ObjectTarget { kind: ObjectKind::Text, index: 0 }]).is_err()); + let delete = Edit::delete_objects(0, vec![ObjectTarget { kind: ObjectKind::Text, index: 0 }, ObjectTarget { kind: ObjectKind::Image, index: 0 }]); + assert!(matches!(delete, Some(Edit::DeleteObjects { .. }))); + assert!(Edit::delete_objects(0, Vec::new()).is_none()); + s.apply(a, delete.unwrap()).unwrap(); + assert_eq!(s.get(a).unwrap().can_undo(), Some("Delete content")); + assert!(s.get(a).unwrap().editable_objects(0).unwrap().iter().all(|o| o.text.as_deref() == Some("Left"))); +} + +/// Two pastes of a paragraph and a picture exactly over each other: deleting a paragraph also +/// removes a copy drawn exactly over it, so deleting all four objects together found the second +/// paragraph's number gone and refused the whole delete. Both copies go, in one step. +#[test] +fn objects_pasted_exactly_over_each_other_delete_together_in_one_step() { + let mut s = Session::new().with_clock(|| 1_700_000_000); + let a = s.open("a.pdf", None, source(), None).unwrap(); + let blank = s.create_blank(300.0, 300.0, 1).unwrap(); + let b = s.open_new("b.pdf", blank).unwrap(); + let mut clipboard = ObjectClipboard::new(copy_all_but_left(&s, a)); + let Edit::PasteObjects { clip, .. } = clipboard.paste(s.get(b).unwrap(), 0).unwrap() else { panic!("a paste") }; + for _ in 0..2 { + s.apply(b, Edit::PasteObjects { page: 0, clip: Arc::clone(&clip), offset: [0.0, 0.0] }).unwrap(); + } + let pasted = s.get(b).unwrap().editable_objects(0).unwrap(); + assert_eq!(pasted.iter().filter(|o| o.text.as_deref() == Some("Copied")).count(), 2, "{pasted:?}"); + assert_eq!(pasted.len(), 4); + let delete = Edit::delete_objects(0, pasted.iter().map(|o| o.target).collect()).unwrap(); + s.apply(b, delete).unwrap(); + assert!(s.get(b).unwrap().editable_objects(0).unwrap().is_empty()); + assert_eq!(s.get(b).unwrap().can_undo(), Some("Delete content")); + s.undo(b).unwrap(); + assert_eq!(s.get(b).unwrap().editable_objects(0).unwrap(), pasted, "one undo brings both copies back"); +} + +/// Deleting renumbers what follows in the same inventory, so paragraphs and pictures go from +/// the last to the first, and added content (outside those inventories) after them. +#[test] +fn several_objects_delete_last_first() { + let content = + "BT /F1 12 Tf 20 250 Td (A) Tj ET BT /F1 12 Tf 20 180 Td (B) Tj ET BT /F1 12 Tf 20 110 Td (C) Tj ET BT /F1 12 Tf 20 40 Td (D) Tj ET"; + let mut doc = pdfcraft_cos::Document::open(pdf(&[(content, "<< /Font << /F1 5 0 R >> >>")], &[HELVETICA.into()])).unwrap(); + let t = |kind, index| ObjectTarget { kind, index }; + let targets = [t(ObjectKind::Added, 0), t(ObjectKind::Text, 1), t(ObjectKind::Image, 0), t(ObjectKind::Text, 3), t(ObjectKind::Added, 2)]; + let mut order = Vec::new(); + delete(&mut doc, 0, &targets, |_, e| { + order.push(match e { + Edit::EditTextBlock { block, text, .. } if text.is_empty() => ('t', *block), + Edit::EditPageImage { index, change: crate::ImageEdit::Delete, .. } => ('i', *index), + Edit::DeleteContent { index, .. } => ('a', *index), + other => panic!("unexpected {other:?}"), + }); + Ok(()) + }) + .unwrap(); + assert_eq!(order, [('t', 3), ('t', 1), ('i', 0), ('a', 2), ('a', 0)]); +} diff --git a/crates/engine/src/ocr.rs b/crates/engine/src/ocr.rs index cb31e3b550..512c1c97ad 100644 --- a/crates/engine/src/ocr.rs +++ b/crates/engine/src/ocr.rs @@ -971,10 +971,23 @@ impl Session { pub fn ocr_job(&self, id: DocId, pages: &[usize], settings: OcrSettings) -> Option { let doc = self.get(id)?; let all = doc.info.pages.len(); - let pages: Vec = if pages.is_empty() { (0..all).collect() } else { pages.iter().copied().filter(|p| *p < all).collect() }; + let mut seen = vec![false; all]; + let pages: Vec = if pages.is_empty() { + seen.fill(true); + (0..all).collect() + } else { + let mut deduped = Vec::new(); + for &p in pages { + if seen.get(p) == Some(&false) { + seen[p] = true; + deduped.push(p); + } + } + deduped + }; let native = (0..all) .map(|p| { - if !pages.contains(&p) { + if seen.get(p) != Some(&true) { return None; } doc.page_images(p) @@ -2234,6 +2247,59 @@ mod tests { assert_eq!(page, 0); } + /// A scan selected explicitly is rendered at its native scale, like the same page in an + /// all-pages job: selecting pages must not lose the native-resolution lookup and render + /// the scan above its own resolution at a higher chosen dpi. + #[test] + fn selected_scan_page_uses_native_scale_like_all_pages() { + let mut s = Session::new(); + let page = export::encode_png(2480, 3508, &vec![255; (2480 * 3508 * 4) as usize]).unwrap(); + let bytes = s.create_from_images(&[("a4.png".into(), page)]).unwrap(); + let id = s.open("scan.pdf", None, bytes, None).unwrap(); + let settings = OcrSettings { dpi: 300.0, ..Default::default() }; + let all = s.ocr_job(id, &[], settings.clone()).unwrap(); + let selected = s.ocr_job(id, &[0], settings).unwrap(); + assert_eq!(selected.pages, vec![0]); + let wanted = 300.0 / 72.0; + assert!(all.render_scale(0) < wanted, "native resolution limits the all-pages job: {}", all.render_scale(0)); + assert!( + (selected.render_scale(0) - all.render_scale(0)).abs() < 1e-4, + "selected page {} vs all-pages {}", + selected.render_scale(0), + all.render_scale(0) + ); + } + + /// A job requested with the same page twice carries each page once: a duplicate would + /// otherwise be recognized twice. + #[test] + fn ocr_job_deduplicates_requested_pages() { + let mut s = Session::new(); + let id = s.open("t.pdf", None, Arc::new(fixture(3)), None).unwrap(); + let job = s.ocr_job(id, &[2, 0, 2, 1, 0, 2], OcrSettings::default()).unwrap(); + assert_eq!(job.pages, vec![2, 0, 1], "each requested page appears once, in request order"); + } + + #[test] + fn ocr_job_bounds_page_vector_reservation_by_page_count() { + let mut s = Session::new(); + let id = s.open("t.pdf", None, Arc::new(fixture(3)), None).unwrap(); + let requested = vec![999usize; 1024]; + let job = s.ocr_job(id, &requested, OcrSettings::default()).unwrap(); + assert!(job.pages.is_empty()); + assert!(job.pages.capacity() <= 3, "reservation {} exceeds the 3 pages that can survive deduplication", job.pages.capacity()); + } + + #[test] + fn ocr_job_does_not_reserve_capacity_for_invalid_pages() { + let mut s = Session::new(); + let id = s.open("t.pdf", None, Arc::new(fixture(1024)), None).unwrap(); + let requested = vec![4096usize; 1024]; + let job = s.ocr_job(id, &requested, OcrSettings::default()).unwrap(); + assert!(job.pages.is_empty()); + assert_eq!(job.pages.capacity(), 0, "no valid pages were found, so no capacity should be reserved"); + } + /// #1848: an A4 scan at 300 dpi whose page box is its pixel size (2480 x 3508 pt, one pixel /// per point) gets a smaller bitmap for each lower "Downsample to" choice, and a low-resolution /// letter page is still never shrunk below one pixel per point. diff --git a/crates/engine/src/open.rs b/crates/engine/src/open.rs index 80a5ad30c7..28dd235928 100644 --- a/crates/engine/src/open.rs +++ b/crates/engine/src/open.rs @@ -8,7 +8,8 @@ use pdfcraft_render::{DocInfo, OpenError, RenderConfig, RenderPool, inspect}; use crate::history::{Editor, Keys, TextBlocksCache}; use crate::{ AppearanceCache, DocId, Document, Session, StatusCaptions, TrustStore, XFA_SCRIPTS_OFF, XfaLayout, display_bytes, display_config, guard, - info_fields, js, note_warnings, recovered_render_password, render_threads, same_pages, signatures_of, xfa, xfa_layout, xfa_values_from_datasets, + info_fields, js, note_warnings, print_preview_layers, recovered_render_password, render_threads, same_pages, signatures_of, xfa, xfa_layout, + xfa_values_from_datasets, }; /// Immutable session state needed while a document is prepared away from its owning thread. @@ -17,6 +18,7 @@ pub struct OpenContext { now: Option, js_off: bool, trust: Arc, + highlight: bool, captions: Arc, xfa_time_scale: u32, } @@ -38,6 +40,14 @@ struct DocumentOpenData<'a> { } impl OpenContext { + /// This context with Highlight fields `on`, which the document's scripts read as + /// `app.runtimeHighlight` while it opens. A new document opens with the remembered setting + /// while the session keeps the one of the document in front, which its scripts read. + pub fn with_highlight(mut self, on: bool) -> Self { + self.highlight = on; + self + } + /// Prepare a document without mutating the session that supplied this context. pub fn prepare_open( &self, @@ -59,6 +69,7 @@ impl Session { now: self.now(), js_off: self.js_off, trust: self.trust.clone(), + highlight: self.field_highlight, captions: self.captions.clone(), xfa_time_scale: self.xfa_time_scale.unwrap_or(1), } @@ -99,6 +110,8 @@ impl Session { Ok(self.commit_open(prepared)) } + /// What [`OpenContext::prepare_open`] runs inside its panic guard: read, decrypt and lay out + /// the document, and run an XFA form's opening scripts. fn prepare_open_unguarded( context: &OpenContext, name: String, @@ -125,11 +138,17 @@ impl Session { // pages and fields so the rest of the engine works on it. The original bytes stay: the // laid-out form is one appended revision. let opts = SaveOptions { mod_date: context.now.map(pdfcraft_cos::pdf_date), ..SaveOptions::default() }; - // Write `work` as one more revision and reopen it: what the document then is. + // Write `work` as one more revision and reopen it: what the document then is. The notes + // about damage repaired while reading the file stay with it (Document Properties), once. let rebase = |work: &pdfcraft_cos::Document| -> Result<(Arc>, DocInfo, pdfcraft_cos::Document), String> { let new_bytes = write_incremental(work, &opts).map(Arc::new).map_err(|e| e.to_string())?; let new_info = inspect(new_bytes.clone(), render_password.as_deref()).map_err(|e| e.to_string())?; - let new_cos = pdfcraft_cos::Document::open_with_password(new_bytes.clone(), password).map_err(|e| e.to_string())?; + let mut new_cos = pdfcraft_cos::Document::open_with_password(new_bytes.clone(), password).map_err(|e| e.to_string())?; + for line in work.repair_log() { + if !new_cos.repair_log().contains(line) { + new_cos.note_repair(line.clone()); + } + } Ok((new_bytes, new_info, new_cos)) }; let (bytes, info, cos, xfa) = match (info.xfa, cos) { @@ -203,6 +222,7 @@ impl Session { out: &mut script_output, datasets: None, ran_away: false, + highlight: context.highlight, time_scale: context.xfa_time_scale, }; let changes = xfa::on_open(&mut work, &tpl, &mut run); @@ -254,9 +274,6 @@ impl Session { }; match rebuilt { Some((b, i, mut c)) => { - for line in cos.repair_log() { - c.note_repair(line.clone()); - } c.note_repair(format!("the page tree is missing; its {} pages were found by scanning the file", i.pages.len())); (b, i, Ok(Ok(c))) } @@ -283,7 +300,7 @@ impl Session { /// The last step of opening: build the document record and register it. fn build_document(context: &OpenContext, data: DocumentOpenData<'_>) -> Document { let DocumentOpenData { name, path, bytes, mut info, structure: cos, render_password, password, xfa } = data; - let config = RenderConfig { password: render_password.as_deref().map(Arc::from), ..Default::default() }; + let mut config = RenderConfig { password: render_password.as_deref().map(Arc::from), ..Default::default() }; let (mut editor, read_only_reason) = match cos { Ok(Ok(cos)) => { let keys = Keys { render: render_password.clone(), reopen: password.map(str::to_owned) }; @@ -318,6 +335,8 @@ impl Session { } } } + // The print preview draws the layers as they print. + config.print_layers = print_preview_layers(editor.as_ref().map(|e| &e.cos), &info.layers); let display = display_bytes(editor.as_mut(), &bytes, &mut signatures, &context.captions); let renderer = RenderPool::new(display.clone(), render_threads(), display_config(&config)); let (form, adopted) = editor @@ -365,6 +384,7 @@ impl Session { xfa, xfa_template: None, xfa_scripts_off: false, + xfa_exited: Default::default(), fonts_stale: false, xfa_warnings: Vec::new(), edit_notices: Vec::new(), diff --git a/crates/engine/src/optimizer.rs b/crates/engine/src/optimizer.rs index 30a14c5695..1a741c0912 100644 --- a/crates/engine/src/optimizer.rs +++ b/crates/engine/src/optimizer.rs @@ -12,11 +12,12 @@ use pdfcraft_cos::{SaveOptions, write_full}; use crate::{DocId, EditError, Hidden, OptimizeReport, Session, guard, optimize}; /// Where an optimization is. -#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +#[derive(Clone, Copy, Debug, PartialEq, Eq)] pub enum OptimizeStage { - /// Removing the user data chosen under Discard User Data. - #[default] + /// Removing the user data chosen under Discard User Data, if any was. Discarding, + /// Reading what the pages draw: `done` of `total` read. + Reading { done: usize, total: usize }, /// Merging the fragments of a page picture into one image. Fragments, /// About to process image `done` (0-based) of `total`. @@ -29,14 +30,21 @@ pub enum OptimizeStage { Writing, } +impl Default for OptimizeStage { + fn default() -> Self { + OptimizeStage::Reading { done: 0, total: 0 } + } +} + impl OptimizeStage { - /// How far along the whole run is, from 0 to 1. Images take most of the time; the rest are - /// small fixed steps. + /// How far along the whole run is, from 0 to 1, weighed by where the time goes: reading pages + /// and pictures. The rest are small fixed steps. pub fn fraction(self) -> f32 { + let part = |done: usize, total: usize| if total == 0 { 1.0 } else { done.min(total) as f32 / total as f32 }; match self { - OptimizeStage::Discarding => 0.0, - OptimizeStage::Fragments => 0.03, - OptimizeStage::Images { done, total } => 0.05 + 0.75 * if total == 0 { 1.0 } else { done.min(total) as f32 / total as f32 }, + OptimizeStage::Discarding | OptimizeStage::Fragments => 0.0, + OptimizeStage::Reading { done, total } => 0.25 * if total == 0 { 0.0 } else { part(done, total) }, + OptimizeStage::Images { done, total } => 0.25 + 0.55 * part(done, total), OptimizeStage::CleaningUp => 0.8, OptimizeStage::Merging => 0.85, OptimizeStage::Writing => 0.9, @@ -91,10 +99,10 @@ impl OptimizeJob { pdfcraft_forms::clear_password_values(&mut cos).map_err(|e| EditError::Write(e.to_string()))?; pdfcraft_xfa::clear_password_values(&mut cos).map_err(|e| EditError::Write(e.to_string()))?; let mut step = |stage| if progress(stage) { Ok(()) } else { Err(EditError::Cancelled) }; - step(OptimizeStage::Discarding)?; let discarded = if discard.is_empty() { Vec::new() } else { + step(OptimizeStage::Discarding)?; // Every ticked category, as `remove_hidden` decides between them: with form fields // ticked, discarding hidden text also takes the hidden text the flattened appearances // bring onto the page, which no scan before flattening counts. It refuses when its scan @@ -112,6 +120,7 @@ impl OptimizeJob { }; let report = optimize::optimize_with_progress(&mut cos, &settings, &mut |s| { step(match s { + optimize::Stage::Reading { done, total } => OptimizeStage::Reading { done, total }, optimize::Stage::Fragments => OptimizeStage::Fragments, optimize::Stage::Images { done, total } => OptimizeStage::Images { done, total }, optimize::Stage::CleanUp => OptimizeStage::CleaningUp, diff --git a/crates/engine/src/print_raster.rs b/crates/engine/src/print_raster.rs index 4ed5895860..c5a1295e92 100644 --- a/crates/engine/src/print_raster.rs +++ b/crates/engine/src/print_raster.rs @@ -11,7 +11,7 @@ use pdfcraft_print::{Content, Raster}; use pdfcraft_render::{PRINT_TAG, PageRenderer, RenderRequest}; use crate::export::encode_png; -use crate::{Document, guard}; +use crate::{Document, guard, print_layer_states}; /// `doc`'s pages `pages` (0-based, each once, in the order given) as an image-only PDF, one /// page per entry, the size of the source page as displayed, drawing what `content` prints. @@ -22,11 +22,19 @@ pub fn rasterize(doc: &Document, pages: &[usize], raster: &Raster, content: Cont let mut config = src.config.clone(); // The copy below holds only what prints, so the viewer's own comment hiding plays no part. config.hide_comments = false; - let printed = guard(|| -> Result, String> { + // Printing is not the View event: a layer the auto state turns off on screen keeps its + // configured state on paper, as in a print that goes out as vectors. + config.for_print = true; + let (printed, layers) = guard(|| -> Result<(Vec, Option>), String> { let mut cos = pdfcraft_cos::Document::open_with_password(src.bytes.clone(), config.password.as_deref()).map_err(|e| e.to_string())?; pdfcraft_print::keep_printed(&mut cos, content); - write_incremental(&cos, &SaveOptions::default()).map_err(|e| e.to_string()) + // The layers as the vector print sets them: as shown, then the Print event's auto state. + let layers = print_layer_states(&cos, &doc.info.layers); + Ok((write_incremental(&cos, &SaveOptions::default()).map_err(|e| e.to_string())?, layers)) })??; + if let Some(layers) = layers { + config.layers = std::sync::Arc::new(layers.iter().map(|l| (l.id.0 as i32, l.id.1 as i32, l.visible)).collect()); + } let mut renderer = PageRenderer::new(std::sync::Arc::new(printed), config); let mut images: Vec<(String, Vec)> = Vec::with_capacity(pages.len()); let mut used: Option = None; diff --git a/crates/engine/src/rich_copy.rs b/crates/engine/src/rich_copy.rs index 2e579adb19..a551ad2e9c 100644 --- a/crates/engine/src/rich_copy.rs +++ b/crates/engine/src/rich_copy.rs @@ -382,7 +382,7 @@ mod tests { let bytes = Arc::new(pdfcraft_cos::write_full(&cos, &Default::default()).unwrap()); let mut s = Session::new(); let id = s.open("locked.pdf", None, bytes, None).unwrap(); - let text = pdfcraft_render::PageText { glyphs: Vec::new(), line_of: Vec::new(), space_before: Vec::new() }; + let text = pdfcraft_render::PageText { glyphs: Vec::new(), line_of: Vec::new(), space_before: Vec::new(), truncated: false }; let err = s.get(id).unwrap().rich_copy(0, &text, &[]).unwrap_err(); assert!(matches!(err, crate::EditError::NotPermitted(_)), "{err:?}"); } diff --git a/crates/engine/src/session.rs b/crates/engine/src/session.rs index e5029ccbd6..bd83242682 100644 --- a/crates/engine/src/session.rs +++ b/crates/engine/src/session.rs @@ -10,12 +10,13 @@ use crate::check_refresh_panic; use crate::edit::{EditCtx, check_permission, run_edit}; use crate::history::{Keys, MAX_UNDO, Scope, keys_after, scope_of, uses_scripts}; use crate::{ - BookmarkPage, BookmarkRow, CombineSource, Combined, DataFormat, DocId, Document, Edit, EditError, Hidden, ImageResolution, - LOW_RESOLUTION_PRINT_DPI, Label, ObjectKind, OptimizeReport, RecoverySnapshot, SignOptions, SourceKind, SplitPart, StatusCaptions, SummarySort, - TrustStore, XFA_REMOVED_PREFIX, XFA_SCRIPTS_OFF, apply_layer_state, carry_verdict_marks, comment_list, comment_summary, decode_text, - display_bytes, display_config, guard, info_fields, js, local_utc_offset, note_warnings, open_source, open_source_with, optimize, outline_rows, - parse_ymd, print, print_raster, render_threads, scan_colours, set_default_layer_state, shown_under_redactions, signatures_of, source_kind, - sync_xfa_state, undrawn_notices, use_layer_choices, writable, xfa, xfa_sync_datasets, + BookmarkPage, BookmarkRow, CombineSource, Combined, DataFormat, DocId, Document, Edit, EditError, Hairlines, HairlinesFixed, Hidden, + ImageResolution, LOW_RESOLUTION_PRINT_DPI, Label, ObjectKind, OptimizeReport, RecoverySnapshot, SignOptions, SourceKind, SplitPart, + StatusCaptions, SummarySort, TrustStore, XFA_REMOVED_PREFIX, XFA_SCRIPTS_OFF, apply_layer_state, carry_verdict_marks, comment_list, + comment_summary, decode_text, display_bytes, display_config, guard, info_fields, js, local_utc_offset, note_warnings, open_source, + open_source_with, optimize, outline_rows, parse_ymd, print, print_layer_states, print_raster, render_threads, scan_colours, + set_default_layer_state, shown_under_redactions, signatures_of, source_kind, sync_xfa_state, undrawn_notices, use_layer_choices, writable, xfa, + xfa_sync_datasets, }; mod editing; @@ -44,6 +45,9 @@ pub struct Session { pub(crate) date_format: Option, /// Preferences ▸ Date format ▸ Language, when not following the interface language. pub(crate) date_language: Option, + /// Highlight fields, as the viewer shows it: what forms' scripts read as + /// `app.runtimeHighlight` (off by default, as in a session without a viewer). + pub(crate) field_highlight: bool, /// What the XFA scripts' time limits are multiplied by (`None`: 1); see /// [`Session::with_xfa_time_scale`]. pub(crate) xfa_time_scale: Option, diff --git a/crates/engine/src/session/editing.rs b/crates/engine/src/session/editing.rs index ae0a83a8f8..1584fb5209 100644 --- a/crates/engine/src/session/editing.rs +++ b/crates/engine/src/session/editing.rs @@ -6,7 +6,7 @@ use super::{ inspect, js, keys_after, note_warnings, render_threads, run_edit, scan_colours, scope_of, shown_under_redactions, signatures_of, sync_xfa_state, undrawn_notices, uses_scripts, write_incremental, xfa, xfa_sync_datasets, }; -use crate::{EditNotice, MarkKind}; +use crate::{EditNotice, MarkKind, print_preview_layers}; /// Whether the edit, or any edit inside a batch, deletes pages. fn deletes_pages(edit: &Edit) -> bool { @@ -36,6 +36,7 @@ impl Session { let now = self.now(); let today = self.today(); let js_off = self.js_off; + let highlight = self.field_highlight; let xfa_time_scale = self.xfa_time_scale.unwrap_or(1); let doc = self.doc_mut(id)?; doc.edit_notices.clear(); @@ -48,6 +49,7 @@ impl Session { } cx.xfa_time_scale = xfa_time_scale; cx.pages = doc.info.pages.len(); + cx.highlight = highlight; cx.today = today; let reason = doc.read_only_reason.clone().unwrap_or_default(); let signed = doc.is_signed(); @@ -59,6 +61,7 @@ impl Session { // in the paper's colour and draw the new text in the ink's. let edit = match edit { Edit::EditTextBlock { page, block, text, mut style } => { + let plain = style == pdfcraft_edit::BlockStyle::default(); // Only invisible text is sampled; the blocks are usually cached from opening it. if style.cover.is_none() && let Some(info) = doc.info.pages.get(page) @@ -67,6 +70,12 @@ impl Session { { style.cover = Some(paper); style.color = style.color.or(Some(ink)); + // A text edit alone leaves the words it doesn't change as the scan shows them. + style.changed_words_only = plain; + // Nothing to redraw: leave undo history and the dirty flag alone. + if plain && b.same_words(&text) { + return Ok(None); + } } Edit::EditTextBlock { page, block, text, style } } @@ -78,13 +87,20 @@ impl Session { } let mut next = editor.cos.clone(); if !js_off && uses_scripts(&edit) { - cx.js = Some(js::JsRunner::new(&next, &name)); + cx.js = Some(js::JsRunner::new(&next, &name, highlight)); } cx.renamed_fields.clear(); cx.redaction_overlays.clear(); // `next` is a copy: if the edit fails or crashes, the document is unchanged. guard(|| run_edit(&mut next, &edit, &mut cx)) .unwrap_or_else(|m| Err(EditError::Invalid(format!("{} failed unexpectedly ({m}); the document was not changed", edit.label()))))?; + // A field's enter or exit scripts that changed nothing leave no edit behind (no undo step, + // nothing unsaved); what they showed or asked for still goes out. Only an event on its own: + // a batch keeps the other edits it holds. + if cx.xfa_unchanged && matches!(edit, Edit::XfaEvent { .. }) { + take_xfa_output(doc, cx.xfa_out, cx.xfa_ran_away); + return Ok(None); + } // Redaction removed what its own interpreter places under the marks. The marks were drawn // over the page as the renderer shows it, so the renderer has the last word. if let Edit::ApplyRedactions { pages } = &edit { @@ -123,7 +139,8 @@ impl Session { }; // An XFA form's scripts answer the change: exit and validate scripts of the field, then // every calculation (and a reset recalculates). They read the data, so it is brought up - // to date first. + // to date first. The fields whose exit scripts ran are noted once the edit is kept. + let mut exited: Vec = Vec::new(); if let Some(tpl) = cx.xfa.clone() { if is_xfa && matches!(scope_of(&edit), Scope::Form | Scope::Full | Scope::Content) { let datasets = &mut cx.xfa_datasets; @@ -132,6 +149,8 @@ impl Session { .map_err(EditError::Write)?; cx.xfa_out.errors.extend(notes); } + /// The fields edit `e` (and the edits it batches) gives new values, and whether it + /// resets the form. fn collect(e: &Edit, changed: &mut Vec, reset: &mut bool) { match e { Edit::SetFieldValue { name, .. } => changed.push(name.clone()), @@ -144,19 +163,24 @@ impl Session { let mut reset = false; collect(&edit, &mut changed, &mut reset); if !changed.is_empty() || reset { + let highlight = cx.highlight(); let mut run = xfa::XfaRun { page_count: cx.pages, out: &mut cx.xfa_out, datasets: cx.xfa_datasets, ran_away: false, + highlight, time_scale: cx.xfa_time_scale, }; // Every changed field's scripts, then the calculations once (after a reset, // only those). - let ran = guard(|| xfa::on_changes(&mut next, &tpl, &changed, &mut run)) + let ran = guard(|| xfa::on_changes(&mut next, &tpl, &changed, &mut run, &mut exited)) .unwrap_or_else(|m| Err(format!("the form's scripts failed unexpectedly ({m})"))); cx.xfa_datasets = run.datasets; cx.xfa_ran_away |= run.ran_away; + // The next edit of a batch reads the highlight this one's scripts set. + let on = run.highlight; + cx.set_highlight(on); if let Err(e) = ran { cx.xfa_out.errors.push(e); } @@ -194,7 +218,13 @@ impl Session { if signed && ((rewrites(&next) && !rewrites(&editor.cos)) || (next.is_modified() && next.revisions().is_empty())) { return Err(EditError::SignedRewrite(edit.label().to_string())); } - let notices = edit_notices(&edit, &editor.cos, &next); + let mut notices = edit_notices(&edit, &editor.cos, &next); + // Places applied redactions left that still hold the removed text, and what became of + // the PDF/A or PDF/UA identification in the XMP they removed: said, not silent. + notices.extend(cx.redaction_elsewhere.filter(|e| e.total() > 0 || e.unsearched).map(EditNotice::RedactionElsewhere)); + if cx.redaction_identification != pdfcraft_redact::Identification::default() { + notices.push(EditNotice::RedactionIdentification(cx.redaction_identification)); + } let previous = std::mem::replace(&mut editor.cos, next); let keys = keys_after(&edit).unwrap_or_else(|| editor.keys.clone()); let previous_keys = std::mem::replace(&mut editor.keys, keys); @@ -223,6 +253,10 @@ impl Session { doc.dirty = true; doc.generation += 1; doc.edit_notices = notices; + // Only this edit's exits are skipped on the next focus loss: one an earlier change ran is + // no reason to skip a later one. + doc.xfa_exited.clear(); + doc.xfa_exited.extend(exited); if !cx.xfa_removed.is_empty() { let removed: Vec = cx.xfa_removed.iter().map(|n| format!("{XFA_REMOVED_PREFIX}{n}")).collect(); doc.xfa_warnings.retain(|w| w.starts_with(XFA_REMOVED_PREFIX)); @@ -235,15 +269,11 @@ impl Session { if let Some(js) = cx.js { doc.js_output.append(js.output); } - doc.js_output.append(cx.xfa_out); - if cx.xfa_ran_away { - doc.xfa_template = None; - doc.xfa_scripts_off = true; - doc.js_output.errors.push(XFA_SCRIPTS_OFF.into()); - } + take_xfa_output(doc, cx.xfa_out, cx.xfa_ran_away); Ok(cx.prepared) } + /// Undo the last edit of document `id`, keeping it for [`Session::redo`]. Returns its label. pub fn undo(&mut self, id: DocId) -> Result { let doc = self.doc_mut(id)?; let editor = doc.editor.as_mut().ok_or(EditError::NothingToUndo)?; @@ -269,10 +299,13 @@ impl Session { } doc.edit_notices.clear(); Self::sync_dirty(doc); + // The change whose exit ran is undone (or redone without its scripts): skip no exit. + doc.xfa_exited.clear(); doc.generation += 1; Ok(label) } + /// Redo the last edit of document `id` that [`Session::undo`] took back. Returns its label. pub fn redo(&mut self, id: DocId) -> Result { let doc = self.doc_mut(id)?; let editor = doc.editor.as_mut().ok_or(EditError::NothingToRedo)?; @@ -298,6 +331,8 @@ impl Session { } doc.edit_notices.clear(); Self::sync_dirty(doc); + // The change whose exit ran is undone (or redone without its scripts): skip no exit. + doc.xfa_exited.clear(); doc.generation += 1; Ok(label) } @@ -434,10 +469,17 @@ impl Session { | Edit::ReplaceImage { .. } | Edit::PasteContent { .. } | Edit::EditPageImage { .. } - ) || matches!(e, Edit::MoveObjects { objects, .. } if objects.iter().any(|o| o.kind == ObjectKind::Added)) + ) || matches!(e, Edit::MoveObjects { objects, .. } | Edit::DeleteObjects { objects, .. } if objects.iter().any(|o| o.kind == ObjectKind::Added)) }); let touches_marks = edit.is_none_or(|e| { - matches!(e, Edit::AddHeaderFooter { .. } | Edit::AddWatermark { .. } | Edit::AddBackground { .. } | Edit::RemoveMarks { .. }) + matches!( + e, + Edit::AddHeaderFooter { .. } + | Edit::AddWatermark { .. } + | Edit::AddBackground { .. } + | Edit::AddPrinterMarks { .. } + | Edit::RemoveMarks { .. } + ) }); (touches_added, touches_marks) } @@ -515,6 +557,8 @@ impl Session { } fn refresh_with_indexes(doc: &mut Document, invalidate_markers: bool, invalidate_links: bool) -> Result<(), EditError> { + #[cfg(test)] + crate::record_full_refresh(); // A full refresh may follow structural or annotation changes: cached appearances // may be stale, so they are rebuilt below. if let Some(editor) = doc.editor.as_mut() { @@ -545,6 +589,7 @@ impl Session { } } doc.info = info; + doc.config.print_layers = print_preview_layers(Some(&editor.cos), &doc.info.layers); doc.fonts_stale = false; let mut form = pdfcraft_forms::fields(&editor.cos); xfa::mark_script_buttons(&editor.cos, &mut form); @@ -564,6 +609,17 @@ impl Session { } } +/// Hand on what an edit's XFA scripts showed or asked for (`out`), and when one of them ran away, +/// turn the form's scripts off and say so. +fn take_xfa_output(doc: &mut Document, out: js::JsOutput, ran_away: bool) { + doc.js_output.append(out); + if ran_away { + doc.xfa_template = None; + doc.xfa_scripts_off = true; + doc.js_output.errors.push(XFA_SCRIPTS_OFF.into()); + } +} + /// What `edit`, taking `prev` to `next`, wants said: never silent (AGENTS.md §4). fn edit_notices(edit: &Edit, prev: &pdfcraft_cos::Document, next: &pdfcraft_cos::Document) -> Vec { // What it drew with `?` (a value or comment text its font can't draw, stored as typed). diff --git a/crates/engine/src/session/files.rs b/crates/engine/src/session/files.rs index f4e25bc2f6..4177d1705f 100644 --- a/crates/engine/src/session/files.rs +++ b/crates/engine/src/session/files.rs @@ -1,7 +1,7 @@ use super::{ - Arc, CombineSource, Combined, DataFormat, DocId, EditError, Hidden, ImageResolution, LOW_RESOLUTION_PRINT_DPI, OptimizeReport, SaveOptions, - Session, SourceKind, SplitPart, decode_text, guard, open_source, open_source_with, optimize, print, print_raster, set_default_layer_state, - source_kind, writable, write_full, + Arc, CombineSource, Combined, DataFormat, DocId, EditError, Hairlines, HairlinesFixed, Hidden, ImageResolution, LOW_RESOLUTION_PRINT_DPI, + OptimizeReport, SaveOptions, Session, SourceKind, SplitPart, decode_text, guard, open_source, open_source_with, optimize, print, + print_layer_states, print_raster, set_default_layer_state, source_kind, writable, write_full, }; impl Session { @@ -25,15 +25,17 @@ impl Session { self.write_new(&pdfcraft_create::from_images_with_resolution(images, resolution)?) } - /// A new document from plain text (US Letter, 11 pt Helvetica). Text Helvetica can't draw is + /// A new document from plain text (US Letter, 11 pt Helvetica, or the bundled Inter face for + /// text with Latin, Greek or Cyrillic letters Helvetica can't draw). Text neither can draw is /// refused, naming the line and the character (never set as `?`). pub fn create_from_text(&self, title: &str, text: &str) -> Result>, EditError> { self.write_new(&pdfcraft_create::from_text(title, text, pdfcraft_create::LETTER, 11.0)?) } /// [`Self::create_from_text`] for a report PdfCraft writes from documents (comment summaries, - /// compare reports), which can quote any language: characters Helvetica can't draw are written - /// as `[U+XXXX]`, and the report's first line says so, rather than the report being refused. + /// compare reports), which can quote any language: when the bundled Inter face can't draw it + /// all, characters Helvetica can't draw are written as `[U+XXXX]`, and the report's first line + /// says so, rather than the report being refused. pub(crate) fn create_report(&self, title: &str, text: &str) -> Result>, EditError> { let (doc, _escaped) = pdfcraft_create::from_text_escaped(title, text, pdfcraft_create::LETTER, 11.0)?; self.write_new(&doc) @@ -114,6 +116,33 @@ impl Session { .map_err(EditError::Write) } + /// What Convert colours would change on `pages` of `id`, without changing the document. + pub fn count_colours(&self, id: DocId, pages: &[usize], settings: &pdfcraft_edit::ConvertColours) -> pdfcraft_edit::ColoursConverted { + let Some(doc) = self.get(id) else { return Default::default() }; + let Some(editor) = doc.editor.as_ref() else { return Default::default() }; + let mut cos = editor.cos.clone(); + let mut recolour = |doc: &pdfcraft_cos::Document, s: &pdfcraft_cos::Stream, t: pdfcraft_edit::ColourTarget| { + let target = match t { + pdfcraft_edit::ColourTarget::Gray => pdfcraft_optimize::ImageTarget::Gray, + pdfcraft_edit::ColourTarget::Rgb => pdfcraft_optimize::ImageTarget::Rgb, + pdfcraft_edit::ColourTarget::Cmyk => pdfcraft_optimize::ImageTarget::Cmyk, + }; + pdfcraft_optimize::recolour_image(doc, s, target) + }; + // The preview runs the image decoders over the document's own bytes; a panic in one is + // a count of nothing, not a crash. + guard(|| pdfcraft_edit::convert_colours(&mut cos, pages, settings, &mut recolour).unwrap_or_default()).unwrap_or_default() + } + + /// How many hairlines [`Edit::FixHairlines`] would change on `pages` of `id`, and on how many + /// pages, without changing the document (the edit itself refuses when there is nothing to fix). + pub fn count_hairlines(&self, id: DocId, pages: &[usize], settings: &Hairlines) -> HairlinesFixed { + let Some(doc) = self.get(id) else { return HairlinesFixed::default() }; + let Some(editor) = doc.editor.as_ref() else { return HairlinesFixed::default() }; + let mut cos = editor.cos.clone(); + pdfcraft_edit::fix_hairlines(&mut cos, pages, settings).unwrap_or_default() + } + /// XFDF with these comments (0-based page, position in the page's `/Annots`) and their /// replies: what Copy puts on the clipboard; [`Edit::PasteComments`] takes it back in. pub fn export_comments(&self, id: DocId, which: &[(usize, usize)]) -> Result { @@ -164,10 +193,21 @@ impl Session { pdfcraft_cos::Document::open_with_password(doc.bytes.clone(), doc.password.as_deref()).map_err(|e| EditError::Write(e.to_string()))? } }; + // Comments and fields without an appearance print as the page shows them (and as Print + // as Image already printed them): the vector print drew only appearance streams, so a + // text box without one was on screen but not on paper. ISO 32000-2 §12.5.3: an + // annotation with the Print flag is printed. + if let Some(displayed) = guard(|| crate::document::as_displayed(&cos)).ok().flatten() { + cos = displayed; + } // #3046: the layers print as they are shown. Once the Layers panel (or a set-layer // action) has changed one, the print copy's default configuration says which are on, so a // hidden layer stays off on paper too; untouched, the document's own configuration prints. - if !doc.config.layers.is_empty() { + // On top of either, the configuration's auto state for the Print event sets the layers it + // names (`print_layer_states`), as printing does in a viewer (ISO 32000-2 §8.11.4.4). + if let Some(printed) = print_layer_states(&cos, &doc.info.layers) { + set_default_layer_state(&mut cos, &printed); + } else if !doc.config.layers.is_empty() { set_default_layer_state(&mut cos, &doc.info.layers); } // Signature states as the screen shows them, so not when its display copy could not be @@ -189,7 +229,13 @@ impl Session { Err(_) => return Vec::new(), }, }; - (0..doc.info.pages.len()).map(|i| pages.contains(&i) && pdfcraft_print::text_reads_up(&cos, i)).collect() + // The page tree is read once, and each page is looked up, not searched for, in `pages`. + let mut out = vec![false; doc.info.pages.len()]; + let wanted: Vec = pages.iter().copied().filter(|&p| p < out.len()).collect::>().into_iter().collect(); + for (p, up) in wanted.iter().zip(pdfcraft_print::texts_read_up(&cos, &wanted)) { + out[*p] = up; + } + out } /// Combine whole files, in order, into new PDF bytes (one bookmark per file). @@ -294,7 +340,8 @@ impl Session { /// [`Self::combine_unlocked`] where sources with the same `groups` key (by position) are one /// file split into several runs placed apart: each file is opened and copied once, so it keeps - /// one bookmark, its links between its own pages, its fields and its attachments whole (see + /// one bookmark, and its links between its own pages and its fields whole across its runs; its + /// attachments come along when its runs use all of its pages (see /// `pdfcraft_organize::combine_grouped`). A group's later sources use its first source's bytes /// and password; a source without a key is a file of its own. pub fn combine_grouped(&self, sources: &[CombineSource], groups: &[u64], passwords: &[Option<&str>]) -> Result>, EditError> { diff --git a/crates/engine/src/session/save.rs b/crates/engine/src/session/save.rs index 34f4d4405d..dd32cdbc5c 100644 --- a/crates/engine/src/session/save.rs +++ b/crates/engine/src/session/save.rs @@ -145,6 +145,8 @@ impl Session { editor.redo.clear(); Self::adopt_keys(doc); Self::sync_dirty(doc); + // The changes whose exits ran are gone with the rest: skip no exit. + doc.xfa_exited.clear(); doc.generation += 1; Self::refresh(doc).inspect_err(|_| Self::show_released(doc)) } @@ -217,6 +219,15 @@ impl Session { out } + /// The snapshot [`Self::autosave_snapshots`] handed out for `id` could not be stored: the + /// document is due again, so the next autosave tries it again. + pub fn autosave_failed(&mut self, id: DocId) { + if let Some(d) = self.docs.iter_mut().find(|d| d.id == id) { + // Any generation but the current one leaves it due. + d.snapshot_generation = d.generation.wrapping_sub(1); + } + } + /// Mark a document opened from a recovery file: it has unsaved changes and belongs at /// `path` (where Save writes), as when the session ended. pub fn mark_recovered(&mut self, id: DocId, path: Option) { diff --git a/crates/engine/src/session/view.rs b/crates/engine/src/session/view.rs index 5ef135d6c8..7232f64631 100644 --- a/crates/engine/src/session/view.rs +++ b/crates/engine/src/session/view.rs @@ -62,20 +62,54 @@ impl Session { return None; } - /// Today's date in local time: (year, month, day). With an injected clock (tests) the clock - /// is taken as local time. /// A dynamic stamp's second line: "By Ada at 2:14 pm, Oct 02, 2026" (local time). pub fn stamp_by_line(&self, author: &str) -> String { + let (date, time) = self.stamp_date_time(); + let who = if author.trim().is_empty() { String::new() } else { format!("By {} ", author.trim()) }; + format!("{who}at {time}, {date}") + } + + /// A custom stamp's text line (#1741): `template` with `{name}` (the author), `{date}` + /// ("Oct 02, 2026") and `{time}` ("2:14 pm") filled in now, in local time, as the dynamic + /// stamps' "By" line has them. Cut to [`MAX_STAMP_LINE`](crate::MAX_STAMP_LINE) characters. + pub fn stamp_line(&self, template: &str, author: &str) -> String { + let (date, time) = self.stamp_date_time(); + // One pass over the template, so a placeholder the author's name happens to contain + // (`Ada {date}`) stays as written rather than being filled in too. + let mut filled = String::new(); + let mut rest = template.trim(); + while let Some(c) = rest.chars().next() { + if let Some(after) = rest.strip_prefix("{name}") { + filled.push_str(author.trim()); + rest = after; + } else if let Some(after) = rest.strip_prefix("{date}") { + filled.push_str(&date); + rest = after; + } else if let Some(after) = rest.strip_prefix("{time}") { + filled.push_str(&time); + rest = after; + } else { + filled.push(c); + rest = rest.get(c.len_utf8()..).unwrap_or_default(); + } + } + filled.chars().take(crate::MAX_STAMP_LINE).collect() + } + + /// Now, as the stamps write it: ("Oct 02, 2026", "2:14 pm"), local time. + fn stamp_date_time(&self) -> (String, String) { let offset = self.local_offset(); let d = self.now().map(|t| pdfcraft_cos::pdf_date(t + offset)).unwrap_or_default(); let num = |a: usize, b: usize| d.get(a..b).and_then(|s| s.parse::().ok()).unwrap_or(0); let (y, mo, day, hh, mm) = (num(2, 6), num(6, 8), num(8, 10), num(10, 12), num(12, 14)); const MONTHS: [&str; 12] = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"]; let h12 = if hh % 12 == 0 { 12 } else { hh % 12 }; - let who = if author.trim().is_empty() { String::new() } else { format!("By {} ", author.trim()) }; - format!("{who}at {h12}:{mm:02} {}, {} {day:02}, {y}", if hh < 12 { "am" } else { "pm" }, MONTHS[(mo.clamp(1, 12) - 1) as usize]) + let month = MONTHS.get(mo.clamp(1, 12) as usize - 1).copied().unwrap_or("Jan"); + (format!("{month} {day:02}, {y}"), format!("{h12}:{mm:02} {}", if hh < 12 { "am" } else { "pm" })) } + /// Today's date in local time: (year, month, day). With an injected clock (tests) the clock + /// is taken as local time. pub fn today(&self) -> (i64, u32, u32) { let offset = self.local_offset(); self.now().map(|t| pdfcraft_cos::pdf_date(t + offset)).as_deref().and_then(parse_ymd).unwrap_or((1970, 1, 1)) @@ -100,11 +134,11 @@ impl Session { } /// Web addresses in the text of every page (Create links from URLs): (page, one rectangle - /// per line in user space, URI). Text that already has a link over it is skipped. + /// per line in user space, URI). Text that already has a link over it is skipped, and so is + /// text on a layer the viewer hides: a link there would sit over nothing the reader can see. pub fn find_urls(&self, id: DocId) -> Vec<(usize, Vec<[f64; 4]>, String)> { let Some(doc) = self.get(id) else { return Vec::new() }; - let config = RenderConfig { password: doc.password.as_deref().map(Arc::from), ..Default::default() }; - let mut r = pdfcraft_render::PageRenderer::new(doc.bytes.clone(), config); + let mut r = pdfcraft_render::PageRenderer::new(doc.bytes.clone(), doc.config.clone()); let mut out = Vec::new(); for page in 0..doc.info.pages.len() { let res = r.render(pdfcraft_render::RenderRequest { page, kind: pdfcraft_render::RequestKind::Text, scale: 1.0, ..Default::default() }); @@ -120,17 +154,7 @@ impl Session { let rects: Vec<[f64; 4]> = text .line_rects(glyphs) .into_iter() - .map(|v| { - let q = doc.info.pages[page].view_rect_to_quad(v); - let xs = [q[0], q[2], q[4], q[6]]; - let ys = [q[1], q[3], q[5], q[7]]; - [ - xs.iter().copied().fold(f64::MAX, f64::min), - ys.iter().copied().fold(f64::MAX, f64::min), - xs.iter().copied().fold(f64::MIN, f64::max), - ys.iter().copied().fold(f64::MIN, f64::max), - ] - }) + .filter_map(|v| line_user_rect(&doc.info.pages[page], v)) .filter(|r| { !doc.links().iter().any(|l| l.page == page && l.rect[0] < r[2] && r[0] < l.rect[2] && l.rect[1] < r[3] && r[1] < l.rect[3]) }) @@ -143,6 +167,40 @@ impl Session { out } + /// The addresses written as plain text that the viewer makes clickable + /// ([`crate::text_links::find_text_links`]), leaving out any a link annotation lies over, from + /// the 0-based page `from` on. One call reads at most + /// [`MAX_TEXT_LINK_PAGES_PER_SCAN`](crate::text_links::MAX_TEXT_LINK_PAGES_PER_SCAN) pages + /// and stops after the page that brings it to + /// [`MAX_TEXT_LINKS_PER_SCAN`](crate::text_links::MAX_TEXT_LINKS_PER_SCAN) addresses; then + /// `next_page` says where to go on. + pub fn text_links(&self, id: DocId, from: usize) -> crate::text_links::TextLinkScan { + use crate::text_links::{MAX_TEXT_LINK_PAGES_PER_SCAN, MAX_TEXT_LINKS_PER_SCAN, TextLinkScan}; + let Some(doc) = self.get(id) else { return TextLinkScan::default() }; + // The viewer's settings, so text on a layer it hides is not a link. + let mut r = pdfcraft_render::PageRenderer::new(doc.bytes.clone(), doc.config.clone()); + let mut scan = TextLinkScan::default(); + let pages = doc.info.pages.len(); + for (page, info) in doc.info.pages.iter().enumerate().skip(from).take(MAX_TEXT_LINK_PAGES_PER_SCAN) { + let res = r.render(pdfcraft_render::RenderRequest { page, kind: pdfcraft_render::RequestKind::Text, scale: 1.0, ..Default::default() }); + if let Some(text) = res.text { + let annotations: Vec<_> = doc.info.links.iter().filter(|l| l.page == page).collect(); + for link in crate::text_links::find_text_links(&text) { + if !annotations.iter().any(|l| link.covered_by(info, l.rect)) { + scan.links.push((page, link)); + } + } + } + let next = page.saturating_add(1); + let read = next.saturating_sub(from); + if next < pages && (read >= MAX_TEXT_LINK_PAGES_PER_SCAN || scan.links.len() >= MAX_TEXT_LINKS_PER_SCAN) { + scan.next_page = Some(next); + break; + } + } + scan + } + /// A page of the bookmark list in outline order: up to `limit` bookmarks from position `offset`. /// The document's own structure is read for it, so every bookmark can be reached by paging. pub fn bookmark_page(&self, id: DocId, offset: usize, limit: usize) -> Option { @@ -287,6 +345,8 @@ impl Session { /// with its number, author, type, date, text and replies, grouped by page. pub fn summarize_comments(&self, id: DocId, sort: SummarySort) -> Result>, EditError> { let doc = self.get(id).ok_or(EditError::NoDocument)?; + // The summary copies the comments' text out into a new document. + doc.check_copying()?; let text = comment_summary(&doc.name, &doc.info.annotations, sort); self.create_report(&format!("Summary of Comments on {}", doc.name), &text) } @@ -313,3 +373,112 @@ impl Session { &self.docs } } + +/// Convert a view-space text line rectangle into a normalized, finite PDF user-space rectangle. +fn line_user_rect(page: &pdfcraft_render::PageInfo, v: [f32; 4]) -> Option<[f64; 4]> { + if !v.iter().all(|x| x.is_finite()) { + return None; + } + let r = page.view_rect_to_user(v).map(f64::from); + r.iter().all(|x| x.is_finite()).then_some(r) +} + +#[cfg(test)] +mod tests { + use super::line_user_rect; + use crate::{Edit, Session}; + use pdfcraft_render::PageInfo; + use std::sync::Arc; + + /// Build a sample `PageInfo` with a non-origin crop box `[10.0, 20.0, 210.0, 320.0]` (`200 × 300`). + fn sample_page(rotation: u16) -> PageInfo { + let (width, height) = if rotation % 180 == 90 { (300.0, 200.0) } else { (200.0, 300.0) }; + PageInfo { width, height, label: String::new(), crop: [10.0, 20.0, 210.0, 320.0], rotation } + } + + #[test] + fn line_user_rect_rejects_non_finite_coordinates() { + let page = sample_page(0); + assert_eq!(line_user_rect(&page, [f32::NAN; 4]), None); + assert_eq!(line_user_rect(&page, [f32::NAN, 10.0, 50.0, 20.0]), None); + assert_eq!(line_user_rect(&page, [10.0, 20.0, f32::INFINITY, 30.0]), None); + assert_eq!(line_user_rect(&page, [10.0, f32::NEG_INFINITY, 50.0, 30.0]), None); + + for bad_crop_box in [[f32::NAN, 0.0, 200.0, 300.0], [0.0, f32::INFINITY, 200.0, 300.0], [0.0, 0.0, f32::NEG_INFINITY, 300.0]] { + let mut bad_crop = sample_page(0); + bad_crop.crop = bad_crop_box; + assert_eq!(line_user_rect(&bad_crop, [10.0, 20.0, 50.0, 40.0]), None); + } + + // Finite inputs whose crop span overflows f32 during view-to-user mapping must also be rejected. + let overflow_page = PageInfo { width: 1.0, height: 1.0, label: String::new(), crop: [-f32::MAX, 0.0, f32::MAX, 300.0], rotation: 0 }; + assert_eq!(line_user_rect(&overflow_page, [1.0, 10.0, 2.0, 20.0]), None); + } + + #[test] + fn line_user_rect_matches_view_rect_to_user_across_rotations() { + let v = [12.0, 24.0, 96.0, 48.0]; + let cases = [ + (0, [22.0, 272.0, 106.0, 296.0]), + (90, [34.0, 32.0, 58.0, 116.0]), + (180, [114.0, 44.0, 198.0, 68.0]), + (270, [162.0, 224.0, 186.0, 308.0]), + ]; + for (rotation, expected) in cases { + let page = sample_page(rotation); + let got = line_user_rect(&page, v).expect("finite view rect maps to user space"); + for (g, e) in got.into_iter().zip(expected) { + assert!((g - e).abs() < 1e-4, "rotation {rotation}: got {got:?}, expected {expected:?}"); + } + } + } + + #[test] + fn find_urls_returns_finite_user_rects_on_rotated_page() { + let mut objs: Vec> = vec![ + b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(), + b"<< /Type /Pages /Kids [4 0 R] /Count 1 /MediaBox [0 0 200 300] >>".to_vec(), + b"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>".to_vec(), + b"<< /Type /Page /Parent 2 0 R /Contents 5 0 R /Resources << /Font << /F1 3 0 R >> >> >>".to_vec(), + ]; + let body = "BT /F1 12 Tf 20 150 Td (Visit https://example.com today) Tj ET"; + objs.push(format!("<< /Length {} >>\nstream\n{body}\nendstream", body.len()).into_bytes()); + let mut pdf = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offsets.push(pdf.len()); + pdf.extend_from_slice(format!("{} 0 obj\n", i + 1).as_bytes()); + pdf.extend_from_slice(o); + pdf.extend_from_slice(b"\nendobj\n"); + } + let xref = pdf.len(); + pdf.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offsets { + pdf.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + pdf.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes()); + + let mut session = Session::new(); + let id = session.open("urls.pdf", None, Arc::new(pdf), None).expect("opens"); + let expected_rect = [46.004, 147.6, 153.38, 159.6]; + let base_urls = session.find_urls(id); + assert_eq!(base_urls.len(), 1); + let base_rect = base_urls[0].1[0]; + for (got, exp) in base_rect.into_iter().zip(expected_rect) { + assert!((got - exp).abs() < 1e-2, "0 deg: got {base_rect:?}, expected {expected_rect:?}"); + } + + for step in [90, 180, 270] { + session.apply(id, Edit::RotatePages { pages: vec![0], degrees: 90 }).expect("rotates"); + let urls = session.find_urls(id); + assert_eq!(urls.len(), 1); + assert_eq!(urls[0].0, 0); + assert_eq!(urls[0].2, "https://example.com"); + assert_eq!(urls[0].1.len(), 1); + let r = urls[0].1[0]; + for (got, exp) in r.into_iter().zip(expected_rect) { + assert!((got - exp).abs() < 1e-2, "rotation {step} deg: got {r:?}, expected {expected_rect:?}"); + } + } + } +} diff --git a/crates/engine/src/signature_image.rs b/crates/engine/src/signature_image.rs index 074adfc82d..198904850d 100644 --- a/crates/engine/src/signature_image.rs +++ b/crates/engine/src/signature_image.rs @@ -241,6 +241,7 @@ impl SignatureImage { name: if initials { "Initials" } else { "Signature" }.into(), file: MarkFile { name: "signature.png".into(), bytes: self.bytes.clone(), page: 0 }, author: author.into(), + line: None, }], }) } diff --git a/crates/engine/src/test_hooks.rs b/crates/engine/src/test_hooks.rs index d6e504599e..1f6bd94822 100644 --- a/crates/engine/src/test_hooks.rs +++ b/crates/engine/src/test_hooks.rs @@ -40,3 +40,17 @@ pub(crate) fn record_link_scan() { counts.set((markers, links + 1)); }); } + +thread_local! { + static FULL_REFRESHES: std::cell::Cell = const { std::cell::Cell::new(0) }; +} + +/// A whole-document refresh (re-inspection) ran on this thread. +pub(crate) fn record_full_refresh() { + FULL_REFRESHES.with(|n| n.set(n.get() + 1)); +} + +/// How many whole-document refreshes ran on this thread since the last call, and starts over. +pub(crate) fn take_full_refreshes() -> usize { + FULL_REFRESHES.with(|n| n.replace(0)) +} diff --git a/crates/engine/src/tests.rs b/crates/engine/src/tests.rs index 4df8608946..3912bfd5ec 100644 --- a/crates/engine/src/tests.rs +++ b/crates/engine/src/tests.rs @@ -106,6 +106,7 @@ fn custom_image_stamp_keeps_displayed_orientation_on_rotated_pages() { name: "Original quadrants".into(), file: MarkFile { name: "quadrants.png".into(), bytes: bytes.clone(), page: 0 }, author: "Test".into(), + line: None, }, ) .unwrap(); @@ -441,6 +442,7 @@ fn explicit_image_stamps_keep_displayed_orientation_on_rotated_pages() { name: "Original quadrants".into(), file: MarkFile { name: "quadrants.png".into(), bytes: bytes.clone(), page: 0 }, author: "Test".into(), + line: None, }, ) .unwrap(); @@ -493,6 +495,7 @@ fn pdf_page_stamps_keep_displayed_orientation_on_rotated_pages() { name: "Half red".into(), file: MarkFile { name: "half.pdf".into(), bytes: bytes.clone(), page: 0 }, author: "Test".into(), + line: None, }, ) .unwrap(); @@ -522,6 +525,7 @@ fn pdf_page_stamps_fit_inside_an_explicit_box() { name: "Diagram".into(), file: MarkFile { name: "diagram.pdf".into(), bytes: bytes.clone(), page: 0 }, author: "Test".into(), + line: None, }; session.apply(id, stamp([56.0, 400.0, 539.0, 700.0])).unwrap(); assert_eq!(session.get(id).unwrap().info.annotations[0].rect, [56.0, 429.25, 539.0, 670.75]); @@ -1420,6 +1424,35 @@ fn utf16_and_code_page_text_files_convert_to_their_text() { assert_eq!(page_texts(&s, id), ["Crème brûlée"]); } +/// Create ▸ PDF from a text file in Cyrillic was refused, and a comment summary quoting Cyrillic +/// wrote each letter as `[U+XXXX]`. Both are set in the bundled Inter face now and read back as +/// typed. +#[test] +fn cyrillic_text_files_and_comment_summaries_read_back_as_typed() { + let mut s = Session::new(); + let text = "Договор № 12\nЖълтата дюля беше щастлива, че пухът, който цъфна, замръзна като гьон."; + let (_, pdf) = s.convert_to_pdf("бележки.txt", &Arc::new(text.as_bytes().to_vec())).unwrap(); + let id = s.open_new("бележки.pdf", pdf).unwrap(); + assert_eq!(page_texts(&s, id), [text]); + // A sticky note in Russian, summarized. + let note = "Проверьте сумму в разделе 3"; + let hex: String = note.encode_utf16().map(|u| format!("{u:04X}")).collect(); + let pdf = format!( + "%PDF-1.7 +1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj +2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj +3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 300 200] /Annots [4 0 R] >> endobj +4 0 obj << /Type /Annot /Subtype /Text /Rect [20 140 40 160] /T (Reviewer) /Contents >> endobj +trailer << /Root 1 0 R >> +%%EOF" + ); + let id = s.open("note.pdf", None, Arc::new(pdf.into_bytes()), None).expect("opens"); + let summary = s.summarize_comments(id, SummarySort::Page).unwrap(); + let id = s.open_new("summary.pdf", summary).unwrap(); + let shown = page_texts(&s, id).join("\n"); + assert!(shown.contains(note) && !shown.contains("[U+"), "{shown}"); +} + #[test] fn a_file_split_around_another_combines_in_order_with_one_bookmark() { let mut s = Session::new(); @@ -1647,11 +1680,28 @@ fn copy_restricted_documents_refuse_extraction_unless_opened_by_the_owner() { assert_eq!(s.compare_report(oid, id).map(|_| ()), refused()); assert!(s.compare(oid, oid).is_ok()); assert_eq!(s.mark_differences(id, oid).map(|_| ()), refused()); + // The comment summary copies comment text out like the exports do. + assert_eq!(s.summarize_comments(id, SummarySort::Page).map(|_| ()), refused()); // Unencrypted documents allow everything. let (s, plain) = session_with(1); assert!(s.get(plain).unwrap().check_copying().is_ok() && extracting.iter().all(|c| enabled(&s, plain, c))); } +/// The renderer can open an encrypted file the editing parser rejects. Its permission bits are +/// then unknown, which is not the same as none: copying and printing stay refused. +#[test] +fn an_encrypted_document_without_an_editor_allows_neither_copying_nor_printing() { + let mut s = Session::new(); + let id = s.open("r.pdf", None, protected("", "owner", -1 ^ 0b1_0000), None).unwrap(); + s.doc_mut(id).unwrap().editor = None; + let d = s.get(id).unwrap(); + assert!(d.info.encrypted && d.permissions().is_none()); + assert!(!d.allows_copying() && !d.allows_printing()); + let (mut s, plain) = session_with(1); + s.doc_mut(plain).unwrap().editor = None; + assert!(s.get(plain).unwrap().allows_copying(), "an unencrypted document is not restricted"); +} + /// Table 22, bit 3 (#2669): a document that forbids printing disables Print up front instead /// of refusing once the dialog is confirmed. Low-resolution printing (bit 3 on, bit 12 off) is /// still printing, and the owner password lifts the restriction. @@ -1779,6 +1829,45 @@ fn low_resolution_printing_prints_only_the_chosen_content() { } } +/// A comment and a field the file leaves without an appearance print as the page shows them: +/// a text box (`/FreeText` with only `/Contents` and `/DA`) and a check box set on with +/// `/NeedAppearances`. The vector print drew only appearance streams, so both were on screen and +/// in Print as Image but not on paper (pdf.js `bug1865341`, `freetext_print_no_appearance`). +/// ISO 32000-2 §12.5.3: an annotation with the Print flag is printed. +#[test] +fn comments_and_fields_without_an_appearance_print_as_shown() { + let content = ""; + let pdf = format!( + "%PDF-1.7 + 1 0 obj << /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [6 0 R] /NeedAppearances true /DA (/Helv 0 Tf 0 g) >> >> endobj + 2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj + 3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Contents 4 0 R /Annots [5 0 R 6 0 R] >> endobj + 4 0 obj << /Length {} >> stream +{content} +endstream endobj + 5 0 obj << /Type /Annot /Subtype /FreeText /F 4 /P 3 0 R /Rect [10 120 190 190] /DA (/Helv 40 Tf 0 g) /Contents (WWWW) >> endobj + 6 0 obj << /Type /Annot /Subtype /Widget /FT /Btn /T (Box) /V /Yes /AS /Yes /F 4 /P 3 0 R /Rect [10 10 110 110] /MK << /CA (8) >> >> endobj + trailer << /Root 1 0 R >> +%%EOF", + content.len(), + ); + let mut s = Session::new(); + let id = s.open("bare.pdf", None, Arc::new(pdf.into_bytes()), None).unwrap(); + let settings = + print::Settings { pages: vec![0], paper: (200.0, 200.0), layout: print::Layout::Size(print::SizeMode::Actual), ..Default::default() }; + let (w, h, rgba) = printed_pixels(s.print_pdf(id, &settings).unwrap()); + assert_eq!((w, h), (200, 200)); + // Dark pixels in the text box (top half) and in the check box (bottom half). + let dark = |y0: usize, y1: usize| { + (y0..y1) + .flat_map(|y| (0..w as usize).map(move |x| (x, y))) + .filter(|&(x, y)| rgba.get((y * w as usize + x) * 4).is_some_and(|&r| r < 128)) + .count() + }; + assert!(dark(10, 80) > 50, "the text box prints"); + assert!(dark(90, 190) > 50, "the check box prints"); +} + #[test] fn the_permissions_password_unlocks_an_open_document() { // Only a permissions password (#995, #1077): opens without asking. Changes are allowed, @@ -2279,6 +2368,29 @@ fn highlight_multiplies_over_text() { assert!(bg[0] > 240 && bg[1] > 240 && bg[2] < 30, "{bg:?}"); } +/// Converting the appearances of comments and form fields needs the permission to comment; +/// converting the page alone needs only the permission to modify. +#[test] +fn converting_colours_of_comments_needs_the_comment_permission() { + let mut cos = pdfcraft_cos::Document::open(Arc::new(fixture(1))).unwrap(); + let params = pdfcraft_cos::NewEncryption { + algorithm: pdfcraft_cos::Algorithm::Aes256, + user_password: "", + owner_password: "own", + permissions: !(1 << 5), + encrypt_metadata: true, + seed: [7; 32], + }; + cos.set_encryption(¶ms).unwrap(); + let bytes = pdfcraft_cos::write_full(&cos, &pdfcraft_cos::SaveOptions::default()).unwrap(); + let mut s = Session::new(); + let id = s.open("locked.pdf", None, Arc::new(bytes), None).unwrap(); + let with = ConvertColours { annotations: true, ..ConvertColours::default() }; + assert_eq!(s.apply(id, Edit::ConvertColours { pages: vec![0], settings: with }), Err(EditError::NotPermitted("comments"))); + let without = s.apply(id, Edit::ConvertColours { pages: vec![0], settings: ConvertColours::default() }); + assert!(!matches!(without, Err(EditError::NotPermitted(_))), "{without:?}"); +} + #[test] fn comment_permission_is_enforced() { let mut cos = pdfcraft_cos::Document::open(Arc::new(fixture(1))).unwrap(); @@ -3120,6 +3232,66 @@ fn setting_the_calculation_order_recalculates_and_undoes() { assert!(matches!(s.apply(id, Edit::SetCalculationOrder { names: vec!["qty".into()] }), Err(EditError::Form(_)))); } +/// A Calculate script that sets its own field through the field object, not `event.value`, +/// gives the field that value, text included; when it sets `event.value` as well, that wins. +#[test] +fn a_calculate_script_setting_its_own_field_object_keeps_the_value() { + let (mut s, id) = session_with(1); + for (i, name) in ["origin", "copy", "both"].into_iter().enumerate() { + let y = 250.0 - 40.0 * i as f64; + let kind = NewField::Text { multiline: false }; + s.apply(id, Edit::AddField { page: 0, rect: [20.0, y, 180.0, y + 22.0], kind, name: Some(name.into()) }).unwrap(); + } + let script = |name: &str, js: &str| Edit::SetFieldScript { name: name.into(), event: "calculate".into(), script: Some(js.into()) }; + s.apply(id, script("copy", "var o = this.getField(\"origin\");\nthis.getField(\"copy\").value = o.value;")).unwrap(); + s.apply(id, script("both", "this.getField(\"both\").value = \"field\";\nevent.value = this.getField(\"origin\").value + \"!\";")).unwrap(); + s.apply(id, Edit::SetFieldValue { name: "origin".into(), value: FieldValue::Text("abc".into()) }).unwrap(); + let value = |s: &Session, n: &str| s.get(id).unwrap().form.iter().find(|f| f.name == n).and_then(|f| f.value.first().cloned()); + assert_eq!(value(&s, "copy").as_deref(), Some("abc")); + assert_eq!(value(&s, "both").as_deref(), Some("abc!"), "event.value decides when the script sets both"); + s.apply(id, Edit::SetFieldValue { name: "origin".into(), value: FieldValue::Text("12".into()) }).unwrap(); + assert_eq!(value(&s, "copy").as_deref(), Some("12")); +} + +/// A Calculate script that assigns `event.value` the value its field already holds keeps it, even +/// after writing another value through its field object; and a combo box a script moves to +/// another option selects that option in `/I` too. +#[test] +fn a_calculate_script_assigning_event_value_wins_and_a_combo_keeps_its_selection() { + let (mut s, id) = session_with(1); + s.apply( + id, + Edit::AddField { page: 0, rect: [20.0, 250.0, 180.0, 272.0], kind: NewField::Text { multiline: false }, name: Some("origin".into()) }, + ) + .unwrap(); + s.apply(id, Edit::AddField { page: 0, rect: [20.0, 210.0, 180.0, 232.0], kind: NewField::Text { multiline: false }, name: Some("kept".into()) }) + .unwrap(); + let kind = NewField::Combo { options: vec!["Small".into(), "Large".into()], editable: false }; + s.apply(id, Edit::AddField { page: 0, rect: [20.0, 170.0, 180.0, 192.0], kind, name: Some("size".into()) }).unwrap(); + s.apply(id, Edit::SetFieldValue { name: "size".into(), value: FieldValue::Choice(vec!["Small".into()]) }).unwrap(); + let script = |name: &str, js: &str| Edit::SetFieldScript { name: name.into(), event: "calculate".into(), script: Some(js.into()) }; + s.apply( + id, + script( + "kept", + "var v = event.value; +this.getField(\"kept\").value = \"field\"; +event.value = v;", + ), + ) + .unwrap(); + s.apply(id, script("size", "event.value = this.getField(\"origin\").value > 5 ? \"Large\" : \"Small\";")).unwrap(); + s.apply(id, Edit::SetFieldValue { name: "origin".into(), value: FieldValue::Text("9".into()) }).unwrap(); + assert!(s.take_js_output(id).errors.is_empty(), "both scripts ran to the end"); + let doc = s.get(id).unwrap(); + let value = |n: &str| doc.form.iter().find(|f| f.name == n).and_then(|f| f.value.first().cloned()); + assert_eq!(value("kept").as_deref(), None, "event.value, set to the field's own value, decides"); + assert_eq!(value("size").as_deref(), Some("Large")); + let size = doc.form.iter().find(|f| f.name == "size").map(|f| f.obj).unwrap(); + let selected = doc.editor.as_ref().unwrap().cos.get(size).as_dict().and_then(|d| d.get(b"I")).cloned(); + assert_eq!(selected, Some(pdfcraft_cos::Object::Array(vec![pdfcraft_cos::Object::Int(1)]))); +} + /// One 612×300 page with the content stream `body`. Object 4 is Helvetica; `resources` is what /// the page's /Resources holds. fn one_line_page(resources: &str, body: &str) -> Vec { @@ -3135,19 +3307,7 @@ fn one_line_page_with(resources: &str, stream_entries: &str, body: &str) -> Vec< "<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>".to_string(), format!("<< /Length {} {stream_entries} >>\nstream\n{body}\nendstream", body.len()), ]; - let mut out = b"%PDF-1.7\n".to_vec(); - let mut offsets = Vec::new(); - for (i, o) in objs.iter().enumerate() { - offsets.push(out.len()); - out.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes()); - } - let xref = out.len(); - out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); - for o in offsets { - out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); - } - out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes()); - out + pdf_of(&objs) } /// Put a redaction mark over `needle` where page 1 shows it, as Find text and redact does. @@ -3214,6 +3374,55 @@ fn text_after_an_inline_image_is_redacted_where_the_page_shows_it() { ); } +/// #2539 review: Apply redactions tells the caller where the removed text is still held that +/// it leaves as it is (here a bookmark title), and the next edit, or an undo, forgets it. +#[test] +fn applied_redactions_report_a_bookmark_that_still_holds_the_removed_text() { + let mut s = Session::new().with_clock(|| 1_700_000_000); + let page = one_line_page("/Font << /F1 4 0 R >>", "BT /F1 10 Tf 20 150 Td (Codename BLUEFIN report) Tj ET"); + let id = s.open("bookmark.pdf", None, Arc::new(page), None).unwrap(); + s.apply(id, Edit::AddBookmark { parent: vec![], index: 0, title: "Bluefin plans".into(), page: 0, left: None, top: None }).unwrap(); + s.apply(id, Edit::AddBookmark { parent: vec![], index: 1, title: "Annual figures".into(), page: 0, left: None, top: None }).unwrap(); + mark_shown_text(&mut s, id, "BLUEFIN"); + assert_eq!(s.get(id).unwrap().redaction_elsewhere(), None, "only applying redactions reports it"); + s.apply(id, Edit::ApplyRedactions { pages: None }).unwrap(); + let found = s.get(id).unwrap().redaction_elsewhere().expect("applying reports what it left"); + assert_eq!(found, crate::RedactionElsewhere { bookmarks: 1, ..Default::default() }); + // One of the edit notices, so tools and the interface say it with the others. + let said: Vec = s.get(id).unwrap().edit_notices.iter().map(ToString::to_string).collect(); + assert_eq!(said, ["the redacted text also appears in 1 bookmark title, which redaction doesn't change (Remove hidden information removes them)"]); + assert_eq!(outline_titles(&s.get(id).unwrap().info.outline), ["Bluefin plans→1", "Annual figures→1"], "the bookmark is reported, not changed"); + s.undo(id).unwrap(); + assert_eq!(s.get(id).unwrap().redaction_elsewhere(), None, "an undo forgets it"); +} + +/// #2539 review: applying redactions that remove the document's XMP packet, which named the +/// redacted word and said the file conforms to PDF/A-2b, writes the identification back and +/// says so; tools list it with the other edit notices, and an undo forgets it. +#[test] +fn applied_redactions_report_the_pdfa_identification_written_back() { + let xmp = r#"Bluefin report"#; + let body = "BT /F1 10 Tf 20 150 Td (Codename BLUEFIN report) Tj ET"; + let objs = [ + "<< /Type /Catalog /Pages 2 0 R /Metadata 6 0 R >>".to_string(), + "<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 300] >>".to_string(), + "<< /Type /Page /Parent 2 0 R /Resources << /Font << /F1 4 0 R >> >> /Contents 5 0 R >>".to_string(), + "<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>".to_string(), + format!("<< /Length {} >>\nstream\n{body}\nendstream", body.len()), + format!("<< /Type /Metadata /Subtype /XML /Length {} >>\nstream\n{xmp}\nendstream", xmp.len()), + ]; + let mut s = Session::new().with_clock(|| 1_700_000_000); + let id = s.open("pdfa.pdf", None, Arc::new(pdf_of(&objs)), None).unwrap(); + mark_shown_text(&mut s, id, "BLUEFIN"); + s.apply(id, Edit::ApplyRedactions { pages: None }).unwrap(); + let kept = crate::RedactionIdentification { pdfa: crate::RedactionClaim::Kept, pdfua: crate::RedactionClaim::None }; + assert_eq!(s.get(id).unwrap().redaction_identification(), Some(kept)); + let said: Vec = s.get(id).unwrap().edit_notices.iter().map(ToString::to_string).collect(); + assert_eq!(said, ["the document's XMP metadata held the redacted text and was removed; its PDF/A identification was written back on its own"]); + s.undo(id).unwrap(); + assert_eq!(s.get(id).unwrap().redaction_identification(), None, "an undo forgets it"); +} + /// A word whose font and size an ExtGState /Font entry (`gs`) sets is placed where the renderer /// draws it, so a mark drawn over it as displayed takes it out of the file. #[test] @@ -3441,6 +3650,185 @@ fn printing_keeps_hidden_layers_hidden() { assert!(red > 100, "shown again, Layer A prints again: {red} red"); } +/// A PDF of `objs`, numbered from 1, the first being the catalog. +fn pdf_of_objects(objs: &[Vec]) -> Vec { + let mut pdf = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offsets.push(pdf.len()); + pdf.extend_from_slice(format!("{} 0 obj\n", i + 1).as_bytes()); + pdf.extend_from_slice(o); + pdf.extend_from_slice(b"\nendobj\n"); + } + let xref = pdf.len(); + pdf.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offsets { + pdf.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + pdf.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes()); + pdf +} + +/// ISO 32000-2 §8.11.4.4: the default configuration's auto state for the Print event sets the +/// layers it names when the document prints. Red is on screen but `/PrintState /OFF`, green is off +/// on screen but `/PrintState /ON`: on paper, vector or as an image, green prints and red doesn't. +/// Two blue squares are on screen and `/PrintState /OFF` too, but no Print recommendation reaches +/// them (one is named only in the View entry, the other in a Print entry without the Print +/// category), so they print as shown. +#[test] +fn printing_applies_the_print_auto_state() { + let content = b"/OC /A BDC 1 0 0 rg 10 10 80 80 re f EMC /OC /B BDC 0 1 0 rg 110 10 80 80 re f EMC \ + /OC /C BDC 0 0 1 rg 210 10 40 40 re f EMC /OC /D BDC 0 0 1 rg 210 55 40 40 re f EMC"; + let screen_only = b"/Usage << /View << /ViewState /ON >> /Print << /PrintState /OFF >> >> >>"; + let objs: Vec> = vec![ + b"<< /Type /Catalog /Pages 2 0 R /OCProperties << /OCGs [5 0 R 6 0 R 7 0 R 8 0 R] /D << /OFF [6 0 R] \ + /AS [<< /Event /Print /OCGs [5 0 R 6 0 R] /Category [/Print] >> << /Event /View /OCGs [5 0 R 6 0 R 7 0 R] /Category [/View] >> \ + << /Event /Print /OCGs [8 0 R] /Category [/Language] >>] >> >> >>" + .to_vec(), + b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(), + b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 100] /Contents 4 0 R \ + /Resources << /Properties << /A 5 0 R /B 6 0 R /C 7 0 R /D 8 0 R >> >> >>" + .to_vec(), + [format!("<< /Length {} >>\nstream\n", content.len()).as_bytes(), content, b"\nendstream"].concat(), + [b"<< /Type /OCG /Name (View only) ".as_slice(), screen_only].concat(), + b"<< /Type /OCG /Name (Print only) /Usage << /View << /ViewState /OFF >> /Print << /PrintState /ON >> >> >>".to_vec(), + [b"<< /Type /OCG /Name (View entry) ".as_slice(), screen_only].concat(), + [b"<< /Type /OCG /Name (Other category) ".as_slice(), screen_only].concat(), + ]; + let pdf = pdf_of_objects(&objs); + + let colours = |page: &pdfcraft_render::RenderedPage| { + let px = page.rgba.as_chunks::<4>().0; + let red = px.iter().filter(|p| p[0] > 200 && p[1] < 60 && p[2] < 60).count(); + let green = px.iter().filter(|p| p[1] > 200 && p[0] < 60 && p[2] < 60).count(); + let blue = px.iter().filter(|p| p[2] > 200 && p[0] < 60 && p[1] < 60).count(); + (red, green, blue) + }; + let mut s = Session::new(); + let id = s.open("auto-state.pdf", None, Arc::new(pdf), None).unwrap(); + let (red, green, shown_blue) = colours(&render_page(&s, id)); + assert!(red > 100 && green == 0, "on screen, red shows and green doesn't: {red} red, {green} green"); + // The print dialog's preview shows the page as it prints, the view as it shows; so after a + // Layers panel change too. + let preview = |s: &Session, tag: u64| { + let doc = s.get(id).unwrap(); + let mut renderer = pdfcraft_render::PageRenderer::new(doc.display.clone(), display_config(&doc.config)); + colours(&renderer.render(pdfcraft_render::RenderRequest { page: 0, scale: 2.0, tag, ..Default::default() })) + }; + let view_entry = s.get(id).unwrap().info.layers.iter().position(|l| l.name == "View entry").unwrap(); + let mut printed_blue = Vec::new(); + for _ in 0..2 { + let (red, green, blue) = preview(&s, pdfcraft_render::PRINT_TAG); + assert!(red == 0 && green > 100, "the preview shows the print: {red} red, {green} green"); + printed_blue.push(blue); + let (red, green, _) = preview(&s, 0); + assert!(red > 100 && green == 0, "the view is unchanged: {red} red, {green} green"); + // Hidden after the first round, shown again after the second. + assert!(s.set_layer_visible(id, view_entry, printed_blue.len() == 2)); + } + // Hidden in the panel, View entry (no Print recommendation) leaves the preview too. + assert!(printed_blue[1] * 4 < printed_blue[0] * 3, "the preview follows the panel: {printed_blue:?} blue"); + for raster in [None, Some(print::Raster { dpi: 72.0, grayscale: false })] { + let bytes = Arc::new(s.print_pdf(id, &print::Settings { pages: vec![0], raster, ..Default::default() }).unwrap()); + let mut out = Session::new(); + let printed = out.open("print.pdf", None, bytes.clone(), None).unwrap(); + let (red, green, blue) = colours(&render_page(&out, printed)); + assert_eq!(red, 0, "the view-only layer doesn't print (as image: {})", raster.is_some()); + assert!(green > 100, "the print-only layer prints (as image: {}): {green} green", raster.is_some()); + // Both blue squares print: the one shown on screen alone is half of them. + assert!( + blue * 4 > shown_blue * 3, + "the layers no Print recommendation reaches print as shown (as image: {}): {blue} of {shown_blue} blue", + raster.is_some() + ); + // The vector copy's states are final: nothing a reader applies when drawing is left to + // turn them back (the auto state, or a layer's `/Usage /View /ViewState`). + let layers = &out.get(printed).unwrap().info.layers; + if raster.is_none() { + let cos = pdfcraft_cos::Document::open(bytes).unwrap(); + assert_eq!(layers.len(), 4); + for layer in layers { + let group = cos.get(pdfcraft_cos::ObjRef::new(layer.id.0, layer.id.1)); + assert!(group.as_dict().is_some_and(|g| !g.contains(b"Usage")), "{} keeps its /Usage", layer.name); + } + let root = cos.trailer().get(b"Root").map(|r| cos.resolve(r)).unwrap(); + let properties = root.as_dict().and_then(|r| r.get(b"OCProperties")).map(|p| cos.resolve(p)).unwrap(); + let config = properties.as_dict().and_then(|p| p.get(b"D")).map(|d| cos.resolve(d)).unwrap(); + assert!(config.as_dict().is_some_and(|d| !d.contains(b"AS")), "the print copy keeps /AS"); + } + } +} + +/// The print auto state of `layers` (object numbers from 5) in a document whose default +/// configuration has `auto` as its `/AS` and `groups` as the layers' dictionaries. +fn print_states_of(auto: &str, groups: &[&str]) -> Vec { + let ocgs: String = (0..groups.len()).map(|i| format!("{} 0 R ", i + 5)).collect(); + let mut objs: Vec> = vec![ + format!("<< /Type /Catalog /Pages 2 0 R /OCProperties << /OCGs [{ocgs}] /D << /AS {auto} >> >> >>").into_bytes(), + b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(), + b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 100 100] /Contents 4 0 R >>".to_vec(), + b"<< /Length 0 >>\nstream\n\nendstream".to_vec(), + ]; + objs.extend(groups.iter().map(|g| g.as_bytes().to_vec())); + let cos = pdfcraft_cos::Document::open(Arc::new(pdf_of_objects(&objs))).unwrap(); + let layers: Vec = (0..groups.len()).map(|i| Layer { id: (i as u32 + 5, 0), name: format!("{i}"), visible: true, initial: true }).collect(); + crate::print_layer_states(&cos, &layers).expect("there are Print entries").iter().map(|l| l.visible).collect() +} + +/// ISO 32000-2 §8.11.4.4: a group prints only if every category of every Print entry naming it +/// recommends ON. A Print category (`/PrintState /ON`) and a View category of another Print entry +/// (`/ViewState /OFF`) disagree, so the group stays off; with both ON it prints. A `/Category` +/// written as a single name (Table 101 asks for an array) is read as one. +#[test] +fn a_layer_prints_only_if_every_recommendation_is_on() { + let auto = "[<< /Event /Print /OCGs [5 0 R 6 0 R] /Category [/Print] >> << /Event /Print /OCGs [5 0 R 6 0 R] /Category [/View] >> \ + << /Event /Print /OCGs [7 0 R] /Category /Print >>]"; + let states = print_states_of( + auto, + &[ + "<< /Type /OCG /Name (Disagree) /Usage << /View << /ViewState /OFF >> /Print << /PrintState /ON >> >> >>", + "<< /Type /OCG /Name (Agree) /Usage << /View << /ViewState /ON >> /Print << /PrintState /ON >> >> >>", + "<< /Type /OCG /Name (Single name) /Usage << /Print << /PrintState /OFF >> >> >>", + ], + ); + assert_eq!(states, [false, true, false]); +} + +/// At most 1,024 `/AS` entries are read: an entry past them that would turn the layer off is not +/// reached, while the same entry within them is. +#[test] +fn the_print_auto_state_reads_a_bounded_number_of_entries() { + let layer = "<< /Type /OCG /Name (Layer) /Usage << /Print << /PrintState /OFF >> >> >>"; + let other = "<< /Type /OCG /Name (Other) >>"; + let auto = |before: usize| { + let filler = "<< /Event /Print /OCGs [6 0 R] /Category [/Print] >> ".repeat(before); + format!("[{filler} << /Event /Print /OCGs [5 0 R] /Category [/Print] >>]") + }; + assert_eq!(print_states_of(&auto(1023), &[layer, other]), [false, true], "the 1,024th entry is read"); + assert_eq!(print_states_of(&auto(1024), &[layer, other]), [true, true], "the 1,025th entry is not"); +} + +/// Reading the print auto state is bounded over all `/AS` entries together: 1,024 Print entries +/// sharing one `/Category` array of 70,000 names, `/Print` last, read 65,536 names in all, find no +/// Print category and recommend nothing, instead of reading the array 1,024 times. +#[test] +fn the_print_auto_state_reads_a_bounded_number_of_categories() { + let names = format!("[{} /Print]", "/View ".repeat(70_000)); + let entries = "<< /Event /Print /OCGs [5 0 R] /Category 6 0 R >> ".repeat(1024); + let objs: Vec> = vec![ + format!("<< /Type /Catalog /Pages 2 0 R /OCProperties << /OCGs [5 0 R] /D << /AS [{entries}] >> >> >>").into_bytes(), + b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(), + b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 100 100] /Contents 4 0 R >>".to_vec(), + b"<< /Length 0 >>\nstream\n\nendstream".to_vec(), + b"<< /Type /OCG /Name (Layer) /Usage << /Print << /PrintState /OFF >> >> >>".to_vec(), + names.into_bytes(), + ]; + let cos = pdfcraft_cos::Document::open(Arc::new(pdf_of_objects(&objs))).unwrap(); + let layer = Layer { id: (5, 0), name: "Layer".into(), visible: true, initial: true }; + let printed = crate::print_layer_states(&cos, std::slice::from_ref(&layer)).expect("there are Print entries"); + assert!(printed.iter().all(|l| l.visible), "past the bound nothing is read, so the layer prints as shown"); +} + #[test] fn printing_lays_out_sheets_that_render() { let (s, id) = session_with(5); @@ -3466,6 +3854,45 @@ fn printing_lays_out_sheets_that_render() { assert_eq!(s4.print_pdf(id4, &print::Settings { pages: vec![0], ..Default::default() }), Err(EditError::NotPermitted("printing"))); } +/// A dynamic XFA form's screen-only and print-only objects (`relevant`): on screen, the +/// screen-only field and draw show and the print-only ones don't; printed, the screen-only field +/// is left off and the print-only one is on paper. (Draws follow the layers' state on screen +/// when printed until printing applies the layers' print usage; other viewers read it from the +/// file.) +#[test] +fn xfa_screen_only_and_print_only_objects_show_and_print_where_the_form_says() { + let field = |name: &str, attrs: &str| { + format!( + r#"{name} value"# + ) + }; + let draw = |name: &str, attrs: &str| { + format!(r#"{name} text"#) + }; + let xdp = format!( + r##" +"##, + field("screenField", r#"relevant="-print""#), + field("paperField", r#"presence="invisible" relevant="+print""#), + draw("screenDraw", r#"relevant="-print""#), + draw("paperDraw", r#"presence="invisible" relevant="+print""#), + ); + let mut s = Session::new(); + let id = s.open("form.pdf", None, Arc::new(pdfcraft_xfa::fixtures::shell(&xdp)), None).expect("opens"); + let screen = page_texts(&s, id).join("\n"); + assert!(screen.contains("screenField value") && screen.contains("screenDraw text"), "on screen: {screen}"); + assert!(!screen.contains("paperField value") && !screen.contains("paperDraw text"), "on screen: {screen}"); + let sheet = s.print_pdf(id, &print::Settings { pages: vec![0], ..Default::default() }).unwrap(); + let mut out = Session::new(); + let printed = out.open("print.pdf", None, Arc::new(sheet), None).unwrap(); + let paper = page_texts(&out, printed).join("\n"); + assert!(!paper.contains("screenField value"), "printed: {paper}"); + assert!(paper.contains("paperField value"), "printed: {paper}"); +} + /// A one-page document whose red square is in a layer; `off` switches the layer off by default. fn red_layer_pdf(off: bool) -> Vec { let content = "/OC /L1 BDC 1 0 0 rg 0 0 100 100 re f EMC"; @@ -3904,7 +4331,9 @@ fn an_optimize_job_reports_its_stages_and_can_be_cancelled() { }) .unwrap(); assert!(bytes.starts_with(b"%PDF-")); - assert_eq!(stages.first(), Some(&OptimizeStage::Discarding)); + assert_eq!(stages.first(), Some(&OptimizeStage::Reading { done: 0, total: 2 }), "it says what it does first"); + assert!(!stages.contains(&OptimizeStage::Discarding), "nothing was chosen to discard: {stages:?}"); + assert!(stages.iter().any(|st| matches!(st, OptimizeStage::Images { .. })), "{stages:?}"); assert_eq!(&stages[stages.len() - 3..], [OptimizeStage::CleaningUp, OptimizeStage::Merging, OptimizeStage::Writing]); let fractions: Vec = stages.iter().map(|st| st.fraction()).collect(); assert!(fractions.windows(2).all(|w| w[0] <= w[1]), "the bar never goes back: {fractions:?}"); @@ -3915,6 +4344,20 @@ fn an_optimize_job_reports_its_stages_and_can_be_cancelled() { "no images: no division by zero" ); + // What the bar shows before anything is reported says nothing of user data either. + assert_eq!(OptimizeStage::default(), OptimizeStage::Reading { done: 0, total: 0 }); + assert_eq!(OptimizeStage::default().fraction(), 0.0); + assert!(OptimizeStage::Reading { done: 2, total: 4 }.fraction() > 0.0, "reading pages moves the bar"); + + // Discarding says so when something is. This document has no metadata, which only stops it after + // the stage is reported. + let mut with_discard = Vec::new(); + let _ = s.optimize_job(id, &settings, &[crate::Hidden::Metadata]).unwrap().run(|st| { + with_discard.push(st); + true + }); + assert_eq!(with_discard.first(), Some(&OptimizeStage::Discarding)); + // Cancelled at the merge: nothing is written, the open document is untouched. let before = s.get(id).unwrap().bytes.clone(); let r = s.optimize_job(id, &settings, &[]).unwrap().run(|st| st != OptimizeStage::Merging); @@ -4593,6 +5036,25 @@ fn split_by_size_and_bookmarks_and_page_filters() { assert_eq!(filter_pages(info, &all, PageParity::Odd, PageOrientation::Portrait), [0, 2, 4]); } +/// A zoomed link follows its target page, not its number: deleting a page before the target keeps +/// the link on that page at its zoom. Deleting the target page drops the link's destination, as +/// for any link to a deleted page, and undo brings it back with its zoom. +#[test] +fn a_zoomed_link_follows_its_page_and_loses_it_when_the_page_is_deleted() { + let (mut s, id) = session_with(3); + let zoom = LinkZoom::Percent(200.0); + let rect = [10.0, 10.0, 60.0, 30.0]; + s.apply(id, Edit::AddLink { page: 0, rect, action: LinkAction::Page { page: 2, zoom }, style: LinkStyle::default() }).unwrap(); + let action = |s: &Session| s.get(id).unwrap().links()[0].action.clone(); + s.apply(id, Edit::DeletePages { pages: vec![1] }).unwrap(); + assert_eq!(action(&s), LinkAction::Page { page: 1, zoom }, "the link follows its page"); + s.apply(id, Edit::DeletePages { pages: vec![1] }).unwrap(); + // The destination is removed, not left pointing at a page that is gone. + assert_eq!(action(&s), LinkAction::Other("no action".into()), "no destination left"); + s.undo(id).unwrap(); + assert_eq!(action(&s), LinkAction::Page { page: 1, zoom }, "undo restores the destination and its zoom"); +} + #[test] fn links_from_urls_and_link_edits() { // One page whose text contains a web address. @@ -4634,8 +5096,12 @@ fn links_from_urls_and_link_edits() { assert_eq!(s.get(id).unwrap().links().len(), 1); // Link Properties and delete. let l = s.get(id).unwrap().links()[0].clone(); - s.apply(id, Edit::SetLink { page: 0, index: l.index, rect: None, action: Some(LinkAction::Page(0)), style: None }).unwrap(); - assert_eq!(s.get(id).unwrap().links()[0].action, LinkAction::Page(0)); + s.apply( + id, + Edit::SetLink { page: 0, index: l.index, rect: None, action: Some(LinkAction::Page { page: 0, zoom: LinkZoom::FitPage }), style: None }, + ) + .unwrap(); + assert_eq!(s.get(id).unwrap().links()[0].action, LinkAction::Page { page: 0, zoom: LinkZoom::FitPage }); s.apply(id, Edit::DeleteLink { page: 0, index: l.index }).unwrap(); assert!(s.get(id).unwrap().links().is_empty()); assert!(s.apply(id, Edit::RemoveLinks { pages: None }).is_err(), "nothing left to remove"); @@ -4801,6 +5267,85 @@ fn comment_summary_includes_replies_to_replies() { assert!(text.contains(" NESTED_REPLY"), "the nested reply is indented under its parent: {text}"); } +/// ISO 32000-2 §12.5.6.2: a group subordinate (`/RT /Group`) is part of its primary's comment, +/// not a reply: the summary leaves it out and writes the replies to it as the comment's. +#[test] +fn comment_summary_leaves_group_members_out_and_keeps_their_replies() { + let (mut s, id) = session_with(1); + s.apply(id, rect_comment(0, [40.0, 40.0, 90.0, 90.0])).unwrap(); + s.apply(id, Edit::ReplyToAnnotation { page: 0, index: 0, text: "MEMBER_TEXT".into(), author: "Bob".into() }).unwrap(); + s.apply(id, Edit::ReplyToAnnotation { page: 0, index: 1, text: "REPLY_TO_MEMBER".into(), author: "Carol".into() }).unwrap(); + let mut all = s.get(id).unwrap().info.annotations.clone(); + all.iter_mut().find(|a| a.contents.as_deref() == Some("MEMBER_TEXT")).unwrap().grouped = true; + let text = comment_summary("doc.pdf", &all, SummarySort::Page); + assert!(!text.contains("MEMBER_TEXT") && !text.contains("Bob"), "{text}"); + assert!(text.lines().any(|l| l == " REPLY_TO_MEMBER"), "the reply is the comment's, one level in: {text}"); +} + +/// The summary follows `/IRT` down from a comment at most 64 steps, group subordinates included: +/// a reply under a longer chain of subordinates is left out, one under a short chain is written, +/// and a subordinate listed at its own primary's position (a loop, from a malformed listing) +/// ends the walk. +#[test] +fn comment_summary_follows_a_group_chain_a_bounded_number_of_steps() { + let (mut s, id) = session_with(1); + s.apply(id, rect_comment(0, [40.0, 40.0, 90.0, 90.0])).unwrap(); + let top = s.get(id).unwrap().info.annotations.iter().find(|a| a.is_comment()).unwrap().clone(); + let at = |a: &pdfcraft_render::Annotation| (a.page, a.index); + let under = |index: usize, parent: (usize, usize), grouped: bool, text: &str| pdfcraft_render::Annotation { + index, + parent: Some(parent), + grouped, + contents: Some(text.into()), + ..top.clone() + }; + for (members, written) in [(10, true), (100, false)] { + let mut all = vec![top.clone()]; + let mut up = at(&top); + for k in 0..members { + let m = under(1000 + k, up, true, "MEMBER"); + up = at(&m); + all.push(m); + } + all.push(under(5000, up, false, "DEEP_REPLY")); + let text = comment_summary("doc.pdf", &all, SummarySort::Page); + assert_eq!(text.contains("DEEP_REPLY"), written, "{members}: {text}"); + assert!(!text.contains("MEMBER"), "{members}: {text}"); + } + let looped = vec![top.clone(), under(top.index, at(&top), true, "LOOP")]; + let text = comment_summary("doc.pdf", &looped, SummarySort::Page); + assert!(!text.contains("LOOP"), "{text}"); +} + +/// Deleting a group's primary removes the subordinate's `/IRT` in the same edit, as Acrobat does: +/// the subordinate lists as a comment of its own and keeps `/RT /Group`. One undo puts the group +/// back, `/IRT` included. +#[test] +fn deleting_a_group_primary_releases_its_member_in_one_undo_step() { + let bytes = pdf_with_annots(&[ + b"<< /Type /Annot /Subtype /Square /Rect [10 10 50 50] /NM (primary) /Contents (PRIMARY_TEXT) >>", + b"<< /Type /Annot /Subtype /Circle /Rect [60 60 90 90] /NM (member) /IRT 4 0 R /RT /Group >>", + ]); + let mut s = Session::new().with_clock(|| 1_700_000_000); + let id = s.open("group.pdf", None, Arc::new(bytes), None).unwrap(); + let names = |s: &Session| -> Vec<(Option, bool, bool)> { + s.get(id).unwrap().info.annotations.iter().map(|a| (a.name.clone(), a.is_comment(), a.is_group_member())).collect() + }; + // Listed by top edge: the Circle is the higher one. + let grouped = vec![(Some("member".into()), false, true), (Some("primary".into()), true, false)]; + assert_eq!(names(&s), grouped); + s.apply(id, Edit::DeleteAnnotation { page: 0, index: 0 }).unwrap(); + assert_eq!(names(&s), [(Some("member".into()), true, false)], "a comment of its own"); + let member = |s: &Session| { + let cos = &s.get(id).unwrap().editor.as_ref().unwrap().cos; + cos.get(pdfcraft_cos::ObjRef::new(5, 0)).as_dict().cloned().unwrap() + }; + assert!(member(&s).get(b"IRT").is_none() && member(&s).name(b"RT") == Some(b"Group".as_slice()), "{:?}", member(&s)); + s.undo(id).unwrap(); + assert_eq!(names(&s), grouped, "one undo puts the group back"); + assert!(member(&s).reference(b"IRT").is_some()); +} + #[test] fn reading_colors_set_and_invalidated() { let (mut s, id) = session_with(1); @@ -5016,6 +5561,7 @@ fn sign_prepared_with( v.set(b"Contents".to_vec(), Object::String(PdfString { bytes: vec![0; reserve], hex: true })); v.set(b"M".to_vec(), PdfString::literal(date.as_bytes().to_vec())); value(&mut v); + let certifies = v.get(b"Reference").is_some(); let sig = doc.add(Object::Dict(v)); // The field by name, never by number: a full save (encryption, repairs) is // free to renumber objects, and a hardcoded reference would stamp the value @@ -5025,6 +5571,12 @@ fn sign_prepared_with( let fields = doc.resolve(&af).as_dict().expect("AcroForm dict").get(b"Fields").cloned().expect("Fields"); let field = doc.resolve(&fields).as_array().expect("Fields array").iter().filter_map(|o| o.as_ref()).next().expect("signature field"); doc.update_dict(field, |d| d.set(b"V".to_vec(), Object::Ref(sig))).unwrap(); + if certifies { + // A certification is the signature the catalog's /Perms /DocMDP names (ISO 32000-2, 12.8.2.2). + let mut perms = Dict::new(); + perms.set(b"DocMDP".to_vec(), Object::Ref(sig)); + doc.update_dict(root, |c| c.set(b"Perms".to_vec(), Object::Dict(perms))).unwrap(); + } let save = pdfcraft_cos::SaveOptions { object_streams: false, ..pdfcraft_cos::SaveOptions::default() }; let mut out = pdfcraft_cos::write_incremental(doc, &save).unwrap(); let mark = format!("0 {} {} {}", LAYERED_BR_MARK[0], LAYERED_BR_MARK[1], LAYERED_BR_MARK[2]); @@ -6080,6 +6632,20 @@ fn form_javascript_validates_calculates_and_formats() { assert!(s.run_javascript(id, "1", None).is_err()); } +#[test] +fn javascript_field_rect_moves_first_widget_and_undoes() { + let mut s = Session::new(); + let id = s.open("rect.pdf", None, Arc::new(scripted_form()), None).unwrap(); + let initial = s.get(id).unwrap().form.iter().find(|f| f.name == "price").unwrap().widgets[0].rect; + s.run_javascript(id, "getField('price').rect = [30, 40, 130, 60];", Some("go")).unwrap(); + assert_eq!(s.get(id).unwrap().form.iter().find(|f| f.name == "price").unwrap().widgets[0].rect, [30.0, 40.0, 130.0, 60.0]); + assert_eq!(s.get(id).unwrap().can_undo(), Some("Run JavaScript")); + s.undo(id).unwrap(); + assert_eq!(s.get(id).unwrap().form.iter().find(|f| f.name == "price").unwrap().widgets[0].rect, initial); + s.redo(id).unwrap(); + assert_eq!(s.get(id).unwrap().form.iter().find(|f| f.name == "price").unwrap().widgets[0].rect, [30.0, 40.0, 130.0, 60.0]); +} + /// XFA forms check `xfa_installed` and `xfa_version` in a document script that runs before /// each of their scripts, and alert when either is missing. #[test] @@ -6179,7 +6745,7 @@ fn inactive_xfa_and_ordinary_pdfs_keep_document_helpers_and_alerts() { let mut cos = pdfcraft_cos::Document::open(Arc::new(bytes)).unwrap(); pdfcraft_forms::set_document_script(&mut cos, "helper", Some("function twice(n) { return n * 2; } app.alert('PDF message');")).unwrap(); // The raw dynamic shell has not been laid out: XFA's presence alone is not enough. - let runner = js::JsRunner::new(&cos, "control.pdf"); + let runner = js::JsRunner::new(&cos, "control.pdf", false); let out = pdfcraft_js::run("twice(3)", &pdfcraft_js::Event::doc("Console"), &runner.doc, &[], &runner.doc_scripts, Default::default()); assert!(out.error.is_none(), "{out:?}"); assert_eq!(out.result.as_deref(), Some("6")); @@ -6675,6 +7241,8 @@ fn guard_turns_a_panic_into_an_error() { assert_eq!(guard(|| -> u8 { panic!("page {n} is bad") }), Err("page 3 is bad".to_string())); } +/// Measurement edits are single undoable steps that leave the file unchanged when they fail, and +/// respect the document's annotation permissions. #[test] fn measurement_edits_are_transactional_and_honour_annotation_permissions() { let mut session = Session::default(); @@ -6710,6 +7278,172 @@ fn measurement_edits_are_transactional_and_honour_annotation_permissions() { ); } +/// The synthetic focus form ([`pdfcraft_xfa::fixtures::focus_template`]). +fn focus_form() -> Arc> { + Arc::new(pdfcraft_xfa::fixtures::shell(&pdfcraft_xfa::fixtures::focus_template())) +} + +/// The value of field `name` in document `id`. +fn field_text(s: &Session, id: DocId, name: &str) -> String { + s.get(id).and_then(|d| d.form.iter().find(|f| f.name == name).and_then(|f| f.value.first().cloned())).unwrap_or_default() +} + +/// A field's exit scripts run every time it loses focus, changed or not, and its enter scripts +/// every time it gains it; each run that changes the form is one undo step. +#[test] +fn xfa_enter_and_exit_scripts_run_on_every_focus_change() { + use crate::js::FieldFocus::{Enter, Exit}; + let mut s = Session::new(); + let id = s.open("form.pdf", None, focus_form(), None).expect("opens"); + for _ in 0..3 { + assert!(s.field_focus(id, "probe", 0, Enter).unwrap(), "the enter script counted"); + assert!(s.field_focus(id, "probe", 0, Exit).unwrap(), "the exit script counted"); + } + assert_eq!((field_text(&s, id, "exits"), field_text(&s, id, "enters")), ("3".into(), "3".into())); + assert!(s.get(id).unwrap().dirty); + s.undo(id).unwrap(); + assert_eq!(field_text(&s, id, "exits"), "2", "each run is its own undo step"); +} + +/// A value typed into a field runs its exit with the change; losing focus then doesn't run it +/// again. The next visit runs it as usual. +#[test] +fn an_exit_the_fields_change_ran_does_not_run_twice() { + use crate::js::FieldFocus::{Enter, Exit}; + let mut s = Session::new(); + let id = s.open("form.pdf", None, focus_form(), None).expect("opens"); + s.field_focus(id, "probe", 0, Enter).unwrap(); + s.apply(id, Edit::SetFieldValue { name: "probe".into(), value: FieldValue::Text("x".into()) }).unwrap(); + assert_eq!(field_text(&s, id, "exits"), "1", "the change ran the exit"); + assert!(!s.field_focus(id, "probe", 0, Exit).unwrap(), "not again on losing focus"); + assert_eq!(field_text(&s, id, "exits"), "1"); + s.field_focus(id, "probe", 0, Enter).unwrap(); + s.field_focus(id, "probe", 0, Exit).unwrap(); + assert_eq!(field_text(&s, id, "exits"), "2", "the next visit's exit runs"); +} + +/// The skip lasts only until the next edit, undo or redo: a change that is undone, or followed by +/// another edit, doesn't make a later focus loss skip the field's exit. +#[test] +fn an_exit_skip_does_not_outlive_its_change() { + use crate::js::FieldFocus::{Enter, Exit}; + let mut s = Session::new(); + let id = s.open("form.pdf", None, focus_form(), None).expect("opens"); + s.field_focus(id, "probe", 0, Enter).unwrap(); + s.apply(id, Edit::SetFieldValue { name: "probe".into(), value: FieldValue::Text("x".into()) }).unwrap(); + s.undo(id).unwrap(); + let before = field_text(&s, id, "exits"); + s.field_focus(id, "probe", 0, Exit).unwrap(); + assert_eq!(field_text(&s, id, "exits"), (before.parse::().unwrap() + 1).to_string(), "after an undo, the exit runs"); + // A change, then an edit of another field: the change's skip is gone. + s.field_focus(id, "probe", 0, Enter).unwrap(); + s.apply(id, Edit::SetFieldValue { name: "probe".into(), value: FieldValue::Text("y".into()) }).unwrap(); + s.apply(id, Edit::SetFieldValue { name: "quiet".into(), value: FieldValue::Text("z".into()) }).unwrap(); + let before = field_text(&s, id, "exits"); + s.field_focus(id, "probe", 0, Exit).unwrap(); + assert_eq!(field_text(&s, id, "exits"), (before.parse::().unwrap() + 1).to_string(), "after another edit, the exit runs"); +} + +/// A form-script event can't be batched with other edits: its scripts and those the batch's value +/// edits run would interleave (the field's exit running twice). The batch is refused whole, and +/// the document is left as it was. +#[test] +fn a_form_script_event_in_a_batch_is_refused() { + let mut s = Session::new(); + let id = s.open("form.pdf", None, focus_form(), None).expect("opens"); + let som_of = |s: &Session, name: &str| { + let doc = s.get(id).unwrap(); + let f = doc.form.iter().find(|f| f.name == name).unwrap(); + crate::xfa::widget_som(&doc.editor.as_ref().unwrap().cos, f, 0).unwrap() + }; + let batch = |name: &str, som: String| Edit::Batch { + label: "Fill and leave".into(), + edits: vec![Edit::SetFieldValue { name: name.into(), value: FieldValue::Text("z".into()) }, Edit::XfaEvent { som, activity: "exit".into() }], + }; + // An exit that counts: it would run once for the event and once for the change. + let probe = som_of(&s, "probe"); + assert!(matches!(s.apply(id, batch("probe", probe)), Err(EditError::Invalid(_)))); + assert_eq!((field_text(&s, id, "probe"), field_text(&s, id, "exits")), (String::new(), "0".into())); + // An exit that changes nothing: the batch's value isn't kept either, nor dropped silently. + let quiet = som_of(&s, "quiet"); + assert!(s.apply(id, batch("quiet", quiet)).is_err()); + assert_eq!(field_text(&s, id, "quiet"), ""); + assert!(!s.get(id).unwrap().dirty, "nothing was applied"); +} + +/// A field's exit is noted as run (for the focus loss to skip) only when an exit script ran: +/// with every script skipped, as when the event's budget is spent or a script ran away, nothing +/// is noted and the next focus loss runs the exit. +#[test] +fn on_changes_notes_only_the_exits_that_ran() { + let mut s = Session::new(); + let id = s.open("form.pdf", None, focus_form(), None).expect("opens"); + let doc = s.get(id).unwrap(); + let mut cos = doc.editor.as_ref().unwrap().cos.clone(); + let tpl = doc.xfa_template.clone().unwrap(); + let names = vec!["probe".to_string()]; + let mut exited = Vec::new(); + let mut out = crate::js::JsOutput::default(); + let mut stopped = crate::xfa::XfaRun { page_count: 1, out: &mut out, datasets: None, ran_away: true, highlight: false, time_scale: 1 }; + crate::xfa::on_changes(&mut cos, &tpl, &names, &mut stopped, &mut exited).unwrap(); + assert!(exited.is_empty(), "no exit ran: {exited:?}"); + let mut out = crate::js::JsOutput::default(); + let mut running = crate::xfa::XfaRun { page_count: 1, out: &mut out, datasets: None, ran_away: false, highlight: false, time_scale: 1 }; + crate::xfa::on_changes(&mut cos, &tpl, &names, &mut running, &mut exited).unwrap(); + assert_eq!(exited.len(), 1, "the exit ran: {exited:?}"); +} + +/// Scripts that change nothing leave no edit: the document stays saved, with nothing to undo, +/// and what they showed still reaches the viewer. A field without such scripts runs nothing. +#[test] +fn focus_scripts_that_change_nothing_leave_the_document_clean() { + use crate::js::FieldFocus::{Enter, Exit}; + let mut s = Session::new(); + let id = s.open("form.pdf", None, focus_form(), None).expect("opens"); + s.field_focus(id, "quiet", 0, Enter).unwrap(); + assert!(!s.field_focus(id, "quiet", 0, Exit).unwrap(), "nothing changed"); + let doc = s.get(id).unwrap(); + assert!(!doc.dirty && doc.can_undo().is_none(), "no edit kept"); + let out = s.take_js_output(id); + assert_eq!(out.alerts.iter().map(|m| m.text.as_str()).collect::>(), ["left quiet"]); + // A field without focus scripts, and a field that doesn't exist. + assert!(!s.field_focus(id, "exits", 0, Exit).unwrap()); + assert!(!s.field_focus(id, "nowhere", 0, Enter).unwrap()); + assert!(!s.get(id).unwrap().dirty); +} + +/// A widget the field doesn't have gains or loses no focus: no script runs, rather than the +/// field's own. +#[test] +fn a_widget_the_field_doesnt_have_runs_no_focus_scripts() { + use crate::js::FieldFocus::{Enter, Exit}; + let mut s = Session::new(); + let id = s.open("form.pdf", None, focus_form(), None).expect("opens"); + assert!(!s.field_focus(id, "probe", 3, Enter).unwrap(), "probe has one widget"); + assert!(!s.field_focus(id, "probe", usize::MAX, Exit).unwrap()); + assert_eq!((field_text(&s, id, "enters"), field_text(&s, id, "exits")), ("0".into(), "0".into())); + assert!(!s.get(id).unwrap().dirty); +} + +/// Each radio button of a group is a field of its own: reaching the second button runs its +/// enter script, reaching the first (which has none) runs nothing. Choosing the button changes +/// the group's value, which runs the group's scripts, not the button's: leaving the button +/// still runs its exit. +#[test] +fn a_radio_buttons_own_focus_scripts_run() { + use crate::js::FieldFocus::{Enter, Exit}; + let mut s = Session::new(); + let id = s.open("form.pdf", None, focus_form(), None).expect("opens"); + let group = s.get(id).unwrap().form.iter().find(|f| f.widgets.len() == 2).map(|f| f.name.clone()).expect("the radio group"); + assert!(!s.field_focus(id, &group, 0, Enter).unwrap(), "the first button has no enter script"); + assert!(s.field_focus(id, &group, 1, Enter).unwrap(), "the second button's ran"); + assert!(!s.field_focus(id, &group, 2, Enter).unwrap(), "there is no third button"); + assert_eq!(field_text(&s, id, "picks"), "1"); + s.apply(id, Edit::SetFieldValue { name: group.clone(), value: FieldValue::Radio(Some("L".into())) }).unwrap(); + assert!(s.field_focus(id, &group, 1, Exit).unwrap(), "the button's exit runs on leaving it"); + assert_eq!(field_text(&s, id, "drops"), "1"); +} + /// A laid-out XFA field that doesn't scroll (`hScrollPolicy="off"`) opens as a text field with /// DoNotScroll, and its text box says what fits it: what the editor takes no more than. #[test] @@ -6758,6 +7492,8 @@ fn round_xfa_check_buttons_open_drawn_round() { } } +/// A dynamic XFA form with usage rights is saved without them once laid out, as the layout voids +/// them, with a note saying so. #[test] fn a_dynamic_xfa_form_is_saved_without_the_usage_rights_its_layout_voided() { let xdp = pdfcraft_xfa::fixtures::template(3); @@ -6784,6 +7520,8 @@ fn a_dynamic_xfa_form_is_saved_without_the_usage_rights_its_layout_voided() { } } +/// A dynamic XFA form is laid out from its template on opening (an appended revision, not an +/// edit), takes values from the datasets, and saves its filled values incrementally. #[test] fn dynamic_xfa_forms_are_laid_out_on_open_filled_and_saved_incrementally() { let original = Arc::new(pdfcraft_xfa::fixtures::shell(&pdfcraft_xfa::fixtures::template(3))); @@ -6823,6 +7561,40 @@ fn dynamic_xfa_forms_are_laid_out_on_open_filled_and_saved_incrementally() { assert_eq!(d2.form.iter().find(|f| f.name == "answer").unwrap().value, vec!["N".to_string()]); } +/// A signature box's XFA preSign script runs as one undoable step, showing its notice and +/// changing the form; its postSign script runs too; a box without them runs nothing and changes +/// nothing. +#[test] +fn xfa_signing_scripts_run_before_and_after_signing() { + use crate::js::SignEvent; + let bytes = Arc::new(pdfcraft_xfa::fixtures::shell(&pdfcraft_xfa::fixtures::signing_events_template())); + let mut s = Session::new(); + let id = s.open("signing.pdf", None, bytes, None).expect("opens"); + let field = + |s: &Session, n: &str| s.get(id).unwrap().form.iter().find(|f| f.name == n || f.name.ends_with(&format!(".{n}[0]"))).cloned().unwrap(); + let sign = field(&s, "sign").name; + let plain = field(&s, "plain").name; + assert!(s.run_sign_event(id, &sign, SignEvent::Before).unwrap(), "it has a preSign script"); + assert_eq!(field(&s, "when").value, ["signed"]); + assert!(field(&s, "when").read_only(), "and it locks the field"); + assert_eq!(s.take_js_output(id).alerts, [crate::js::Message::plain("Signing notice")]); + assert!(s.run_sign_event(id, &sign, SignEvent::After).unwrap()); + assert_eq!(field(&s, "after").value, ["done"]); + // Each one is an undo step of its own. + s.undo(id).unwrap(); + assert!(field(&s, "after").value.is_empty()); + assert_eq!(field(&s, "when").value, ["signed"]); + // No signing scripts: nothing runs, nothing to undo. + let label = s.get(id).unwrap().undo_label().cloned(); + assert!(!s.run_sign_event(id, &plain, SignEvent::Before).unwrap()); + assert!(!s.run_sign_event(id, "no such field", SignEvent::Before).unwrap()); + assert_eq!(s.get(id).unwrap().undo_label().cloned(), label); + // Nor with JavaScript off. + s.set_javascript(false); + assert!(!s.run_sign_event(id, &sign, SignEvent::Before).unwrap()); +} + +/// A signature box the file already holds a signature for is drawn with that signature's picture. #[test] fn a_signed_xfa_signature_box_shows_the_picture_the_file_holds() { let mut s = Session::new().with_clock(|| 1_700_000_000); @@ -6934,6 +7706,160 @@ fn static_xfa_forms_take_their_values_from_the_datasets_and_write_them_back() { assert!(!d2.info.warnings.iter().any(|w| w.contains("taken from this form's XFA data")), "nothing was rewritten"); } +/// A static XFA exclusion group laid out as a radio group with `/Opt [(1) (2)]` and on states +/// named by position (`/0`, `/1`; ISO 32000-2 Table 230): the data's item `1` is the first +/// button, not the second one's state `/1`, and a click stores the clicked button's item. +/// Public benefit forms are built this way; their "English" came back as "French". +#[test] +fn a_static_xfa_radio_group_with_numeric_items_keeps_the_picked_button() { + let lang = "form1[0].page1[0].lang[0]"; + let data_of = |bytes: Arc>| { + let cos = pdfcraft_cos::Document::open(bytes).unwrap(); + let data = pdfcraft_xfa::parse_datasets(&pdfcraft_xfa::read_packets(&cos).unwrap().unwrap().xdp).unwrap(); + data.text_at(&pdfcraft_xfa::som_to_path(lang)).map(str::to_owned) + }; + let states = |s: &Session, id: DocId| { + let f = s.get(id).unwrap().form.iter().find(|f| f.name == lang).unwrap().clone(); + (f.value.clone(), f.widgets.iter().map(|w| w.state.clone().unwrap_or_default()).collect::>()) + }; + let form = |data: &str| Arc::new(pdfcraft_xfa::fixtures::static_radio_shell(&format!("{data}"))); + let mut s = Session::new().with_clock(|| 1_700_000_000); + // Data filled elsewhere: item 1 (English) turns the first button on, item 2 the second. + let id = s.open("one.pdf", None, form("1"), None).unwrap(); + assert_eq!(states(&s, id), (vec!["0".to_string()], vec!["0".to_string(), "Off".to_string()])); + let id = s.open("two.pdf", None, form("2"), None).unwrap(); + assert_eq!(states(&s, id), (vec!["1".to_string()], vec!["Off".to_string(), "1".to_string()])); + // A click on a button (its on state, as the page sends it) stores that button's item. + let id = s.open("blank.pdf", None, form(""), None).unwrap(); + for (state, item) in [("0", "1"), ("1", "2")] { + s.apply(id, Edit::SetFieldValue { name: lang.into(), value: FieldValue::Radio(Some(state.into())) }).unwrap(); + let saved = s.save_bytes(id).unwrap(); + assert_eq!(data_of(saved.clone()).as_deref(), Some(item), "clicked {state}"); + // Reopened, the same button is on and nothing is rewritten. + let again = s.open("again.pdf", None, saved, None).unwrap(); + assert_eq!(states(&s, again).0, vec![state.to_string()], "clicked {state}"); + assert!(!s.get(again).unwrap().info.warnings.iter().any(|w| w.contains("taken from this form's XFA data")), "clicked {state}"); + } +} + +/// The XFA data layer keeps its own copy of the forms crate's name spelling (it can't depend on +/// forms), and a radio group's value only finds its button while the two agree: UTF-8 as +/// itself, `#` and bytes that aren't UTF-8 as `#XX`. +#[test] +fn xfa_and_forms_spell_names_alike() { + for bytes in [&b"Yes"[..], b"", b"a b", b"A#1", "はい".as_bytes(), b"\x82\xcd\x82\xa2", b"\xff#\x00", b"\x7f\x20~"] { + assert_eq!(pdfcraft_xfa::data::name_text(bytes), pdfcraft_forms::name_text(bytes), "{bytes:?}"); + assert_eq!(pdfcraft_forms::name_bytes(&pdfcraft_xfa::data::name_text(bytes)), bytes, "{bytes:?}"); + } +} + +/// Taking the XFA out of a radio group whose buttons carry `/PCOn` as well as `/Opt` (the item +/// saving writes ahead of the `/Opt` entry) notes no lost data when the data holds that item: +/// one equal to the on state, and one equal to neither the state nor the `/Opt` entry. +#[test] +fn stripping_xfa_takes_a_radio_items_pcon_as_the_fields_value() { + let lang = "form1[0].page1[0].lang[0]"; + let lost = |s: &Session, id: DocId| { + s.get(id).unwrap().info.warnings.iter().filter(|w| w.contains("held a value other than the form's XFA data")).cloned().collect::>() + }; + let mut s = Session::new().with_clock(|| 1_700_000_000); + // `/PCOn` equals the state: the data's `0` is the first button, whose `/Opt` entry is `1`. + let form = pdfcraft_xfa::fixtures::static_radio_shell_with_pcon("0", ["0", "1"]); + let id = s.open("pcon-state.pdf", None, Arc::new(form), None).unwrap(); + assert_eq!(s.get(id).unwrap().form.iter().find(|f| f.name == lang).unwrap().value, ["0"]); + s.apply(id, Edit::XfaToStatic).unwrap(); + assert_eq!(lost(&s, id), Vec::::new()); + // `/PCOn` is neither: a click writes `en` to the data for the first button. + let form = pdfcraft_xfa::fixtures::static_radio_shell_with_pcon("", ["en", "fr"]); + let id = s.open("pcon-item.pdf", None, Arc::new(form), None).unwrap(); + s.apply(id, Edit::SetFieldValue { name: lang.into(), value: FieldValue::Radio(Some("0".into())) }).unwrap(); + s.apply(id, Edit::XfaToStatic).unwrap(); + assert_eq!(lost(&s, id), Vec::::new()); + // The data names a button by its `/Opt` entry (`2`, as data from elsewhere does), while + // saving would write that button's `/PCOn` (`fr`). Opening turned that button on, so the + // field shows what the data selects and nothing is lost, although the spellings differ. + let form = pdfcraft_xfa::fixtures::static_radio_shell_with_pcon("2", ["en", "fr"]); + let id = s.open("opt-item.pdf", None, Arc::new(form), None).unwrap(); + assert_eq!(s.get(id).unwrap().form.iter().find(|f| f.name == lang).unwrap().value, ["1"]); + s.apply(id, Edit::XfaToStatic).unwrap(); + assert_eq!(lost(&s, id), Vec::::new()); +} + +/// Opening reads a radio group's data as saving writes it: with `/PCOn` items `4` `1` next to +/// `/Opt [(1) (2)]` and states `/0` `/1`, a click on the second button saves its item `1`, and +/// reopening turns the second button on again, not the first one whose `/Opt` entry is `1`; the +/// first button saves `4` and comes back too. Taking the XFA out then notes no lost data. +#[test] +fn a_radio_item_that_is_another_buttons_export_reopens_on_its_own_button() { + let lang = "form1[0].page1[0].lang[0]"; + let data_of = |bytes: Arc>| { + let cos = pdfcraft_cos::Document::open(bytes).unwrap(); + let data = pdfcraft_xfa::parse_datasets(&pdfcraft_xfa::read_packets(&cos).unwrap().unwrap().xdp).unwrap(); + data.text_at(&pdfcraft_xfa::som_to_path(lang)).map(str::to_owned) + }; + let value = |s: &Session, id: DocId| s.get(id).unwrap().form.iter().find(|f| f.name == lang).unwrap().value.clone(); + let form = Arc::new(pdfcraft_xfa::fixtures::static_radio_shell_with_pcon("", ["4", "1"])); + let mut s = Session::new().with_clock(|| 1_700_000_000); + for (state, item) in [("1", "1"), ("0", "4")] { + let id = s.open("blank.pdf", None, form.clone(), None).unwrap(); + s.apply(id, Edit::SetFieldValue { name: lang.into(), value: FieldValue::Radio(Some(state.into())) }).unwrap(); + let saved = s.save_bytes(id).unwrap(); + assert_eq!(data_of(saved.clone()).as_deref(), Some(item), "clicked {state}"); + let again = s.open("again.pdf", None, saved, None).unwrap(); + assert_eq!(value(&s, again), [state], "clicked {state}"); + s.apply(again, Edit::XfaToStatic).unwrap(); + assert!(!s.get(again).unwrap().info.warnings.iter().any(|w| w.contains("held a value other than the form's XFA data")), "clicked {state}"); + } + // Data from elsewhere that no button saves is still read as an export value, then a state. + let id = s + .open( + "two.pdf", + None, + Arc::new(pdfcraft_xfa::fixtures::static_radio_shell_with_pcon("2", ["4", "1"])), + None, + ) + .unwrap(); + assert_eq!(value(&s, id), ["1"]); + let id = s + .open( + "zero.pdf", + None, + Arc::new(pdfcraft_xfa::fixtures::static_radio_shell_with_pcon("0", ["4", "1"])), + None, + ) + .unwrap(); + assert_eq!(value(&s, id), ["0"]); +} + +/// A radio group whose buttons carry `/PCOn` items `1` `2` next to `/Opt [(1) (2)]` and on +/// states named by position (`/0`, `/1`): a click on either button saves that button's own item +/// to the datasets, reopening turns the same button on with nothing rewritten, and taking the XFA +/// out notes no lost data. The state `/1` used to match the first button's item `1`, so the +/// second button saved `1` and came back as the first. +#[test] +fn a_radio_group_with_pcon_items_saves_and_reopens_the_clicked_button() { + let lang = "form1[0].page1[0].lang[0]"; + let data_of = |bytes: Arc>| { + let cos = pdfcraft_cos::Document::open(bytes).unwrap(); + let data = pdfcraft_xfa::parse_datasets(&pdfcraft_xfa::read_packets(&cos).unwrap().unwrap().xdp).unwrap(); + data.text_at(&pdfcraft_xfa::som_to_path(lang)).map(str::to_owned) + }; + let value = |s: &Session, id: DocId| s.get(id).unwrap().form.iter().find(|f| f.name == lang).unwrap().value.clone(); + let form = Arc::new(pdfcraft_xfa::fixtures::static_radio_shell_with_pcon("", ["1", "2"])); + let mut s = Session::new().with_clock(|| 1_700_000_000); + for (state, item) in [("0", "1"), ("1", "2")] { + let id = s.open("blank.pdf", None, form.clone(), None).unwrap(); + s.apply(id, Edit::SetFieldValue { name: lang.into(), value: FieldValue::Radio(Some(state.into())) }).unwrap(); + let saved = s.save_bytes(id).unwrap(); + assert_eq!(data_of(saved.clone()).as_deref(), Some(item), "clicked {state}"); + let again = s.open("again.pdf", None, saved, None).unwrap(); + assert_eq!(value(&s, again), [state], "clicked {state}"); + assert!(!s.get(again).unwrap().info.warnings.iter().any(|w| w.contains("taken from this form's XFA data")), "clicked {state}"); + s.apply(again, Edit::XfaToStatic).unwrap(); + assert!(!s.get(again).unwrap().info.warnings.iter().any(|w| w.contains("held a value other than the form's XFA data")), "clicked {state}"); + } +} + /// A static XFA form whose values are only in its datasets takes every one on opening, as /// filling the fields one at a time would leave it: values, appearances and the note; a /// read-only field keeps its own value. @@ -7279,12 +8205,12 @@ fn xfa_data_import_adds_and_removes_rows_of_a_dynamic_form() { assert_eq!(names(&s).iter().filter(|n| n.starts_with("amount")).count(), 3, "{:?}", names(&s)); assert_eq!(value(&s, "total"), vec!["15".to_string()], "qty 3 × price 5, calculated after the import: {:?}", s.take_js_output(id)); assert_eq!(value(&s, "grand"), vec!["42.5".to_string()]); - // More rows than the layout shows: the document says so. - let rows: String = (0..60).map(|i| format!("r{i}1")).collect(); + // More rows than the layout shows (1 000 for one repeating subform): the document says so. + let rows: String = (0..1010).map(|i| format!("r{i}1")).collect(); let file = format!("
1{rows}
"); s.apply(id, Edit::ImportData { name: "many.xml".into(), bytes: Arc::new(file.into_bytes()) }).unwrap(); let out = s.take_js_output(id); - assert!(out.errors.iter().chain(out.console.iter()).any(|l| l.contains("60 rows")), "{out:?}"); + assert!(out.errors.iter().chain(out.console.iter()).any(|l| l.contains("1010 rows") && l.contains("first 1000")), "{out:?}"); // A file with one row takes the others away. let file = "
1only7
"; s.apply(id, Edit::ImportData { name: "one.xml".into(), bytes: Arc::new(file.as_bytes().to_vec()) }).unwrap(); @@ -7356,6 +8282,119 @@ fn the_xfa_open_budget_stops_the_calculations_it_has_no_time_for() { assert_eq!(pdfcraft_js::xfa_session::engines_started(), engines, "no script ran, so no engine started"); } +/// Opening lays the form out once after its initialize scripts, not after each one that +/// hides a subform (XFA 3.3 pp. 388, 390: initialize fires once data binding is complete; +/// the layout process has not run yet). Laying it out after every script spent the opening's +/// time on the first few, and a form whose initialize scripts hide its optional pages was +/// saved half initialized: the next opening hid more of it, and reset the data typed into +/// what it hid. +#[test] +fn xfa_initialize_scripts_that_hide_subforms_lay_the_form_out_once() { + // Shaped like a form with optional extra pages: a check box, then more sections than one + // event may lay the form out again for, each hiding itself while the box is clear. + let sections = 60; + let mut body = String::from( + r#"100"#, + ); + for n in 0..sections { + body.push_str(&format!( + r#""# + )); + } + let tpl = format!( + r#" + + + +"# + ); + let bytes = Arc::new(pdfcraft_xfa::fixtures::shell(&tpl)); + let mut s = Session::new().with_clock(|| 1_700_000_000).with_xfa_time_scale(100); + let id = s.open("extras.pdf", None, bytes, None).expect("opens"); + let out = s.take_js_output(id); + assert!(out.errors.is_empty(), "{:?}", out.errors); + let names = |s: &Session, id| s.get(id).unwrap().form.iter().map(|f| f.name.clone()).collect::>(); + let opened = names(&s, id); + assert!(opened.iter().all(|n| !n.contains("note")), "every extra section is hidden on opening: {opened:?}"); + assert!(opened.iter().any(|n| n.contains("wanted")), "{opened:?}"); + // User input survives repeated saves and openings, with stable field and page counts + // (strengthened by wireless89 in the review of #4791). + let wanted = opened.iter().find(|n| n.contains("wanted")).unwrap().clone(); + s.apply(id, Edit::SetFieldValue { name: wanted.clone(), value: FieldValue::Check(true) }).unwrap(); + let pages = s.get(id).unwrap().info.pages.len(); + let mut saved = s.save_bytes(id).unwrap(); + for cycle in 0..3 { + let again = s.open("again.pdf", None, saved, None).expect("reopens"); + let out = s.take_js_output(again); + assert!(out.errors.is_empty(), "cycle {cycle}: {out:?}"); + assert_eq!(names(&s, again), opened, "cycle {cycle}: stable fields"); + let doc = s.get(again).unwrap(); + assert_eq!(doc.info.pages.len(), pages, "cycle {cycle}: stable pages"); + assert_eq!(doc.form.iter().find(|f| f.name == wanted).unwrap().value, ["1"], "cycle {cycle}: input survives"); + saved = s.save_bytes(again).unwrap(); + } +} + +/// A script that resets several subforms one after another (`xfa.host.resetData` once for +/// each, as forms with optional pages do) resets every one of them, and nothing else; resets +/// in a row are applied together. +#[test] +fn xfa_resets_in_a_row_reset_every_named_subform_and_nothing_else() { + let sections = 5; + let mut body = String::from( + r#"100"#, + ); + let resets: String = (0..sections).map(|n| format!("xfa.host.resetData('form.extra{n}');")).collect(); + body.push_str(&format!( + r#""# + )); + body.push_str(r#""#); + // A field no script touches: the resets must leave what was typed in it. + body.push_str(r#"default"#); + for n in 0..sections { + body.push_str(&format!( + r#"default"# + )); + } + let notes: String = (0..sections).map(|n| format!("typed {n}")).collect(); + let tpl = format!( + r#" + + +
typedtyped other{notes}
 + +"# + ); + let mut s = Session::new().with_clock(|| 1_700_000_000).with_xfa_time_scale(100); + // Without the resets the sections show what was typed. + let untouched = Arc::new(pdfcraft_xfa::fixtures::shell(&tpl.replace(&resets, ""))); + let plain = s.open("plain.pdf", None, untouched, None).expect("opens"); + let typed: Vec<_> = s.get(plain).unwrap().form.iter().filter(|f| f.name.contains("note")).map(|f| f.value.clone()).collect(); + assert_eq!(typed, (0..sections).map(|n| vec![format!("typed {n}")]).collect::>()); + let bytes = Arc::new(pdfcraft_xfa::fixtures::shell(&tpl)); + let id = s.open("resets.pdf", None, bytes, None).expect("opens"); + let out = s.take_js_output(id); + assert!(out.errors.is_empty(), "{:?}", out.errors); + let form = &s.get(id).unwrap().form; + let notes: Vec<_> = form.iter().filter(|f| f.name.contains("note")).map(|f| f.value.clone()).collect(); + assert_eq!(notes.len(), sections, "{form:?}"); + assert!(notes.iter().all(|v| v == &vec!["default".to_string()]), "every section is reset: {notes:?}"); + let keep = form.iter().find(|f| f.name.contains("keep")).map(|f| f.value.clone()); + assert_eq!(keep, Some(vec!["kept".to_string()]), "the script's later write stands"); + let other = form.iter().find(|f| f.name.contains("other")).map(|f| f.value.clone()); + assert_eq!(other, Some(vec!["typed other".to_string()]), "a field outside the named subforms keeps its value"); +} + /// The scripts of one event share the form as the engine holds it: a failed script's /// half-done writes reach nobody, and a script reads the saved value a choice list was given /// by its shown text. @@ -7527,6 +8566,57 @@ fn repeated_xfa_print_buttons_run_their_click_script() { } } +/// A button on the page master runs its click script as the copy on the page that was clicked: +/// the field it writes next to it changes on that page only. +#[test] +fn a_page_master_button_writes_the_field_on_its_own_page() { + let template = pdfcraft_xfa::fixtures::repeated_print_button_template().replace( + "", + r#" +