Mirrors/pdfcraft
1
0 Fork 0
mirror of https://github.com/storytold/pdfcraft.git synced 2026-11-11 08:51:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'upstream/main' into lane3-6504

Browse Source 
# Conflicts:
#	crates/annot/src/appearance.rs
#	crates/annot/src/draw.rs
#	crates/annot/src/info.rs
#	crates/annot/src/modify.rs
This commit is contained in:
mtlcl410 committed 2026-11-09 02:29:36 +08:00
2 parents 6a238f9fd7 1afbc1254a
commit aa45114740
234 files changed
+23465 -1985
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.cargo/config.toml
+5
View File
@@ -1,2 +1,7 @@
[alias]
xtask = "run -p xtask --"
# WebAssembly SIMD (every current browser has it; there's no runtime detection on wasm). The OCR
# engine's kernels (rten-simd) and other vectorised code only use it when compiled in.
[target.wasm32-unknown-unknown]
rustflags = ["-C", "target-feature=+simd128"]
.github/workflows/release.yml
+8
View File
@@ -19,7 +19,14 @@ concurrency:
cancel-in-progress: false
jobs:
# Regenerates contributors/contributors.json for the release commit; every build bundles it.
credits:
permissions:
contents: read
pull-requests: read
uses: storytold/craft-common-actions/.github/workflows/refresh-contributors.yml@main
release:
needs: credits
# Intentionally tracked at @main (not SHA-pinned) so org-wide CI improvements, security fixes, and build optimizations propagate without PR churn across all craftapp repos.
uses: storytold/craft-common-actions/.github/workflows/release.yml@main
with:
@@ -27,6 +34,7 @@ jobs:
app-title: PdfCraft
craft-fonts-ref: b076417b5880bd4154bd22f00695b1d28c6bea83
version: ${{ inputs.version }}
contributors-artifact: ${{ needs.credits.outputs.artifact-name }}
secrets:
APPLE_CERTIFICATE: ${{ secrets.APPLE_CERTIFICATE }}
APPLE_CERTIFICATE_PASSWORD: ${{ secrets.APPLE_CERTIFICATE_PASSWORD }}
AGENTS.md
+4 -3
View File
@@ -1,4 +1,4 @@
<!-- craft-house-rules:start core source=dde09ea31e2e457f5b4653a6e76f7cdf55944bdf sha256=540e6cea75d12d0710c3dca89658328e81e508ecef102446ca5bb95b7c0b92bb -->
<!-- craft-house-rules:start core source=1315c63e17905daf7dbc6dc6f8c49384e415162b sha256=540e6cea75d12d0710c3dca89658328e81e508ecef102446ca5bb95b7c0b92bb -->
## House rules
These apply in every Craft repository. A later rule may be stricter, not weaker. The source is `standards/house-rules-core.md` in `storytold/craft-contrib`. Propose changes there, not in a copy.
@@ -19,7 +19,7 @@ These apply in every Craft repository. A later rule may be stricter, not weaker.
14. **Keep pull requests moving.** Concurrent pull requests are fine; unanswered actionable feedback is not. Respond in 7 days, a reminder at 14, closure eligible at 30, from when an actionable request waits on the author. Suggestions and unconfirmed bot findings do not start the clock. A fix, explanation or agreed plan resets it; unrelated activity does not. Waiting on a reviewer or maintainer pauses it and does not count against the author. A maintainer decides closure; it is never automatic. Reuse search results, do not poll, and stop at a rate limit.
15. **No agent attribution.** A pull request does not say an agent wrote it. A repository rule may not require that.
<!-- craft-house-rules:end -->
<!-- craft-house-rules:start rust source=dde09ea31e2e457f5b4653a6e76f7cdf55944bdf sha256=387e0ccb4b49e567f60951d7d6fff7305054c89f43ad812b9993dc7c38528013 -->
<!-- craft-house-rules:start rust source=1315c63e17905daf7dbc6dc6f8c49384e415162b sha256=642f46b4ac38cc9e4d10d1ccde19d68cb6bc03d20e4c6236a7535f3047566262 -->
## Rust Craft repositories
These apply to native Craft apps and to shared Rust libraries. Select them by repository type. A repository that merely contains some Rust does not take this block. Rules later in a repository's file may be stricter, but not weaker. The source is `standards/house-rules-rust.md` in [storytold/craft-contrib](https://github.com/storytold/craft-contrib).
@@ -40,6 +40,7 @@ Dependencies and assets use SPDX identifiers. An expression is allowed when ever
- `BSD-2-Clause`
- `BSD-3-Clause`
- `Apache-2.0`
- `Unicode-3.0`
- `OFL-1.1`
- `Ubuntu-font-1.0`
- `Bitstream-Vera`
@@ -173,7 +174,7 @@ PdfCraft opens files from strangers, and people trust it with their work. A malf
- **Layering.** Nothing below L7 depends on egui/winit/eframe/rfd. `cos`, `filters`, `crypt` and `arlington` stay standalone.
- **Page input.** Page modules read `PagePointer` (`crates/ui-egui/src/page_pointer.rs`), never egui's raw pointer state. `page_pointer::tests::page_modules_read_the_pointer_only_through_page_pointer` enforces it.
- **Page cursor.** Page modules claim the cursor for what is under the pointer (`view.cursor.claim`, `crates/ui-egui/src/canvas/page_cursor.rs`) and never set it themselves. `page_pointer::tests::page_modules_claim_the_cursor_and_never_set_it` enforces it.
- **Page presses.** A page module decides what is under a point in one hit test (`Hit` in `crates/ui-egui/src/canvas/page_router.rs`) and acts on a press only through that router. `page_pointer::tests::page_modules_act_on_a_press_only_through_the_router` enforces it.
- **Page presses.** A page module decides what is under a point in one hit test (`Hit` in `crates/ui-egui/src/canvas/page_router.rs`) and acts on a press only through that router. Modules never read the router; they are told. `page_pointer::tests::page_modules_act_on_a_press_only_through_the_router` enforces it.
- **Fidelity.** The PDF object graph is the model. Preserve unknown data. Saves are incremental unless a full rewrite is required. Never silently drop or repair data without recording it.
- **Fonts** are the optional build input `CRAFT_FONTS_DIR`. Code using `pdfcraft_fonts::CRAFT_FONTS` must work when it is empty.
- **Quality gates** before every commit: `cargo fmt --check`, clippy with `-D warnings`, the tests, `cargo xtask layers`, `cargo xtask assets`, and the wasm check. New workspace crates include `workspace = true` under `[lints]`. Scope `cargo clippy -p <crate> --all-targets -- -D warnings` and `cargo test -p <crate>` to the crates you touched; the workspace sweep is CI's job.
ATTRIBUTION.md
+5 -5
View File
@@ -45,7 +45,7 @@ Every asset PdfCraft includes, bundles or uses to build its published material,
| `assets/icons/chevron-up.svg` | Lucide icon "chevron-up" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/chevron-up.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
| `assets/icons/chevrons-left.svg` | Lucide icon "chevrons-left" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/chevrons-left.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
| `assets/icons/chevrons-right.svg` | Lucide icon "chevrons-right" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/chevrons-right.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
| `assets/icons/circle-check.svg` | Lucide icon "circle-check" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/circle-check.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
| `assets/icons/circle-check.svg` | Lucide icon "circle-check" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/circle-check.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs; drawn as PDF path data by crates/sign/src/status_layers.rs into displayed signature status layers |
| `assets/icons/circle-dashed.svg` | Lucide icon "circle-dashed" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/circle-dashed.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
| `assets/icons/circle-dot.svg` | Lucide icon "circle-dot" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/circle-dot.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
| `assets/icons/circle-help.svg` | Lucide icon "circle-help" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/circle-help.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
@@ -174,7 +174,7 @@ Every asset PdfCraft includes, bundles or uses to build its published material,
| `assets/icons/text-select.svg` | Lucide icon "text-select" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/text-select.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
| `assets/icons/trash-2.svg` | Lucide icon "trash-2" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/trash-2.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
| `assets/icons/trash.svg` | Lucide icon "trash" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/trash.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
| `assets/icons/triangle-alert.svg` | Lucide icon "triangle-alert" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/triangle-alert.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
| `assets/icons/triangle-alert.svg` | Lucide icon "triangle-alert" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/triangle-alert.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs; drawn as PDF path data by crates/sign/src/status_layers.rs into displayed signature status layers |
| `assets/icons/type.svg` | Lucide icon "type" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/type.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
| `assets/icons/underline.svg` | Lucide icon "underline" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/underline.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
| `assets/icons/undo-2.svg` | Lucide icon "undo-2" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/undo-2.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
@@ -184,7 +184,7 @@ Every asset PdfCraft includes, bundles or uses to build its published material,
| `assets/icons/x.svg` | Lucide icon "x" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/x.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
| `assets/icons/zoom-in.svg` | Lucide icon "zoom-in" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/zoom-in.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
| `assets/icons/zoom-out.svg` | Lucide icon "zoom-out" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/zoom-out.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
| `assets/fonts/Inter-Regular.ttf` | Inter Regular | The Inter Project Authors (Rasmus Andersson) | OFL-1.1 | https://github.com/rsms/inter | UI font, embedded in the app; demo PDF labels; slanted fallback glyphs in Fill & Sign typed signatures for characters Dancing Script lacks (Cyrillic, Greek), drawn as filled paths |
| `assets/fonts/Inter-Regular.ttf` | Inter Regular | The Inter Project Authors (Rasmus Andersson) | OFL-1.1 | https://github.com/rsms/inter | UI font, embedded in the app; demo PDF labels; slanted fallback glyphs in Fill & Sign typed signatures for characters Dancing Script lacks (Cyrillic, Greek), drawn as filled paths; embedded into displayed signature status captions beyond WinAnsi by crates/sign/src/status_layers.rs |
| `assets/fonts/Inter-Medium.ttf` | Inter Medium | The Inter Project Authors (Rasmus Andersson) | OFL-1.1 | https://github.com/rsms/inter | UI font, embedded in the app; demo PDF labels |
| `assets/fonts/Inter-SemiBold.ttf` | Inter SemiBold | The Inter Project Authors (Rasmus Andersson) | OFL-1.1 | https://github.com/rsms/inter | UI font, embedded in the app; demo PDF labels |
| `assets/fonts/JetBrainsMono-Regular.ttf` | JetBrains Mono Regular | The JetBrains Mono Project Authors | OFL-1.1 | https://github.com/JetBrains/JetBrainsMono | UI monospace font; demo PDF code |
@@ -249,7 +249,7 @@ Every asset PdfCraft includes, bundles or uses to build its published material,
| `assets/icons/thumbs-up.svg` | Lucide icon "thumbs-up" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/thumbs-up.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
| `assets/icons/thumbs-down.svg` | Lucide icon "thumbs-down" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/thumbs-down.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
| `assets/icons/message-square-plus.svg` | Lucide icon "message-square-plus" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/message-square-plus.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
| `assets/icons/circle-x.svg` | Lucide icon "circle-x" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/circle-x.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
| `assets/icons/circle-x.svg` | Lucide icon "circle-x" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/circle-x.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs; drawn as PDF path data by crates/sign/src/status_layers.rs into displayed signature status layers |
| `crates/ui-egui/tests/data/form.pdf` | One-page AcroForm test fixture (text, check box, radio group, combo box) | PdfCraft contributors | MIT OR Apache-2.0 | Contributor-original: hand-written PDF objects, no third-party content | Form-filling UI tests (crates/ui-egui/tests/it/forms.rs, commands.rs) |
| `assets/icons/pentagon.svg` | Lucide icon "pentagon" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/pentagon.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
| `assets/icons/message-square-quote.svg` | Lucide icon "message-square-quote" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/message-square-quote.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
@@ -314,4 +314,4 @@ Not in this repository and never downloaded by it: compiled in only when the bui
| Input | Option | Title | Author | Licence | Source | Attribution | Used for |
|---|---|---|---|---|---|---|---|
| `craft-i18n` | `CRAFT_I18N_DIR` | Interface translations: the craft-i18n catalogs (locales/<code>/common.tsv and pdfcraft.tsv) of the 34 languages PdfCraft offers besides English | The craft-i18n translators, including every translator of PdfCraft's former in-repo catalogs (crates/ui-egui/src/i18n/*.tsv, migrated at pdfcraft@ecdd1e6) | MIT OR Apache-2.0 | https://github.com/storytold/craft-i18n (pinned by commit in craft-i18n.rev, which CI, releases and the flatpak manifests read) | https://github.com/storytold/craft-i18n/blob/main/ATTRIBUTION.md | Interface text in languages other than English, embedded by crates/ui-egui/build.rs and looked up through craft-libs' craft-i18n crate; a build without CRAFT_I18N_DIR is English only. Clean-room translations from PdfCraft's English interface labels |
| `craft-fonts` | `CRAFT_FONTS_DIR` | Japanese fonts: BIZ UDPGothic (Regular, Bold), Shippori Mincho, BIZ UDMincho; native Simplified Chinese font: LXGW WenKai Lite; Arabic font: Noto Sans Arabic | The BIZ UDGothic and BIZ UDMincho Project Authors (Morisawa Inc.), The Shippori Mincho Project Authors (FONTDASU), LXGW and the original Klee Project Authors / Fontworks (LXGW WenKai Lite), The Noto Project Authors (Noto Sans Arabic) | OFL-1.1 | https://github.com/storytold/craft-fonts (pinned by commit in .github/workflows/release.yml and .github/workflows/ci.yml); LXGW WenKai Lite from https://github.com/lxgw/LxgwWenKai-Lite/releases/tag/v1.530 (commit b712c83d98acf977bacfb7fb010207802808bda9) | https://github.com/storytold/craft-fonts/blob/main/ATTRIBUTION.md | Japanese fallback in the interface (BIZ UDPGothic first) and Japanese glyphs in edited PDF text (Shippori Mincho); Simplified Chinese interface fallback on native builds only (LXGW WenKai Lite Regular, font SHA-256 f799a538a3355877e597a0ae04050e897932c9ae7689eb9227c2953c5dd549ea, OFL SHA-256 0333a85fe077b6849f8e417acc44e8bd879442ceb681e271acf93e2d57fd4e2e, read from fonts/simplified-chinese-manifest.txt and never embedded on wasm); Arabic interface text, file names and document titles (Noto Sans Arabic); the web build embeds BIZ UDPGothic Regular and the pinned small Arabic, Hebrew, Thai and Devanagari faces; the selected craft-fonts revision contains no Telugu face. Other non-Japanese CJK faces retain the general 4 MiB cap; the reviewed native-only LXGW fallback has a separate 16 MiB cap. Open-source OFL releases are permitted under current AGENTS.md §1.1 |
| `craft-fonts` | `CRAFT_FONTS_DIR` | Japanese fonts: BIZ UDPGothic (Regular, Bold), Shippori Mincho, BIZ UDMincho; native Simplified Chinese font: LXGW WenKai Lite; Arabic font: Noto Sans Arabic | The BIZ UDGothic and BIZ UDMincho Project Authors (Morisawa Inc.), The Shippori Mincho Project Authors (FONTDASU), LXGW and the original Klee Project Authors / Fontworks (LXGW WenKai Lite), The Noto Project Authors (Noto Sans Arabic) | OFL-1.1 | https://github.com/storytold/craft-fonts (pinned by commit in .github/workflows/release.yml and .github/workflows/ci.yml); LXGW WenKai Lite from https://github.com/lxgw/LxgwWenKai-Lite/releases/tag/v1.530 (commit b712c83d98acf977bacfb7fb010207802808bda9) | https://github.com/storytold/craft-fonts/blob/main/ATTRIBUTION.md | Japanese fallback in the interface (BIZ UDPGothic first) and Japanese glyphs in edited PDF text (Shippori Mincho); Simplified Chinese interface fallback on native builds only (LXGW WenKai Lite Regular, font SHA-256 f799a538a3355877e597a0ae04050e897932c9ae7689eb9227c2953c5dd549ea, OFL SHA-256 0333a85fe077b6849f8e417acc44e8bd879442ceb681e271acf93e2d57fd4e2e, read from fonts/simplified-chinese-manifest.txt and never embedded on wasm); Arabic interface text, file names and document titles (Noto Sans Arabic); status captions of named-layer signatures in Arabic, Japanese, Thai and Devanagari, drawn as filled outlines into the display copy by crates/sign/src/status_layers.rs; the web build embeds BIZ UDPGothic Regular and the pinned small Arabic, Hebrew, Thai and Devanagari faces; the selected craft-fonts revision contains no Telugu face. Other non-Japanese CJK faces retain the general 4 MiB cap; the reviewed native-only LXGW fallback has a separate 16 MiB cap. Open-source OFL releases are permitted under current AGENTS.md §1.1 |
ATTRIBUTION.toml
+5 -5
View File
@@ -425,7 +425,7 @@ source = "https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icon
licence = "ISC AND MIT"
licence_file = "assets/icons/LICENSE-lucide.txt"
kind = "icon"
usage = "UI icon, embedded via crates/ui-egui/src/icon_data.rs"
usage = "UI icon, embedded via crates/ui-egui/src/icon_data.rs; drawn as PDF path data by crates/sign/src/status_layers.rs into displayed signature status layers"
sha256 = "201e790f3a05c65b2c4b0f952c9b9e1f35cd1c5b21698b3f4be131883fe90bc8"
[[asset]]
@@ -1844,7 +1844,7 @@ source = "https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icon
licence = "ISC AND MIT"
licence_file = "assets/icons/LICENSE-lucide.txt"
kind = "icon"
usage = "UI icon, embedded via crates/ui-egui/src/icon_data.rs"
usage = "UI icon, embedded via crates/ui-egui/src/icon_data.rs; drawn as PDF path data by crates/sign/src/status_layers.rs into displayed signature status layers"
sha256 = "55aa37e0b73e7f8cd906952dbdeb96283c6595ca64aba240a17f3455cb989114"
[[asset]]
@@ -1954,7 +1954,7 @@ source = "https://github.com/rsms/inter"
licence = "OFL-1.1"
licence_file = "assets/fonts/OFL-Inter.txt"
kind = "font"
usage = "UI font, embedded in the app; demo PDF labels; slanted fallback glyphs in Fill & Sign typed signatures for characters Dancing Script lacks (Cyrillic, Greek), drawn as filled paths"
usage = "UI font, embedded in the app; demo PDF labels; slanted fallback glyphs in Fill & Sign typed signatures for characters Dancing Script lacks (Cyrillic, Greek), drawn as filled paths; embedded into displayed signature status captions beyond WinAnsi by crates/sign/src/status_layers.rs"
sha256 = "40d692fce188e4471e2b3cba937be967878f631ad3ebbbdcd587687c7ebe0c82"
[[asset]]
@@ -3004,7 +3004,7 @@ source = "https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icon
licence = "ISC AND MIT"
licence_file = "assets/icons/LICENSE-lucide.txt"
kind = "icon"
usage = "UI icon, embedded via crates/ui-egui/src/icon_data.rs"
usage = "UI icon, embedded via crates/ui-egui/src/icon_data.rs; drawn as PDF path data by crates/sign/src/status_layers.rs into displayed signature status layers"
sha256 = "ef1fec17d88794ba6c34c5cf63bdadbd234b3d4dffbcc3d55468b47e3f398472"
[[asset]]
@@ -3142,7 +3142,7 @@ source = "https://github.com/storytold/craft-fonts (pinned by commit in .github/
licence = "OFL-1.1"
attribution = "https://github.com/storytold/craft-fonts/blob/main/ATTRIBUTION.md"
kind = "font"
usage = "Japanese fallback in the interface (BIZ UDPGothic first) and Japanese glyphs in edited PDF text (Shippori Mincho); Simplified Chinese interface fallback on native builds only (LXGW WenKai Lite Regular, font SHA-256 f799a538a3355877e597a0ae04050e897932c9ae7689eb9227c2953c5dd549ea, OFL SHA-256 0333a85fe077b6849f8e417acc44e8bd879442ceb681e271acf93e2d57fd4e2e, read from fonts/simplified-chinese-manifest.txt and never embedded on wasm); Arabic interface text, file names and document titles (Noto Sans Arabic); the web build embeds BIZ UDPGothic Regular and the pinned small Arabic, Hebrew, Thai and Devanagari faces; the selected craft-fonts revision contains no Telugu face. Other non-Japanese CJK faces retain the general 4 MiB cap; the reviewed native-only LXGW fallback has a separate 16 MiB cap. Open-source OFL releases are permitted under current AGENTS.md §1.1"
usage = "Japanese fallback in the interface (BIZ UDPGothic first) and Japanese glyphs in edited PDF text (Shippori Mincho); Simplified Chinese interface fallback on native builds only (LXGW WenKai Lite Regular, font SHA-256 f799a538a3355877e597a0ae04050e897932c9ae7689eb9227c2953c5dd549ea, OFL SHA-256 0333a85fe077b6849f8e417acc44e8bd879442ceb681e271acf93e2d57fd4e2e, read from fonts/simplified-chinese-manifest.txt and never embedded on wasm); Arabic interface text, file names and document titles (Noto Sans Arabic); status captions of named-layer signatures in Arabic, Japanese, Thai and Devanagari, drawn as filled outlines into the display copy by crates/sign/src/status_layers.rs; the web build embeds BIZ UDPGothic Regular and the pinned small Arabic, Hebrew, Thai and Devanagari faces; the selected craft-fonts revision contains no Telugu face. Other non-Japanese CJK faces retain the general 4 MiB cap; the reviewed native-only LXGW fallback has a separate 16 MiB cap. Open-source OFL releases are permitted under current AGENTS.md §1.1"
[[asset]]
path = "crates/sign/tests/data/x509-rsa-sha1.pdf"
Cargo.lock
Generated
+58 -5
View File
@@ -1243,7 +1243,7 @@ dependencies = [
[[package]]
name = "craft-print"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=787a7a0f972fd1d7a7618fdbeca3a5267a0407ff#787a7a0f972fd1d7a7618fdbeca3a5267a0407ff"
source = "git+https://github.com/storytold/craft-libs?rev=7ee35a9fbd47af572d55fd63dbf0333d1f00da17#7ee35a9fbd47af572d55fd63dbf0333d1f00da17"
dependencies = [
"craft-winprint-gdi",
"log",
@@ -1312,10 +1312,21 @@ dependencies = [
"zip",
]
[[package]]
name = "craft-web-api"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=fd5e2feff6ea25bed371a99cc732b3968d621505#fd5e2feff6ea25bed371a99cc732b3968d621505"
dependencies = [
"js-sys",
"serde_json",
"wasm-bindgen",
"wasm-bindgen-futures",
]
[[package]]
name = "craft-web-host"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=2e2312ddaf21bc3d85cd3395d0955c7d239306f6#2e2312ddaf21bc3d85cd3395d0955c7d239306f6"
source = "git+https://github.com/storytold/craft-libs?rev=fd5e2feff6ea25bed371a99cc732b3968d621505#fd5e2feff6ea25bed371a99cc732b3968d621505"
dependencies = [
"wasm-bindgen",
"web-sys",
@@ -1324,7 +1335,7 @@ dependencies = [
[[package]]
name = "craft-winprint-gdi"
version = "0.1.0"
source = "git+https://github.com/storytold/winprint.rs?rev=417ac3c4d14d3da5485a99c7d5ce8f05664ec312#417ac3c4d14d3da5485a99c7d5ce8f05664ec312"
source = "git+https://github.com/storytold/winprint.rs?rev=64e7afd4d1483512385012c340ecaaaf0f3beb7f#64e7afd4d1483512385012c340ecaaaf0f3beb7f"
dependencies = [
"windows-sys 0.61.2",
]
@@ -4254,7 +4265,6 @@ version = "0.18.1"
dependencies = [
"pdfcraft-content",
"pdfcraft-cos",
"pdfcraft-filters",
"pdfcraft-fonts",
"pdfcraft-model",
"thiserror 2.0.21",
@@ -4267,6 +4277,7 @@ dependencies = [
"hayro-syntax",
"pdfcraft-cos",
"pdfcraft-fonts",
"pdfcraft-geom",
"pdfcraft-model",
"thiserror 2.0.21",
]
@@ -4529,10 +4540,22 @@ dependencies = [
"pdfcraft-fonts",
"png 0.18.1",
"rten",
"serde_json",
"sha2 0.11.1",
"tempfile",
"thiserror 2.0.21",
]
[[package]]
name = "pdfcraft-ocr-worker"
version = "0.18.1"
dependencies = [
"js-sys",
"pdfcraft-ocr",
"wasm-bindgen",
"web-sys",
]
[[package]]
name = "pdfcraft-optimize"
version = "0.18.1"
@@ -4627,10 +4650,12 @@ dependencies = [
"pdfcraft-filters",
"pdfcraft-fonts",
"pdfcraft-forms",
"pdfcraft-geom",
"pdfcraft-model",
"pdfcraft-render",
"pdfcraft-testkit",
"pdfcraft-xfa",
"regex",
"sha2 0.11.1",
"thiserror 2.0.21",
]
@@ -4649,6 +4674,7 @@ dependencies = [
"lopdf",
"pdfcraft-cos",
"pdfcraft-fonts",
"pdfcraft-geom",
"serde",
"skrifa 0.44.0",
"thiserror 2.0.21",
@@ -4676,9 +4702,11 @@ dependencies = [
"p521",
"pbkdf2",
"pdfcraft-annot",
"pdfcraft-content",
"pdfcraft-cos",
"pdfcraft-fonts",
"pdfcraft-model",
"pdfcraft-testkit",
"rc2",
"ripemd",
"rsa",
@@ -4754,11 +4782,13 @@ dependencies = [
name = "pdfcraft-web"
version = "0.18.1"
dependencies = [
"craft-web-api",
"craft-web-host",
"eframe",
"egui",
"js-sys",
"pdfcraft-ui-egui",
"serde_json",
"wasm-bindgen-futures",
"web-sys",
]
@@ -5507,6 +5537,29 @@ dependencies = [
"bitflags 2.13.2",
]
[[package]]
name = "regex"
version = "1.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f020237b6c8eed93db2e2cb53c00c60a8e1bc73da7d073199a1180401450218d"
dependencies = [
"aho-corasick",
"memchr",
"regex-automata",
"regex-syntax",
]
[[package]]
name = "regex-automata"
version = "0.4.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ad8553b9b26413251cbf30e620595c7a41b3887f03da04579c0e6b0d6a06b4b2"
dependencies = [
"aho-corasick",
"memchr",
"regex-syntax",
]
[[package]]
name = "regex-syntax"
version = "0.8.11"
@@ -8209,7 +8262,7 @@ dependencies = [
[[package]]
name = "winprint"
version = "0.2.1"
source = "git+https://github.com/storytold/winprint.rs?rev=5f80b9baf6b58937c401656634e42d33f00ddf49#5f80b9baf6b58937c401656634e42d33f00ddf49"
source = "git+https://github.com/storytold/winprint.rs?rev=51934db382f225454cf152a8657eb03789be63c4#51934db382f225454cf152a8657eb03789be63c4"
dependencies = [
"flate2",
"fmt-derive",
Cargo.toml
+9 -2
View File
@@ -24,7 +24,7 @@ craft-keymap = { git = "https://github.com/storytold/craft-libs", version = "0.1
# the commit in `craft-i18n.rev`). Already in the tree through craft-ui.
craft-i18n = { git = "https://github.com/storytold/craft-libs", version = "0.1.0", rev = "787a7a0f972fd1d7a7618fdbeca3a5267a0407ff" }
# Shared system spooler; document composition remains in pdfcraft-print.
craft-print = { git = "https://github.com/storytold/craft-libs", version = "0.1.0", rev = "787a7a0f972fd1d7a7618fdbeca3a5267a0407ff" }
craft-print = { git = "https://github.com/storytold/craft-libs", version = "0.1.0", rev = "7ee35a9fbd47af572d55fd63dbf0333d1f00da17" }
# Whether a path stored in a document may be opened (GoToR link targets).
craft-docpath = { version = "0.1.0", git = "https://github.com/storytold/craft-libs", rev = "787a7a0f972fd1d7a7618fdbeca3a5267a0407ff" }
# craft-update: app-specific pretend versions apply to startup and manual checks.
@@ -70,7 +70,11 @@ pdfcraft-prc-render = { path = "crates/prc-render", version = "0.18.1" }
hayro = "0.7.1"
hayro-syntax = "0.7.2"
lopdf = "0.45"
flate2 = { version = "1.1", default-features = false, features = ["rust_backend"] }
# zlib-rs: miniz_oxide took about 1.7 µs for every fixed-Huffman block, empty or not
# (`pdfcraft_filters::FLATE_READ_WORK`). Other dependencies still build flate2's default
# miniz_oxide backend; flate2 prefers zlib-rs when both are built, as the Windows build already did
# through craft-update. `flate::tests::a_copy_from_before_the_start_is_damage` shows which runs.
flate2 = { version = "1.1", default-features = false, features = ["zlib-rs"] }
weezl = "0.2"
# CCITT Group 4 encoding (pdfcraft-filters); already in the tree through tiff.
fax = "0.2.7"
@@ -80,6 +84,9 @@ hayro-jbig2 = "0.3"
# foundation
thiserror = "2"
# Linear-time matching (no backtracking) for caller-supplied redaction patterns. `perf-dfa` gives
# the lazy-DFA cache that `RegexBuilder::dfa_size_limit` bounds (the limit is inert without it).
regex = { version = "1", default-features = false, features = ["std", "unicode", "perf-dfa"] }
serde = { version = "1", features = ["derive", "rc"] }
serde_json = "1"
log = "0.4"
ROADMAP.md
+27
View File
@@ -1,8 +1,16 @@
# PdfCraft roadmap
> **Last reviewed:** 2026-11-08 | **Last updated:** 2026-11-08 | **Change:** patch (M10 #6514: refresh craft-print to 7ee35a9f, PDF loader to 51934db3 and public GDI kept at 64e7afd4 after #605 merged; preserve crash fix and stream-test repairs; new-graph CI pending; estimates unchanged)
> **Last reviewed:** 2026-11-07 | **Last updated:** 2026-11-07 | **Change:** patch (M10: file-backed Windows PDF loading without whole-job buffering or per-chunk async waits; approved dependency chain retains #5006; physical output still unqualified; estimates unchanged)
> **Last reviewed:** 2026-11-05 | **Last updated:** 2026-11-05 | **Change:** patch (M7 #2964: directly edit Thai source text inside supported Form XObjects; estimates unchanged)
> **Last reviewed:** 2026-11-03 | **Last updated:** 2026-11-03 | **Change:** patch (M5 #5850 review: indirect typewriter intent and composer edits preserve outer whitespace; movement regression commits by outside click; estimates unchanged)
> **Last reviewed:** 2026-11-03 | **Last updated:** 2026-11-03 | **Change:** patch (M5 #5850: Enter adds Fill & Sign line breaks; outside clicks commit, Escape cancels, and saved multiline appearances retain their text and height; estimates unchanged)
> **Last reviewed:** 2026-10-31 | **Last updated:** 2026-10-31 | **Change:** minor (M3/M4, #5330: Crop pages is directly available in the document quick toolbar; estimates unchanged)
> **Last reviewed:** 2026-11-01 | **Last updated:** 2026-11-01 | **Change:** patch (M7, PR #2155 further review: inherited drawing/text state, affine image gestures, paste/selection ownership, no-op scripts and menu permissions; estimates unchanged)
@@ -229,8 +237,12 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md
| 8 | 1.0 polish: performance, right-to-left, missing languages, keyboard-only (M14) | 55–95 |
## Progress log
- **2026-10-13 (M8, redact by regex):** `redact_mark` takes `regex` (caller expression, linear-time engine, length/size/nesting caps, empty-matching refused) with optional `validate` (luhn, mod97, iban). Headless only; issue #1886.
- **2026-11-07 (M7, #2964 / #3185):** Supported Form text can be prepared for direct editing; groups left grouped (unsafe compositing, text nested past eight levels) are reported. Legacy Thai keeps the original font programs, Regular/Bold and text settings, also across same-face subsets; the narrow THSarabunPSK map repair is checked glyph by glyph against the font program; cross-family replacement is explicit. Builds on #3067 and #2811. Unsafe groups and mixed contextual styles remain partial; estimates unchanged.
- **2026-10-28 (M10, #4119 partial):** Scan & OCR ▸ Enhance scanned file cleans scanned pages in place: background removal (level 0–100, as the camera clean-up), edge shadow removal (book gutters, copier borders) and despeckle. Each scan image is decoded, cleaned and written back into its own object at the same size and colour space (JPEG stays JPEG, others lossless, 1-bit stays 1-bit), so the OCR text layer, comments and links stay; one undo step, shared images once, unsupported images (CMYK, indexed, JPEG 2000, JBIG2, CCITT, broken data) reported. `page_enhance_scan` drives it headlessly. Not yet an option of Recognize text. Estimates unchanged.
- **2026-11-07 (M10, `ocr.ocr-web`, port onto #977):** Web OCR rebuilt on main's engines and preprocessing (#977). `OcrJob::run` is now `OcrJob::steps` a page at a time: the page renders, crops to the region and cleans each page, a Web Worker runs ocrs on the result, and `OcrShot::place` maps the words back through the region and preprocessing. Every strategy reads like this when only one engine is present, and the web build has only ocrs. The downloaded models are installed into the engine cache (`ocr::install`), so the Verify screen and the engine-status checks see them too. The batch keeps main's skipped-files bucket, low-confidence notes and cancel. The translations moved to craft-i18n (storytold/craft-i18n#141). The Verify screen starts the model download itself and runs once the models land; before, on a fresh page it failed with the desktop's "set PDFCRAFT_MODELS" message. Known limits: the cleaning steps, and the Verify screen's Run OCR and re-reads, run on the page. Estimates unchanged.
- **2026-11-03 (M11, shared JBIG2 dictionary):** With Monochrome Images set to JBIG2, Optimize has up to 16 JBIG2 images share one symbol dictionary (`/JBIG2Globals`), the letters found on more than one page stored once, when that is smaller (`pdfcraft_filters::encode_jbig2_shared`). Eight pages of 10 pt text rendered at 300 dpi take 41% less in Helvetica and 42% in Times than coded alone, 10% in Courier; groups of 16 save 45%, of 32 only 47%. Only images redaction can store again without the dictionary share one (image masks and DeviceGray named directly, at most 64 Mi pixels), and redacting one stores the rest of its group so, which the group bounds to 15.
- **2026-11-03 (M5, #5850 review):** Resolve indirect IT when recognizing typewriter contents and Fill & Sign marks. Comment listing and property lookup retain their existing direct-Subtype behavior. A real double-click Edit composer preserves the typewriter’s text as written; ordinary comment edits keep trimming. Regressions cover direct Subtype with indirect intent, save/reopen, raw posted contents and same-index comments on different pages. The existing live-movement test now commits its setup with an outside click rather than Enter. Estimates unchanged.
@@ -256,6 +268,7 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md
- 2026-11-01: PR #2155 review follow-up preserves editor crops through group moves and keeps arranged artwork outside added-item streams; unsafe nested-tag extraction is refused. Image handles and flips follow page orientation, added images accept general transforms, and clipboard/text/Home/font-discovery edge cases have focused regressions. Candidate only; estimates unchanged.
- 2026-11-01: PR #2155 maintainer edits remove redundant image/text helpers and add regressions for crop ownership, arrange scope/state fidelity, clipboard precedence, crop-drag bounds, and font scanning. Transform and Unicode-script tools complete the headless paths.
- **2026-10-31 (M3/M4, #5330):** The document quick toolbar offers Crop pages directly through the existing page.crop command, with the command’s assembly-permission check and an active-tool highlight. The existing crop, undo and headless page_set_box paths are unchanged. No toolbar customization or new shortcut. Estimates unchanged.
- **2026-10-18 (M3, #1354, local candidate):** Integrated --two-paged with current main, retaining upstream modal/picker ownership and single-tab reuse. Native launch/opening/password regressions, negative checks against the old wrapper, visual inspection, formatting, strict scoped Clippy, layers, assets, parity and scoped WASM compilation pass. Three existing socket tests are unavailable in this workspace; one existing network test remains ignored. No push or GitHub merge; estimates unchanged.
- **2026-10-31 (M2, PR #4172 review):** Added same-machine pixel oracles for GBK Song, Hei, Kai and FangSong, including all four subset-prefixed Founder aliases. Reference PDFs use explicit Unicode mappings and independently selected installed faces; font data and rasters remain in memory. All eight cases ran; wrong-glyph and wrong-family production mutations each failed the new assertion and were restored. Validation: 422 renderer unit tests, renderer all-target Clippy with -D warnings, formatting, layers/assets/parity and renderer WASM checks passed; existing optional-patch/WASM warnings and parity notes remain. Estimates unchanged.
@@ -266,6 +279,7 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md
- **2026-10-27 (M2, PR #4172 module-split conflicts):** Integrated main 1dd2737b, retaining viewer-only CJK fallback in split session modules and Combine thumbnail workers, and moving renderer regressions to the new test module. Validation: 798 scoped tests passed (371 engine, 412 renderer, 15 Combine; two existing engine tests ignored); scoped all-target Clippy with -D warnings, formatting, layers, assets, parity and renderer/engine WASM checks passed. Estimates unchanged.
- **2026-10-27 (M10, `ocr.ocr-web`, rebase fix):** The web build loads OCR models only when their SHA-256 matches `ATTRIBUTION.toml`. With ONNX loading now on, rten's parser spins forever on some bytes that aren't a model (a corrupt or wrong download), so a bad file now fails at once instead of hanging a worker.
- **2026-10-27 (M10, OCR models):** Model staging downloads and verifies the ONNX pair after the upstream RTEN URL brownout; runtime discovery prefers complete ONNX pairs per directory, retains complete RTEN fallback and preserves directory precedence.
- **2026-10-26 (M2, PR #4172):** Reconciled current main, preserving its Japanese serif/weight selection and both branches' progress records. Installed CJK viewer and preview fallback remains display-only; estimates unchanged.
@@ -277,6 +291,8 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md
- **2026-10-25 (web, same-document host mode):** A page that loads the PdfCraft web module into its own document (no iframe) can opt into the host bridge with `data-host="page"` on the app's canvas — the exact value `page`; any other value keeps the normal app with downloads, unless a same-origin framed `?host=parent` selects the parent bridge (unframed or cross-origin, `?host=parent` also keeps the normal app). It is the `?host=parent` bridge with the transport swapped: the same five messages travel as non-bubbling `CustomEvent`s on the canvas (`detail` holds the parent messages' fields without `type`; the host answers with a `pdfcraft:open` event), so saves reach the host instead of downloading and dirty and start-up-failure reporting are unchanged. Unit tests cover the opt-in rule; no parity or estimate changes.
- **2026-10-24 (M10, #4158 partial):** Word, HTML, RTF and PowerPoint export keep a block that continues another on its line (an italic case name, a link, a superscript) as a run in its own style inside that paragraph, with the source's word space, instead of a paragraph of its own. Style changes that wrap onto the next line still split paragraphs (the reading blocks keep style boundaries). Estimates unchanged.
- **2026-10-24 (M3, window move on Wayland, #3970 follow-up):** After a move or resize by PdfCraft's own title bar or edges, the button release given back to egui goes before the pointer leaving the window, so hover no longer sticks at the old place until the pointer moves. Found by CodeRabbit in review of #3970; a unit test covers it. Estimates unchanged.
- **2026-10-26 (M10, `ocr.ocr-web`, review fix):** On the web, Edit on a scanned page starts the OCR model download, and the page is read once the models load; it was taken for no scan and left unread until the document changed.
- **2026-10-24 (M10, `ocr.ocr-web`, review fix):** On the web, single-file and multiple-file OCR running at once no longer lose a page: the workers' answers go to the run that sent each page, instead of to whichever run collected first, and a cancelled run's late answers are let go. Regression tests cover both start orders and cancellation. Files chosen for Recognize Text in Multiple Files before the models have downloaded now wait for them instead of having to be chosen again, and idle workers close on time even when nothing is happening on the page, a full minute after their last page comes back rather than after it was handed out (checked by hand in Firefox); estimates unchanged.
- **2026-10-24 (M14, #4253):** The annotation quick bar remembers Keep tool selected across restarts, including unpinning. Old and malformed settings keep safe defaults; the opt-in UI control channel reads and changes the same preference. Estimates unchanged.
- **2026-10-22 (M7, #2517):** A partial rewrite of the last paragraph closes its marker after that paragraph's last `ET`, so a row left in the source stays in the paragraph. When the next paragraph shares the text object, the marker closes before that neighbour instead of after the shared `ET`. The extra marker is not added across two marked-content regions. Estimates unchanged.
- **2026-10-15 (M7, #2517):** A blank line between paragraphs stays a paragraph break. An explicit `TL` is the measured baseline gap. Without one, three or more rows at a repeated step stay one paragraph, and a single wider gap stays a break. Adding a row, even when an earlier row is left in the source, does not absorb the next paragraph on a later edit. Estimates unchanged.
@@ -315,6 +331,7 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md
- **2026-10-22 (M14, macOS association recovery candidate):** Successful desktop open/save/sign can queue recovery of an existing per-file binding to the running PDFCraft bundle through NSWorkspace. Metadata-only watcher changes retain undo and avoid false overwrite prompts; real same-length/same-mtime edits remain protected. Rebased on 0.12.0 and reuses #3042's shared atomic metadata preservation; the bounded recovery parser does not narrow save limits. Prior native Preview TEST-signature and Finder workflows passed on macOS 27.0.1; other versions remain unqualified. Estimates unchanged.
- **2026-10-13 (M7, free image rotation):** A selected page image has a rotate knob (Shift: 15 degree steps) and `image_edit` a `rotate_by` action; the angle is written as one undoable `cm`. Snapping to page edges while moving is in #1740; resize snapping is not done. Estimates unchanged.
- **2026-10-20 (M9 #79 review, M2, M8):** text extraction reads a marked-content /ActualText that marks no glyph where it paints (paths and images, cut to what is visible, along the direction they are drawn), so words drawn as outlines can be found, copied and exported; OCR and scan field detection still count such a page as having no text of its own. Redaction strips /ActualText, /Alt, /E and /T from inline marked content around paint it clips, as it already did around paint it removed, so a partly redacted outlined word leaves no name in the file. Named-layer recognition also needs the layer form to draw n1 and n3. Estimates unchanged.
- **2026-10-20 (M4, initial view):** Window and interface options read indirect boolean ViewerPreferences values. Synthetic regressions cover true/false, direct/indirect preference dictionaries and save/reopen; malformed entries retain false defaults. Estimates unchanged.
- **2026-10-20 (M3, #3535):** Escape with a menu open closes the menu and keeps the text selection; the next Escape clears it, as before. One keypress used to do both. A synthetic UI regression checks it. Estimates unchanged.
- **2026-10-16 (M3, close prompt keyboard):** The "Save changes?" prompt opens with Save chosen (Enter saves, as before) and the arrow keys and Tab / Shift+Tab move the choice to Cancel or Don't save, shown with a focus ring; Enter or Space press the chosen button and Escape still cancels. Before, Enter always saved whatever was Tab-focused. Estimates unchanged.
@@ -405,6 +422,8 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md
- **2026-10-16 (M10, #1201 partial):** Lossless image export applies each component's Decode pair. Indexed values round and clip before palette lookup. Color-key masks keep raw samples. The existing 16-bit precision limit remains. JPEG and mask Decode, ICC transforms, and calibrated colors remain outside this fix. Estimates unchanged.
- **2026-10-16 (M3, #2739):** Wrapped revision buttons stay above Binding in Document Properties. A synthetic regression checks control bounds and opens a saved revision at two viewport widths. Estimates unchanged.
- **2026-10-19 (M7, tracked text):** Editable paragraphs and object labels distinguish repeated glyph tracking from word gaps in TJ arrays and separately positioned glyphs. Synthetic regressions cover character spacing, horizontal scaling, rotated baselines, larger word gaps, and rows of one-glyph words or cells (inferred tracking is capped at half an em, so `a b c` and digit or Y/N columns keep their spaces). Estimates unchanged.
- **2026-10-16 (M9, #79):** a signature appearance that follows the named-layer convention (its /N draws one Form that holds and draws n1 and n3) shows the signature's real state: n1, n3 and n4 are redrawn in the display copy with the panel's icons and a caption in the interface language (Helvetica; else shaped in embedded Inter, or in a craft-fonts face for Arabic, Japanese and Telugu; else English, as Chinese always is). Exports, both print modes and the CLI's render show the redrawn state; saves never do. Every signature keeps its badge, since a document can clip or cover its own layers. Estimates unchanged.
- **2026-10-16 (M9):** the page validation overlay moves to the widget's bottom-right corner as it lies on the page (so it turns with the page and the view) and takes a quarter of the widget's shorter side, at most 48 screen points, so it annotates the signer's artwork from a corner instead of covering the middle of it; a field too small for a legible mark keeps its outline, a rectangle the screen cannot place is not drawn at all, and the whole widget answers the pointer. Estimates unchanged.
- **2026-10-16 (M5, #2669, item 5):** The Comments panel groups replies by name and counts all page roots once per frame. Estimates unchanged.
- **2026-10-16 (M7, #2140):** Header & footer (and Bates numbering, which uses the same dialog) can shrink the page content so the header and footer don't cover it: the original content is scaled evenly into the band they leave free, centred across, also on rotated pages; Remove and Update take the scaling away, and margins that leave no room are refused. Dialog checkbox with a preview of the band, `shrink` on `doc_header_footer`, all catalogs. Annotations and added text and images are not moved yet. Estimates unchanged.
- **2026-10-20 (M14, Spanish catalog gaps):** `es.tsv` gains the 143 `(context, English)` keys the other catalogs carried and it was missing, so Spanish covers the whole bundled key union like Lithuanian and Vietnamese: calendar months and weekdays, the `certificate` context, X.509 key usages, signature-revision and digital-ID lines, password validation, eight plural messages, the combine-from-folder strings, the rendering and stale-file notices and the shortcut-spelled tooltips. Counts in `localization-parity.md` re-measured against the current union (`vi` restated against it for the same reason). Status stays `partial`: no translated help, and OS details stay outside the catalog. Estimates unchanged.
@@ -553,6 +572,8 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md
Newest first: the date, what moved, and the new overall estimate.
- **2026-10-28 (M10, #1595):** Export to Markdown: `doc_export_office` with a `.md` or `.markdown` path writes CommonMark with GitHub pipe tables (headings, paragraphs with bold and italic, tables with their first row as the header, a rule between pages); text that Markdown would read as markup is escaped. Images, lists and links are not in the export's layout model and are left out. No menu item yet. Estimates unchanged.
- **2026-10-12 (M10, `ocr.ocr-web`):** OCR in the browser, partly. The web zip carries the text recognition models in `models/` beside `index.html`; the first time Recognize Text is needed the web build downloads them (about 12 MB, with a spinner, an error and a retry). Pages are rendered on the page and read in Web Workers (up to two at once, so the page keeps responding), built with WebAssembly SIMD: 8.5 s a page instead of 20.5 s in wasmtime, still about 17× native. Where workers can't start, the page reads the pages itself. No automated browser test yet. Estimates unchanged.
- **2026-10-12 (M10, `ocr.ocr-web`):** Web OCR shipped: reading scanned pages in Web Workers was checked by hand in a browser (4-page synthetic scan). Still no automated browser test, and a page reads about 17× slower than native.
- **2026-10-12 (M3, GNOME title bar, #553):** On GNOME's Wayland session, whose compositor draws no window frames (it offers no `zxdg_decoration_manager_v1`), PdfCraft no longer shows winit's fallback title bar: it turns the frame off and the tab strip becomes the header bar, as in GNOME's own apps. Minimize, maximize/restore and close are drawn in Adwaita's style, in the order and on the side GNOME's `button-layout` names (Fedora's default shows only close); the window's edges and corners resize it, and a hairline outlines it while it isn't maximized. Preferences ▸ Use system title bar (shown only there) gives the window back winit's frame at once, without a restart (part of #1001), and `PDFCRAFT_TITLEBAR=app|system` overrides the choice. KDE, Sway, Hyprland, COSMIC and X11 keep the frame their desktop draws. Not done: right-clicking the header bar doesn't open GNOME's window menu (egui can't ask for it), the window has no shadow or rounded corners, the resize edges lie just inside the window rather than outside it, and Windows still uses its system frame (the rest of #1001). Estimates unchanged.
- **2026-10-12 (M0, build label):** A local or patched build can carry a label, set at compile time with `PDFCRAFT_BUILD_LABEL`: `pdfcraft --version`, `pdfcraft-cli --version`, About and the log show `0.7.1 (titlebar)`, and `packaging/linux/package.sh` names the files `pdfcraft-0.7.1-titlebar-…` and makes the .rpm/.deb release `1.titlebar`, which upgrades over the release of the same version (and the next release over it). The version itself is unchanged, so the update check still compares it with published releases. Unlabelled builds are as before. Estimates unchanged.
- **2026-10-11 (M6, #1208):** Dropdown and list-box Field Properties expose the existing Left/Center/Right alignment picker. Existing alignment stays intact until changed; cancel, undo/redo and save/reopen are covered through the real UI without changing options, values or flags. Estimates unchanged.
@@ -788,11 +809,15 @@ Newest first: the date, what moved, and the new overall estimate.
| Date | Change | Summary |
|---|---|---|
| 2026-11-08 | patch | M10 #6514/#636: retain main's public GDI Git revision (64e7afd4) to preserve downstream Rust type identity, update the internal PDF loader independently, and refresh the application dependency/lockfile. New-graph CI pending; estimates unchanged. |
| 2026-11-08 | patch | M10 #6514: refresh the craft-print and winprint/GDI pins after #605 merged; retain the Direct2D crash fix and stream-test cleanup repairs, record single-page HUAWEI output with its diagnostic limits, and rerun CI for the new dependency graph. Estimates unchanged. |
| 2026-11-07 | patch | M10 #6514: load Windows PDF print jobs on demand through a read-only file stream, avoiding StorageFile access denial, whole-file buffering and per-chunk async waits; approved dependencies retain the #5006 crash fix. Regression tests pass; physical output remains unqualified; estimates unchanged. |
| 2026-11-03 | patch | M5 #5850 review: indirect typewriter intent and composer edits preserve outer whitespace; movement regression commits by outside click; estimates unchanged. |
| 2026-11-03 | patch | M5 #5850: Enter adds Fill & Sign line breaks; outside clicks commit, Escape cancels, and saved multiline appearances retain their text and height; estimates unchanged. |
| 2026-11-01 | patch | M7 #5657: header/footer, watermark and background operations commit all selected pages together; later-page errors preserve marks, content, shared/inherited resources and pending edits; estimates unchanged. |
| 2026-10-28 | patch | M2, PR #4172: integrate main 5af10276 and retain both roadmap histories; viewer-only CJK and print/output font policies and estimates unchanged. |
| 2026-10-27 | minor | M10: OCR stages SHA-256-verified ONNX models after the upstream RTEN URL brownout; discovery prefers a complete ONNX pair per directory, falls back to a complete RTEN pair, and never mixes formats or directories; language/web limits and estimates unchanged. |
| 2026-10-20 | minor | M9 #79 review, M2, M8: /ActualText that marks no glyph extracts where it paints, as far as it is visible; redaction strips /ActualText around paint it clips; named-layer recognition needs the layer form to draw n1 and n3. Estimates unchanged. |
| 2026-10-23 | minor | M8, #3205 follow-up: phone-pattern redaction includes the +39 country code on labelled Italian geographic numbers with a slash; dates, protocol identifiers and whole-run guards remain. Estimates unchanged. |
| 2026-10-23 | minor | M11: image downsampling measures physical resolution after page-local UserUnit scaling, including shared nested forms; synthetic measurement and optimization regressions. Estimates unchanged. |
| 2026-10-23 | minor | M10: Word table cells use the grid columns they span, including cells after a merged cell; synthetic export regression and CLI reproduction. Estimates unchanged. |
@@ -807,6 +832,8 @@ Newest first: the date, what moved, and the new overall estimate.
| 2026-10-18 | minor | M1, #1143: a live reference may match the xref generation or a same-number header at its indexed offset; neither-match and free slots remain null. Bounded repair diagnostics and synthetic cache, free-slot, edit and classic/packed save regressions; estimates unchanged. |
| 2026-10-18 | minor | M5, PR #1971: nested source `/Page` and `/Pages` dictionaries export as null without traversing page content or the page tree. Synthetic saved/reopened three-page regression retains unrelated metadata and aliases. FDF import semantics, copied-graph limits and estimates unchanged. |
| 2026-10-15 | minor | M5: budget retained FDF entries before cloning; discovery keeps lazy array owners and annotation positions, with alias/cache-eviction stress fixtures. Limits cover the copied graph; source parsing/page discovery retain existing COS/parser limits, with no total source-memory bound. FDF import semantics and estimates unchanged. |
| 2026-10-16 | minor | M9 #79: named-layer signature appearances show the signature's real state in the display copy, exports and both print modes, never in saves; captions follow the interface language. Estimates unchanged. |
| 2026-10-16 | minor | M9: the page validation overlay moves to the widget's bottom-right corner as it lies on the page and stops growing past 48 screen points, so it annotates the signer's artwork from a corner instead of covering the middle of it; a field under 32 screen points keeps its outline. Estimates unchanged. |
| 2026-10-15 | minor | M7: the Edit text caret is as tall as the row it is on in a region with several sizes; estimates unchanged. |
| 2026-10-18 | minor | M4, PR #1919 review: repeated page indices apply each rotation to the latest page-local value, preserving indirect/inherited angles. Synthetic adjacent/interleaved duplicates retain siblings and page contents through incremental and full saves. Estimates unchanged. |
| 2026-10-18 | minor | M3, #3149: page replacement cancels render demand collected for the previous geometry; all viewer raster upload paths reject empty or oversized textures before cache eviction. Synthetic regressions reproduce the reported 1880×2350 upload at a 2048 limit, retain cached images and unsaved edits, and resume bounded backdrops and tiles. Estimates unchanged. |
apps/pdfcraft-cli/src/commands.rs
+3 -2
View File
@@ -366,8 +366,9 @@ pub(crate) fn render(args: &[String]) -> Result<(), CliError> {
Some("pam") => None,
_ => return Err(format!("render: --out {out}: use a .png, .jpg, .tif or .pam name").into()),
};
// The app renders a display copy that draws comments missing their /AP streams; the CLI
// must agree with it (and with independent viewers), while keeping the file it reads intact.
// The app renders a display copy that draws comments missing their /AP streams and the state
// of named-layer signatures (here with no trusted certificates); the CLI must agree with it
// (and with independent viewers), while keeping the file it reads intact.
let bytes = read(path)?;
let password = flag(args, "--password");
// The renderer takes only the user password of R2–R4 files; `Session::open` and the display
apps/pdfcraft-ocr-worker/Cargo.toml
+18
View File
@@ -0,0 +1,18 @@
[package]
name = "pdfcraft-ocr-worker"
description = "PdfCraft in the browser: the Web Worker that recognises text (wasm32; trunk builds it with pdfcraft-web)"
version.workspace = true
edition.workspace = true
license.workspace = true
publish = false
[lints]
workspace = true
[dependencies]
pdfcraft-ocr = { workspace = true }
[target.'cfg(target_arch = "wasm32")'.dependencies]
wasm-bindgen = "0.2"
js-sys = "0.3"
web-sys = { version = "0.3", features = ["DedicatedWorkerGlobalScope", "MessageEvent"] }
apps/pdfcraft-ocr-worker/src/main.rs
+101
View File
@@ -0,0 +1,101 @@
//! PdfCraft in the browser: the Web Worker that recognises text, so reading a scanned page doesn't
//! stop the page responding and several pages are read at once. Trunk builds it alongside
//! `pdfcraft-web` (see apps/pdfcraft-web/index.html); the page starts and feeds the workers
//! (crates/ui-egui/src/workflows/ocr_ui.rs).
//!
//! Messages are plain objects with a `type`:
//!
//! - `ready` (to the page): the worker is listening; send the models now.
//! - `models` (from the page): `{ detection, recognition }`, each a `Uint8Array` holding a model
//! file. Answered with `loaded`, or `failed` with an `error`.
//! - `page` (from the page): `{ id, width, height, pixels }`, the RGBA pixels of a rendered page
//! as a `Uint8Array`. Answered with `lines` `{ id, lines }`, the recognised lines as
//! `pdfcraft_ocr::lines_to_json` writes them, or `failed` `{ id, error }`.
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
#[cfg(target_arch = "wasm32")]
fn main() {
use wasm_bindgen::{JsCast, closure::Closure};
let Ok(scope) = js_sys::global().dyn_into::<web_sys::DedicatedWorkerGlobalScope>() else {
return;
};
let reply_to = scope.clone();
let mut ocr = None;
let on_message = Closure::<dyn FnMut(web_sys::MessageEvent)>::new(move |e: web_sys::MessageEvent| {
let _ = reply_to.post_message(&worker::handle(&mut ocr, &e.data()));
});
scope.set_onmessage(Some(on_message.as_ref().unchecked_ref()));
// The handler lives as long as the worker.
on_message.forget();
let _ = scope.post_message(&worker::message(&[("type", "ready".into())]));
}
#[cfg(not(target_arch = "wasm32"))]
fn main() {
eprintln!("pdfcraft-ocr-worker runs in a browser: trunk builds it with pdfcraft-web");
}
#[cfg(target_arch = "wasm32")]
mod worker {
use pdfcraft_ocr::Ocr;
use wasm_bindgen::{JsCast, JsValue};
/// The models are about 12 MB together.
const MAX_MODEL_BYTES: u32 = 64 << 20;
/// The renderer keeps pages far smaller; anything larger isn't a page.
const MAX_PIXEL_BYTES: u64 = 1 << 30;
pub fn message(fields: &[(&str, JsValue)]) -> JsValue {
let o = js_sys::Object::new();
for (k, v) in fields {
let _ = js_sys::Reflect::set(&o, &(*k).into(), v);
}
o.into()
}
/// Answer one message from the page.
pub fn handle(ocr: &mut Option<Ocr>, data: &JsValue) -> JsValue {
let get = |k: &str| js_sys::Reflect::get(data, &k.into()).unwrap_or(JsValue::UNDEFINED);
let id = get("id");
let reply = match get("type").as_string().as_deref() {
Some("models") => load(&get("detection"), &get("recognition")).map(|o| {
*ocr = Some(o);
message(&[("type", "loaded".into())])
}),
Some("page") => match ocr.as_ref() {
None => Err("the text recognition models aren't loaded".into()),
Some(o) => recognize(o, &get("width"), &get("height"), &get("pixels"))
.map(|lines| message(&[("type", "lines".into()), ("id", id.clone()), ("lines", lines.into())])),
},
_ => Err("unknown message".into()),
};
reply.unwrap_or_else(|error| message(&[("type", "failed".into()), ("id", id), ("error", error.into())]))
}
fn bytes(v: &JsValue, max: u64) -> Result<Vec<u8>, String> {
let a = v.dyn_ref::<js_sys::Uint8Array>().ok_or("expected a Uint8Array")?;
if u64::from(a.length()) > max {
return Err(format!("{} bytes is too large", a.length()));
}
Ok(a.to_vec())
}
fn load(detection: &JsValue, recognition: &JsValue) -> Result<Ocr, String> {
let max = u64::from(MAX_MODEL_BYTES);
Ocr::from_bytes(bytes(detection, max)?, bytes(recognition, max)?).map_err(|e| e.to_string())
}
fn recognize(ocr: &Ocr, width: &JsValue, height: &JsValue, pixels: &JsValue) -> Result<String, String> {
let side = |v: &JsValue| v.as_f64().filter(|v| v.fract() == 0.0 && (1.0..=f64::from(u32::MAX)).contains(v)).map(|v| v as u32);
let (Some(w), Some(h)) = (side(width), side(height)) else {
return Err("the page size is not a positive whole number".into());
};
let px = bytes(pixels, MAX_PIXEL_BYTES)?;
if u64::from(w).checked_mul(u64::from(h)).and_then(|n| n.checked_mul(4)) != Some(px.len() as u64) {
return Err("the pixels don't match the page size".into());
}
ocr.recognize(&px, w, h).map(|lines| pdfcraft_ocr::lines_to_json(&lines)).map_err(|e| e.to_string())
}
}
apps/pdfcraft-web/Cargo.toml
+4 -1
View File
@@ -12,9 +12,12 @@ workspace = true
[dependencies]
pdfcraft-ui-egui = { workspace = true }
# The host messages (storytold/craft-libs): one contract and one bridge for the web builds.
craft-web-host = { git = "https://github.com/storytold/craft-libs", rev = "2e2312ddaf21bc3d85cd3395d0955c7d239306f6", version = "0.1.0" }
# The page command API (storytold/craft-libs#413): window.pdfcraft over the UI control channel.
craft-web-api = { git = "https://github.com/storytold/craft-libs", rev = "fd5e2feff6ea25bed371a99cc732b3968d621505", version = "0.1.0" }
craft-web-host = { git = "https://github.com/storytold/craft-libs", rev = "fd5e2feff6ea25bed371a99cc732b3968d621505", version = "0.1.0" }
eframe = { workspace = true, features = ["default_fonts", "glow", "persistence", "accesskit"] }
egui = { workspace = true, features = ["default_fonts", "monochrome_emoji_fonts"] }
serde_json = { workspace = true }
[target.'cfg(target_arch = "wasm32")'.dependencies]
wasm-bindgen-futures = "0.4"
apps/pdfcraft-web/Trunk.toml
+7
View File
@@ -9,3 +9,10 @@ filehash = true
[serve]
addresses = ["127.0.0.1"]
port = 8766
# Trunk leaves the OCR worker's files unhashed and unnamed in index.html: hash them and name the
# worker for the page (xtask/src/web.rs).
[[hooks]]
stage = "post_build"
command = "cargo"
command_arguments = ["xtask", "web-worker-link"]
apps/pdfcraft-web/index.html
+2
View File
@@ -5,6 +5,8 @@
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>PdfCraft</title>
<link data-trunk rel="rust" data-wasm-opt="z" />
<!-- Recognises text off the page's thread (crates/ui-egui/src/workflows/ocr_web.rs). -->
<link data-trunk rel="rust" href="../pdfcraft-ocr-worker/Cargo.toml" data-type="worker" data-loader-shim data-bindgen-target="web" data-wasm-opt="3" />
<style>
html, body { margin: 0; height: 100%; overflow: hidden; background: #f1f1f3; }
canvas { width: 100%; height: 100%; display: block; outline: none; }
apps/pdfcraft-web/src/api.rs
+127
View File
@@ -0,0 +1,127 @@
//! `window.pdfcraft`: the page command API (`craft-web-api`, storytold/craft-libs#413) over the app's
//! UI control channel (`pdfcraft_ui_egui::control`), opt-in with `?control`.
//!
//! The crate owns the JavaScript side (`request`, `execute`, `commands`, `docInspect`, `uiInspect`,
//! `screenshot`, as promises with a timeout). This module is PdfCraft's part: a [`Request`] that
//! sends each call through the app's [`ControlClient`] and waits for the reply, and the table that
//! names PdfCraft's methods: its control channel drives the UI, so `commands` is `ui.commands`,
//! `execute` runs a registry command with `ui.command`, `docInspect` is `ui.state` and `uiInspect`
//! is `ui.inspect`.
//!
//! The channel keeps the accessibility tree of every frame, so it is off unless the page asks for it
//! (`?control`), as FigCraft's is. PdfCraft's control requests cannot be withdrawn, so a call the page
//! gave up on may still run (or be rejected) when the app reaches it, as it does for a desktop client that timed out.
use serde_json::{Value, json};
/// The reply of a control request as the protocol's `{ok, result}` / `{ok: false, error}` object.
fn reply_object(reply: Result<Value, String>) -> Value {
match reply {
Ok(result) => json!({"ok": true, "result": result}),
Err(error) => json!({"ok": false, "error": error}),
}
}
/// The params of `ui.command`. `execute(command, params)` sends `{command, params}`; `ui.command`
/// takes `{id}` (and `params` for `ui.panel`). A request that already names an `id` is the page's own
/// `request("ui.command", {id})` and goes through unchanged.
fn command_params(params: Value) -> Value {
match params.get("command").and_then(Value::as_str) {
Some(command) if params.get("id").is_none() => json!({"id": command, "params": params.get("params").cloned().unwrap_or(Value::Null)}),
_ => params,
}
}
#[cfg(target_arch = "wasm32")]
pub use web::install;
#[cfg(target_arch = "wasm32")]
mod web {
use std::rc::Rc;
use std::sync::mpsc::{Receiver, TryRecvError};
use craft_web_api::methods::Methods;
use craft_web_api::page::{PageApi, Request};
use pdfcraft_ui_egui::control::{ControlClient, Reply};
use serde_json::{Value, json};
use wasm_bindgen_futures::JsFuture;
use super::{command_params, reply_object};
/// The app id in `window.<app>`.
const APP: &str = "pdfcraft";
/// How often a waiting call looks for the app's reply, in milliseconds (about a frame).
const POLL_MS: i32 = 16;
/// PdfCraft's control methods behind each function.
fn methods() -> Methods {
Methods {
commands: Some("ui.commands"),
execute: Some("ui.command"),
doc_inspect: Some("ui.state"),
ui_inspect: Some("ui.inspect"),
screenshot: Some("ui.screenshot"),
}
}
/// Put `window.pdfcraft` on the page, sending its calls through `client`. An existing
/// `window.pdfcraft` keeps the functions it already has.
pub fn install(client: ControlClient) {
let request: Request = Rc::new(move |method, params| {
let params = if method == "ui.command" { command_params(params) } else { params };
Box::pin(wait(client.send(&method, params)))
});
if let Err(e) = Rc::new(PageApi::new(APP, methods(), request)).install() {
eframe::web_sys::console::warn_1(&format!("PdfCraft: couldn't install window.{APP}: {e:?}").into());
}
}
/// The app's reply, polled about once a frame.
async fn wait(reply: Receiver<Reply>) -> Value {
loop {
match reply.try_recv() {
Ok(reply) => return reply_object(reply),
Err(TryRecvError::Empty) => sleep(POLL_MS).await,
Err(TryRecvError::Disconnected) => return json!({"ok": false, "error": "the app dropped the request"}),
}
}
}
async fn sleep(ms: i32) {
let promise = js_sys::Promise::new(&mut |resolve, _| {
if let Some(window) = web_sys::window() {
let _ = window.set_timeout_with_callback_and_timeout_and_arguments_0(&resolve, ms);
}
});
let _ = JsFuture::from(promise).await;
}
}
#[cfg(test)]
mod tests {
use serde_json::json;
use super::*;
#[test]
fn execute_becomes_a_ui_command_by_id() {
assert_eq!(command_params(json!({"command": "file.word_count", "params": {"a": 1}})), json!({"id": "file.word_count", "params": {"a": 1}}));
assert_eq!(command_params(json!({"command": "file.save", "params": {}})), json!({"id": "file.save", "params": {}}));
}
#[test]
fn a_request_that_names_an_id_goes_through_unchanged() {
let own = json!({"id": "file.save"});
assert_eq!(command_params(own.clone()), own);
let both = json!({"id": "ui.panel", "command": "x", "params": {"reset": true}});
assert_eq!(command_params(both.clone()), both, "an id wins over a command");
assert_eq!(command_params(json!({})), json!({}));
}
#[test]
fn replies_become_the_protocols_objects() {
assert_eq!(reply_object(Ok(json!({"n": 1}))), json!({"ok": true, "result": {"n": 1}}));
assert_eq!(reply_object(Err("boom".to_string())), json!({"ok": false, "error": "boom"}));
}
}
apps/pdfcraft-web/src/main.rs
+9 -1
View File
@@ -3,6 +3,9 @@
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
#[cfg(any(target_arch = "wasm32", test))]
mod api;
#[cfg(target_arch = "wasm32")]
fn main() {
use pdfcraft_ui_egui::PdfCraftApp;
@@ -45,7 +48,7 @@ fn main() {
});
}
// eframe never raises a close request on a browser reload or tab close, so `guard_quit` never
// prompts; ask the browser to confirm instead while a document has unsaved work (#812).
// prompts; ask the browser to confirm instead while a document has unsaved work or an OCR review is open (#812).
warn_before_unload(app.unsaved_flag.clone());
// Host modes (`host` below, over `craft-web-host`): `?host=parent`, a
// same-origin page embedding PdfCraft in an iframe (such as the Nextcloud
@@ -68,6 +71,11 @@ fn main() {
&"PdfCraft: ?host=parent needs a framing page on the same origin; saves download instead".into(),
);
}
// `?control`: `window.pdfcraft`, the page's command API (`api`). Opt-in: the control
// channel keeps the accessibility tree of every frame.
if query_param("control").is_some() {
api::install(app.attach_control(&cc.egui_ctx));
}
Ok(Box::new(app))
}),
)
apps/pdfcraft/src/main.rs
+1 -1
View File
@@ -333,7 +333,7 @@ fn native_options(integrated: bool, app_frame: bool, renderer: eframe::Renderer)
let mut viewport = egui::ViewportBuilder::default()
.with_title("PdfCraft")
.with_inner_size([1440.0, 920.0])
.with_min_inner_size([820.0, 520.0])
.with_min_inner_size(pdfcraft_ui_egui::MIN_WINDOW_SIZE)
.with_drag_and_drop(true)
// Wayland app id: matches packaging/linux/ai.storyteller.pdfcraft.desktop.
.with_app_id(APP_ID);
contributors/people.toml
+5
View File
@@ -186,3 +186,8 @@ sync_github_name = true
[people.helenkwok]
real_name = "Helen Kwok"
[people.vangel76]
real_name = "Valentin Klein"
display_name = "vangel76"
sync_github_name = false
crates/a11y/Cargo.toml
-1
View File
@@ -14,6 +14,5 @@ workspace = true
pdfcraft-cos = { workspace = true }
pdfcraft-model = { workspace = true }
pdfcraft-content = { workspace = true }
pdfcraft-filters = { workspace = true }
pdfcraft-fonts = { workspace = true }
thiserror.workspace = true
crates/a11y/src/alt.rs
+74 -20
View File
@@ -25,6 +25,8 @@ pub enum AltError {
NotAFigure(u32),
#[error("the page has more content than can be edited")]
TooLarge,
#[error("the figure's content may be in a damaged or unreadable content stream, or after the damage, so nothing was changed")]
Damaged,
#[error("{0}")]
Cos(#[from] pdfcraft_cos::CosError),
}
@@ -52,6 +54,8 @@ pub enum ArtifactError {
ParentTree(String),
#[error("the page has more content than can be edited")]
TooLarge,
#[error("the tag's marked content may be in a damaged or unreadable content stream, or after the damage, so nothing was changed")]
Damaged,
#[error(transparent)]
Cos(#[from] pdfcraft_cos::CosError),
}
@@ -372,7 +376,19 @@ fn content_entries(doc: &Document, page: &Dict) -> Vec<Object> {
}
}
type DecodedStream = (Object, Option<(Stream, Vec<u8>)>);
/// How much of a content stream decoded.
#[derive(Clone, Copy, Debug, PartialEq)]
enum Read {
Whole,
/// Damaged partway: the data is what decoded before the damage. What follows it is unknown,
/// the syntax the later streams start in too.
Damaged,
/// Not at all (a filter this reader lacks, say): the data is empty.
Unread,
}
/// A `/Contents` entry and, for a stream, the stream, its decoded data and how much of it decoded.
type DecodedStream = (Object, Option<(Stream, Vec<u8>, Read)>);
/// Decode the page's content streams within the page decode budget. `None` when a stream is
/// rejected because the budget is exhausted.
@@ -383,14 +399,14 @@ fn decode_page_streams(doc: &Document, page: &Dict) -> Option<Vec<DecodedStream>
for entry in entries {
let decoded = match &*doc.resolve(&entry) {
Object::Stream(s) => {
let data = match budget.decode(s) {
Ok(data) => data,
let (data, read) = match budget.decode_with_damage(doc, s, usize::MAX) {
Ok((data, damaged)) => (data, if damaged { Read::Damaged } else { Read::Whole }),
// Content past the budget may hold more of the tag: editing the rest
// would leave that part marked as a tag the tree no longer has.
Err(_) if budget.exhausted() => return None,
Err(_) => Vec::new(),
Err(_) => (Vec::new(), Read::Unread),
};
Some((s.clone(), data))
Some((s.clone(), data, read))
}
_ => None,
};
@@ -403,6 +419,18 @@ fn marked_id(doc: &Document, props: &Dict, op: &pdfcraft_content::Op, ids: &Hash
(op.op == b"BDC").then(|| bdc_mcid(doc, props, op)).flatten().filter(|n| ids.contains(n))
}
/// What [`artifact_marked_content`] did.
#[derive(Debug, PartialEq)]
enum Marked {
/// The streams holding the content were rewritten.
Done,
/// None of the content is on the page, or with `require_all` not all of it: nothing changed.
NotFound,
/// Some of it is, or may be, in a stream damaged partway or one that doesn't decode, neither of
/// which can be rewritten without losing what wasn't read, or after the damage: nothing changed.
Damaged,
}
/// Turn the tag's marked-content blocks into artifacts, rewriting only the streams that
/// contain them. Other content streams, and unknown entries on a rewritten stream, stay.
/// With `require_all`, every id in `ids` must appear on the page before any stream is rewritten.
@@ -412,25 +440,45 @@ fn artifact_marked_content(
streams: Vec<DecodedStream>,
ids: &HashSet<i64>,
require_all: bool,
) -> Result<bool, pdfcraft_cos::CosError> {
) -> Result<Marked, pdfcraft_cos::CosError> {
let props = page_properties(doc, &page.dict);
let decoded: Vec<&[u8]> = streams.iter().filter_map(|(_, s)| s.as_ref().map(|(_, bytes)| bytes.as_slice())).collect();
// The pieces are the streams, in order.
let decoded: Vec<&[u8]> = streams.iter().filter_map(|(_, s)| s.as_ref().map(|(_, bytes, _)| bytes.as_slice())).collect();
let read: Vec<Read> = streams.iter().filter_map(|(_, s)| s.as_ref().map(|(_, _, read)| *read)).collect();
let damage = read.iter().position(|r| *r == Read::Damaged);
let pieces = pdfcraft_content::Pieces::join(&decoded);
let mut found = HashSet::new();
let mut edits = Vec::new();
for op in pieces.parse().ops {
if let Some(id) = marked_id(doc, &props, &op, ids) {
// What follows the damage is unknown: a string or comment the damage left open runs
// on into the next stream, so what reads as an operator after it may not be one. An
// operator whose operands start before a stream not read whole and whose keyword comes
// after it takes them from around what wasn't read: that stream may end them itself.
let (first, last) = pieces.pieces_of(&op);
if damage.is_some_and(|d| last > d) || read.get(first..last).is_some_and(|r| r.iter().any(|r| *r != Read::Whole)) {
return Ok(Marked::Damaged);
}
found.insert(id);
edits.push((op.span, b"/Artifact BMC".to_vec()));
}
}
if if require_all { found != *ids } else { found.is_empty() } {
return Ok(false);
// Content not found may be past the damage, or in a stream that doesn't decode: whether it
// is on the page isn't known.
if found != *ids && read.iter().any(|r| *r != Read::Whole) {
return Ok(Marked::Damaged);
}
let mut spliced = pieces.splice(edits).into_iter();
if if require_all { found != *ids } else { found.is_empty() } {
return Ok(Marked::NotFound);
}
let spliced = pieces.splice(edits);
if read.iter().zip(&decoded).zip(&spliced).any(|((read, old), new)| *read != Read::Whole && *old != new.as_slice()) {
return Ok(Marked::Damaged);
}
let mut spliced = spliced.into_iter();
let mut next = Vec::with_capacity(streams.len());
for (entry, stream) in streams {
let Some((stream, old)) = stream else {
let Some((stream, old, _)) = stream else {
next.push(entry);
continue;
};
@@ -456,7 +504,7 @@ fn artifact_marked_content(
_ => Object::Array(next),
};
doc.update_dict(page.obj, |p| p.set(b"Contents".to_vec(), contents))?;
Ok(true)
Ok(Marked::Done)
}
/// Decorative figure: its marked content on the page becomes an artifact, and the element
@@ -468,11 +516,15 @@ pub fn mark_decorative(doc: &mut Document, obj: ObjRef) -> Result<(), AltError>
let pg = d.get(b"Pg").and_then(Object::as_ref).or_else(|| elements(doc).into_iter().find(|e| e.object == obj).and_then(|e| e.page));
if let Some(page) = pg.and_then(|p| pages.iter().find(|x| x.obj == p)).filter(|_| !ids.is_empty()) {
let streams = decode_page_streams(doc, &page.dict).ok_or(AltError::TooLarge)?;
// The parent tree no longer points those ids at the figure.
if artifact_marked_content(doc, page, streams, &ids, false)?
&& let Some(key) = page.dict.get(b"StructParents").and_then(Object::as_int)
{
clear_parent_tree(doc, key, obj)?;
match artifact_marked_content(doc, page, streams, &ids, false)? {
// The parent tree no longer points those ids at the figure.
Marked::Done => {
if let Some(key) = page.dict.get(b"StructParents").and_then(|o| doc.resolve(o).as_int()) {
clear_parent_tree(doc, key, obj)?;
}
}
Marked::NotFound => {}
Marked::Damaged => return Err(AltError::Damaged),
}
}
// Out of its parent's kids.
@@ -513,10 +565,12 @@ pub fn mark_tag_artifact(doc: &mut Document, obj: ObjRef) -> Result<(), Artifact
let parent = element.dict.get(b"P").and_then(Object::as_ref).ok_or(ArtifactError::NoParent)?;
let replacement = replacement_kids(doc, parent, obj)?;
let streams = decode_page_streams(doc, &page.dict).ok_or(ArtifactError::TooLarge)?;
if !artifact_marked_content(doc, page, streams, &ids, true)? {
return Err(ArtifactError::ContentNotFound);
match artifact_marked_content(doc, page, streams, &ids, true)? {
Marked::Done => {}
Marked::NotFound => return Err(ArtifactError::ContentNotFound),
Marked::Damaged => return Err(ArtifactError::Damaged),
}
if let Some(key) = page.dict.get(b"StructParents").and_then(Object::as_int) {
if let Some(key) = page.dict.get(b"StructParents").and_then(|o| doc.resolve(o).as_int()) {
clear_parent_tree(doc, key, obj).map_err(|e| ArtifactError::ParentTree(e.to_string()))?;
}
doc.update_dict(parent, |p| p.set(b"K".to_vec(), replacement))?;
crates/a11y/src/autotag.rs
+17 -11
View File
@@ -4,7 +4,7 @@
//! which can then be retagged with the existing tag-property tool after human review.
use pdfcraft_content::{Op, parse, serialize_ops};
use pdfcraft_cos::{Dict, Document, ObjRef, Object, Stream};
use pdfcraft_cos::{DecodeBudget, Dict, Document, ObjRef, Object, Stream};
const MAX_PAGES: usize = 1_000;
const MAX_PAGE_BYTES: usize = 16 * 1024 * 1024;
@@ -57,18 +57,24 @@ fn content_bytes(doc: &Document, page: &pdfcraft_model::Page, index: usize) -> R
let Object::Stream(stream) = &*value else { return Err(unsupported(index, "contents include a non-stream object")) };
let remaining = MAX_PAGE_BYTES.saturating_sub(all.len().saturating_add(1));
if remaining == 0 {
return Err(unsupported(index, "decoded content exceeds 16 MiB"));
return Err(unsupported(index, "decoding the content takes more than 16 MiB"));
}
let chain = stream.filters();
let decoded = if chain.is_empty() {
if stream.raw.len() > remaining {
return Err(unsupported(index, "decoded content exceeds 16 MiB"));
// Strictly, and with `/DecodeParms` given by reference: the content is written back from it.
let mut budget = DecodeBudget::new(remaining, 0);
let decoded = budget.decode_strict(doc, stream).map_err(|e| {
// The allowance counts what every filter stage decodes, as a page's does.
if budget.remaining() == 0 {
unsupported(index, "decoding the content takes more than 16 MiB")
} else {
// The decoder's own reason: `CosError::Filter`'s text would say again that the data
// couldn't be decoded.
let why = match e {
pdfcraft_cos::CosError::Filter(why) => why,
other => other.to_string(),
};
unsupported(index, format!("content cannot be decoded strictly: {why}"))
}
stream.raw.to_vec()
} else {
pdfcraft_filters::decode(&chain, &stream.raw, remaining)
.map_err(|e| unsupported(index, format!("content cannot be decoded strictly: {e}")))?
};
})?;
all.extend_from_slice(&decoded);
all.push(b' ');
}
crates/a11y/src/tests.rs
+241
View File
@@ -79,6 +79,68 @@ fn autotag_text_writes_marked_content_and_parent_tree_across_pages() {
assert_eq!(status(&check(&reopened, &Options::default()), Rule::TaggedContent), Status::Passed);
}
/// Page content behind a predictor whose `/DecodeParms` are given by reference is read through
/// them: autotagging writes the text back as it was, marked, not with the predictor's tag bytes in
/// it (a `\0` every eight bytes, which split `24` into `2 4`).
#[test]
fn autotag_text_reads_content_with_parameters_given_by_reference() {
use pdfcraft_cos::{Dict, ObjRef};
let mut content = b"BT /F1 24 Tf 72 700 Td (Hello World) Tj ET".to_vec();
content.resize(content.len().div_ceil(8) * 8, b' ');
// PNG rows of eight bytes, each tagged 0 (no prediction).
let rows: Vec<u8> = content.chunks(8).flat_map(|row| std::iter::once(0).chain(row.iter().copied())).collect();
let mut doc = pdf(
&[
"<< /Type /Catalog /Pages 2 0 R >>".into(),
"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".into(),
"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 600 800] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >>".into(),
stream(""),
FONT.into(),
],
"",
);
let parms: Dict = [(b"Predictor".to_vec(), Object::Int(12)), (b"Columns".to_vec(), Object::Int(8))].into_iter().collect();
let parms = doc.add(Object::Dict(parms));
let mut predicted = pdfcraft_cos::Stream::flate(Dict::new(), &rows);
predicted.dict.set(b"DecodeParms".to_vec(), Object::Ref(parms));
doc.set(ObjRef::new(4, 0), predicted);
assert_eq!(autotag_text(&mut doc).unwrap(), AutoTagResult { pages: 1, paragraphs: 1 });
let page = &pdfcraft_model::pages(&doc)[0];
let written = doc.resolve(page.dict.get(b"Contents").unwrap());
let Object::Stream(written) = &*written else { panic!("page content") };
let text = String::from_utf8_lossy(&written.decoded_strict().unwrap()).into_owned();
assert!(text.contains("/F1 24 Tf") && text.contains("(Hello World) Tj"), "{text}");
}
/// Autotagging refuses a page whose content it can't read whole, and says why: content past
/// 16 MiB, or content that doesn't decode strictly, with the decoder's reason right after.
#[test]
fn autotag_text_says_why_it_cannot_read_a_page() {
use pdfcraft_cos::{Dict, ObjRef, Stream};
let reason = |content: Stream| {
let mut doc = pdf(
&[
"<< /Type /Catalog /Pages 2 0 R >>".into(),
"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".into(),
"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 600 800] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >>".into(),
stream(""),
FONT.into(),
],
"",
);
doc.set(ObjRef::new(4, 0), content);
match autotag_text(&mut doc) {
Err(AutoTagError::Unsupported { reason, .. }) => reason,
other => panic!("{other:?}"),
}
};
assert_eq!(reason(Stream::from_raw(Dict::new(), vec![b' '; 17 << 20])), "decoding the content takes more than 16 MiB");
let mut corrupt = Stream::from_raw(Dict::new(), b"not zlib data".to_vec());
corrupt.dict.set(b"Filter".to_vec(), Object::name("FlateDecode"));
let why = reason(corrupt);
assert!(why.starts_with("content cannot be decoded strictly: FlateDecode: "), "{why}");
}
#[test]
fn autotag_text_refuses_paint_and_does_not_modify_the_document() {
let mut doc = pdf(
@@ -1087,6 +1149,185 @@ fn mark_tag_artifact_validates_parent_and_all_mcids_before_mutating_document() {
assert_eq!(*absent.get(ObjRef::new(9, 0)), before_pt);
}
/// zlib data that reads as `readable` and is then damaged, with more of the stream after the
/// damage: a stored block, then a block of the reserved type (RFC 1951 §3.2.3), then `rest`.
fn damaged_flate(readable: &[u8], rest: &[u8]) -> pdfcraft_cos::Stream {
let len = u16::try_from(readable.len()).unwrap();
let mut raw = vec![0x78, 0x01, 0x00];
raw.extend_from_slice(&len.to_le_bytes());
raw.extend_from_slice(&(!len).to_le_bytes());
raw.extend_from_slice(readable);
raw.push(0x07);
raw.extend_from_slice(rest);
let mut dict = pdfcraft_cos::Dict::new();
dict.set(b"Filter".to_vec(), pdfcraft_cos::Object::name("FlateDecode"));
pdfcraft_cos::Stream::from_raw(dict, raw)
}
/// A tag's content in a stream damaged partway stays tagged: rewriting the stream from what
/// decoded would lose what follows the damage. Nothing changes, the structure tree and the stream
/// included. Content in a whole stream before a damaged one is marked, and the damaged one kept
/// as stored.
#[test]
fn a_tags_content_in_a_stream_damaged_partway_is_not_rewritten() {
use pdfcraft_cos::{ObjRef, Object};
let content = b"/Figure << /MCID 0 >> BDC q 40 0 0 20 10 150 cm /Im Do Q EMC /Photo << /MCID 1 >> BDC 100 100 50 30 re f EMC\n";
let damaged = damaged_flate(content, b"0 0 1 rg 0 0 10 10 re f");
let mut doc = figures_doc();
doc.set(ObjRef::new(4, 0), damaged.clone());
let contents = |doc: &Document| doc.get(ObjRef::new(3, 0)).as_dict().unwrap().get(b"Contents").cloned();
let state = |doc: &Document| (contents(doc), doc.get(ObjRef::new(9, 0)), doc.get(ObjRef::new(6, 0)), doc.get(ObjRef::new(4, 0)));
let before = state(&doc);
assert_eq!(figures(&doc).len(), 2, "read up to the damage");
assert_eq!(mark_tag_artifact(&mut doc, ObjRef::new(7, 0)), Err(ArtifactError::Damaged));
assert_eq!(mark_decorative(&mut doc, ObjRef::new(7, 0)), Err(AltError::Damaged));
assert_eq!(state(&doc), before);
let mut doc = figures_doc();
let other = doc.add(damaged_flate(b"0 g\n", b"0 0 1 rg 0 0 10 10 re f"));
doc.update_dict(ObjRef::new(3, 0), |p| p.set(b"Contents".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(4, 0)), Object::Ref(other)])))
.unwrap();
mark_tag_artifact(&mut doc, ObjRef::new(7, 0)).unwrap();
let Some(Object::Array(list)) = contents(&doc) else { panic!("contents {:?}", contents(&doc)) };
assert_ne!(list.first(), Some(&Object::Ref(ObjRef::new(4, 0))), "the tag's stream is rewritten");
assert_eq!(list.get(1), Some(&Object::Ref(other)), "kept as stored");
}
/// What follows the damage is unknown, the syntax the next stream starts in too, so a tag's
/// content in a stream after a damaged one stays tagged. Here a comment the damage leaves open
/// runs on into the next stream, up to its first end of line, and a string starting there reads
/// as operators, among them what looks like the figure's content: rewritten, it would change the
/// string. Nothing changes, the structure tree included.
#[test]
fn a_tags_content_after_a_damaged_stream_is_not_rewritten() {
use pdfcraft_cos::{Dict, ObjRef, Object, Stream};
let in_a_string = b"(a note\n/Figure << /MCID 0 >> BDC 0 g EMC\n) pop /Photo << /MCID 1 >> BDC 100 100 50 30 re f EMC";
for (damaged, after) in [(b"0 g\n".as_slice(), None), (b"0 g % a comment".as_slice(), Some(in_a_string.as_slice()))] {
let mut doc = figures_doc();
if let Some(after) = after {
doc.set(ObjRef::new(4, 0), Stream::from_raw(Dict::new(), after.to_vec()));
}
let first = doc.add(damaged_flate(damaged, b"\n0 0 1 rg 0 0 10 10 re f"));
doc.update_dict(ObjRef::new(3, 0), |p| p.set(b"Contents".to_vec(), Object::Array(vec![Object::Ref(first), Object::Ref(ObjRef::new(4, 0))])))
.unwrap();
let state = |doc: &Document| {
(doc.get(ObjRef::new(3, 0)).as_dict().unwrap().get(b"Contents").cloned(), doc.get(ObjRef::new(9, 0)), doc.get(ObjRef::new(6, 0)))
};
let before = state(&doc);
assert_eq!(mark_tag_artifact(&mut doc, ObjRef::new(7, 0)), Err(ArtifactError::Damaged), "{damaged:?}");
assert_eq!(mark_decorative(&mut doc, ObjRef::new(7, 0)), Err(AltError::Damaged), "{damaged:?}");
assert_eq!(state(&doc), before, "{damaged:?}");
}
}
/// A page may split its content between an operator's operands and its keyword. When the operands
/// of the figure's `BDC` end the stream before one damaged from its start or one that doesn't
/// decode, the `BDC` the page reads is in what wasn't read: joined without it, those operands and
/// the next tag's read as one `BDC`, which reaches into the stream after it. Neither marking
/// changes anything, the structure tree included.
#[test]
fn a_tags_operands_before_a_stream_not_read_whole_are_not_rewritten() {
use pdfcraft_cos::{Dict, ObjRef, Object, Stream};
let rest = b" BDC q 40 0 0 20 10 150 cm /Im Do Q EMC";
for damaged in [true, false] {
let mut doc = figures_doc();
doc.set(ObjRef::new(4, 0), Stream::from_raw(Dict::new(), b"/Figure << /MCID 0 >>".to_vec()));
let hidden = if damaged {
damaged_flate(b"", rest)
} else {
let filter = doc.add(Object::name("FlateDecode"));
let mut s = Stream::flate(Dict::new(), rest);
s.dict.set(b"Filter".to_vec(), Object::Ref(filter));
s
};
let hidden = doc.add(hidden);
let after = doc.add(Object::Stream(Stream::from_raw(Dict::new(), b"/Photo << /MCID 1 >> BDC 100 100 50 30 re f EMC".to_vec())));
let listed = Object::Array(vec![Object::Ref(ObjRef::new(4, 0)), Object::Ref(hidden), Object::Ref(after)]);
doc.update_dict(ObjRef::new(3, 0), |p| p.set(b"Contents".to_vec(), listed)).unwrap();
let state = |doc: &Document| {
(doc.get(ObjRef::new(3, 0)).as_dict().unwrap().get(b"Contents").cloned(), doc.get(ObjRef::new(9, 0)), doc.get(ObjRef::new(6, 0)))
};
let before = state(&doc);
assert_eq!(mark_tag_artifact(&mut doc, ObjRef::new(7, 0)), Err(ArtifactError::Damaged), "damaged: {damaged}");
assert_eq!(mark_decorative(&mut doc, ObjRef::new(7, 0)), Err(AltError::Damaged), "damaged: {damaged}");
assert_eq!(state(&doc), before, "damaged: {damaged}");
}
}
/// A tag's content not found on the page may be in a stream that doesn't decode: it isn't known
/// to be off the page, so neither marking changes anything, the structure tree included.
#[test]
fn a_tags_content_not_found_beside_a_stream_that_does_not_decode_is_not_taken_for_absent() {
use pdfcraft_cos::{Dict, ObjRef, Object, Stream};
let mut doc = figures_doc();
doc.update_dict(ObjRef::new(7, 0), |d| d.set(b"K".to_vec(), Object::Int(2))).unwrap();
let mut dict = Dict::new();
dict.set(b"Filter".to_vec(), Object::name("NoSuchFilter"));
let other = doc.add(Object::Stream(Stream::from_raw(dict, b"/Figure << /MCID 2 >> BDC 0 g EMC".to_vec())));
doc.update_dict(ObjRef::new(3, 0), |p| p.set(b"Contents".to_vec(), Object::Array(vec![Object::Ref(other), Object::Ref(ObjRef::new(4, 0))])))
.unwrap();
let state = |doc: &Document| {
(doc.get(ObjRef::new(3, 0)).as_dict().unwrap().get(b"Contents").cloned(), doc.get(ObjRef::new(9, 0)), doc.get(ObjRef::new(6, 0)))
};
let before = state(&doc);
assert_eq!(mark_decorative(&mut doc, ObjRef::new(7, 0)), Err(AltError::Damaged));
assert_eq!(mark_tag_artifact(&mut doc, ObjRef::new(7, 0)), Err(ArtifactError::Damaged));
assert_eq!(state(&doc), before);
}
/// A tag's content not found before the damage in a damaged stream may be past it: it isn't known
/// to be off the page, so neither marking changes anything.
#[test]
fn a_tags_content_past_the_damage_is_not_taken_for_absent() {
use pdfcraft_cos::ObjRef;
let damaged =
damaged_flate(b"/Photo << /MCID 1 >> BDC 100 100 50 30 re f EMC\n", b"/Figure << /MCID 0 >> BDC q 40 0 0 20 10 150 cm /Im Do Q EMC");
let mut doc = figures_doc();
doc.set(ObjRef::new(4, 0), damaged);
let contents = |doc: &Document| doc.get(ObjRef::new(3, 0)).as_dict().unwrap().get(b"Contents").cloned();
let before = (contents(&doc), doc.get(ObjRef::new(9, 0)), doc.get(ObjRef::new(6, 0)));
assert_eq!(mark_decorative(&mut doc, ObjRef::new(7, 0)), Err(AltError::Damaged));
assert_eq!(mark_tag_artifact(&mut doc, ObjRef::new(7, 0)), Err(ArtifactError::Damaged));
assert_eq!((contents(&doc), doc.get(ObjRef::new(9, 0)), doc.get(ObjRef::new(6, 0))), before);
}
/// A page's `/StructParents` given by reference is read through it: the parent tree no longer
/// points the marked content at the tag.
#[test]
fn struct_parents_given_by_reference_are_read() {
use pdfcraft_cos::{ObjRef, Object};
let by_reference = || {
let mut doc = figures_doc();
let key = doc.add(Object::Int(0));
doc.update_dict(ObjRef::new(3, 0), |p| p.set(b"StructParents".to_vec(), Object::Ref(key))).unwrap();
doc
};
let nums = |doc: &Document| doc.get(ObjRef::new(9, 0)).as_dict().and_then(|d| d.get(b"Nums").cloned());
let mut doc = by_reference();
mark_decorative(&mut doc, ObjRef::new(7, 0)).unwrap();
assert_eq!(nums(&doc), Some(Object::Array(vec![Object::Int(0), Object::Array(vec![Object::Null, Object::Ref(ObjRef::new(8, 0))])])));
let mut doc = by_reference();
mark_tag_artifact(&mut doc, ObjRef::new(8, 0)).unwrap();
assert_eq!(nums(&doc), Some(Object::Array(vec![Object::Int(0), Object::Array(vec![Object::Ref(ObjRef::new(7, 0)), Object::Null])])));
}
/// A content stream that doesn't decode at all is kept as stored, and a tag in the page's other
/// streams is still marked: only running past the decode budget refuses.
#[test]
fn a_tag_beside_a_stream_that_does_not_decode_is_marked() {
use pdfcraft_cos::{Dict, ObjRef, Object, Stream};
let mut doc = figures_doc();
let mut dict = Dict::new();
dict.set(b"Filter".to_vec(), Object::name("NoSuchFilter"));
let other = doc.add(Object::Stream(Stream::from_raw(dict, b"0 g".to_vec())));
doc.update_dict(ObjRef::new(3, 0), |p| p.set(b"Contents".to_vec(), Object::Array(vec![Object::Ref(other), Object::Ref(ObjRef::new(4, 0))])))
.unwrap();
mark_tag_artifact(&mut doc, ObjRef::new(7, 0)).unwrap();
let contents = doc.get(ObjRef::new(3, 0)).as_dict().unwrap().get(b"Contents").cloned();
let Some(Object::Array(list)) = contents else { panic!("contents {contents:?}") };
assert_eq!(list.first(), Some(&Object::Ref(other)), "kept as stored");
}
fn ref_(n: u32) -> Option<pdfcraft_cos::ObjRef> {
Some(pdfcraft_cos::ObjRef::new(n, 0))
}
crates/annot/Cargo.toml
+2
View File
@@ -11,6 +11,8 @@ workspace = true
[dependencies]
pdfcraft-cos = { workspace = true }
pdfcraft-fonts = { workspace = true }
# The usable-quad rule `/QuadPoints` readers share.
pdfcraft-geom = { workspace = true }
pdfcraft-model = { workspace = true }
thiserror = { workspace = true }
crates/annot/src/appearance.rs
+27 -14
View File
@@ -1,11 +1,12 @@
//! Appearance streams (§12.5.5) for the comment types PdfCraft creates, execution plan M5.2.
//!
//! [`build`] draws a normal appearance (`/AP /N`) from the annotation dictionary alone, so the
//! same code serves new comments and restyled ones. Drawings are in page space with
//! [`build`] draws a normal appearance (`/AP /N`) from the annotation dictionary and the
//! objects it points at, so the same code serves new comments and restyled ones. Drawings are in page space with
//! `/BBox = /Rect` (identity matrix), except note icons, which are drawn in a 20 × 20 box.
//! It returns `None` for anything it cannot draw faithfully (cloudy borders, unknown line
//! endings, indirect geometry), so callers never replace an appearance with a worse one.
//! Callers that hold the document follow references first (`draw::with_direct_values`).
//! endings), so callers never replace an appearance with a worse one. It resolves `/QuadPoints`
//! itself; for the other values, callers that hold the document follow references first
//! (`draw::with_direct_values`).
//!
//! The note icons are PdfCraft's own drawings (AGENTS.md §1). Text boxes use the standard
//! Helvetica font with WinAnsi encoding; line breaking uses [`text_width`], an approximation of
@@ -15,7 +16,7 @@ use pdfcraft_cos::{Dict, Object, PdfString, Stream};
use pdfcraft_fonts::{first_undrawable, literal, win_ansi};
pub use pdfcraft_fonts::{helvetica_width as text_width, wrap};
use crate::{NOTE_SIZE, Rgb, n};
use crate::{Document, NOTE_SIZE, Rgb, n};
mod symbols;
@@ -167,6 +168,18 @@ fn nums(d: &Dict, key: &[u8]) -> Option<Vec<f64>> {
d.get(key)?.as_array()?.iter().map(|o| o.as_f64()).collect()
}
/// The usable `/QuadPoints` quads, by the rule redaction apply reads them with
/// ([`pdfcraft_geom::quad_points::usable_quads`]): a malformed or non-finite quad is not drawn (it
/// would write `NaN` into the stream) and the others still are. The array and its entries are
/// resolved first, as the other readers resolve them, so an indirect quad is drawn too. `None`
/// when none is usable.
fn quads(context: &AppearanceContext<'_>) -> Option<Vec<[f64; 8]>> {
let doc = context.doc;
let q =
pdfcraft_geom::quad_points::usable_quads(doc.resolve(context.dict.get(b"QuadPoints")?).as_array()?.iter().map(|x| doc.resolve(x).as_f64()));
(!q.is_empty()).then_some(q)
}
/// `/C`-style colour arrays: none (transparent), gray, RGB or CMYK.
fn color(d: &Dict, key: &[u8]) -> Option<Option<Rgb>> {
let c = match d.get(key) {
@@ -449,6 +462,7 @@ fn caret_layout(d: &Dict, rect: [f64; 4]) -> ([f64; 4], Option<(f64, f64, f64)>)
#[derive(Clone, Copy)]
struct AppearanceContext<'a> {
doc: &'a Document,
dict: &'a Dict,
subtype: &'a [u8],
rect: [f64; 4],
@@ -485,8 +499,8 @@ impl AppearanceDrawing {
/// Draw the normal appearance of an annotation, or `None` if this subtype/variant isn't supported.
/// Text the standard fonts can't draw comes out as `?`: [`undrawable_text`] says which, and
/// [`crate::set_appearance`] marks the annotation with it (`pdfcraft_fonts::UNDRAWN_MARK`).
pub fn build(d: &Dict) -> Option<Stream> {
let context = appearance_context(d)?;
pub fn build(doc: &Document, d: &Dict) -> Option<Stream> {
let context = appearance_context(doc, d)?;
let drawing = AppearanceDrawing::new(&context);
match context.subtype {
b"Text" => return draw_icon(&context, "Note", [1.0, 0.82, 0.0], drawing),
@@ -500,7 +514,7 @@ pub fn build(d: &Dict) -> Option<Stream> {
finish_appearance(&context, drawing)
}
fn appearance_context(d: &Dict) -> Option<AppearanceContext<'_>> {
fn appearance_context<'a>(doc: &'a Document, d: &'a Dict) -> Option<AppearanceContext<'a>> {
// Imported measurement appearances may include leaders, captions and formatting that
// this builder cannot reproduce. Preserve them rather than silently losing detail.
if d.contains(b"Measure") && !d.contains(b"PCMeasureValue") {
@@ -515,6 +529,7 @@ fn appearance_context(d: &Dict) -> Option<AppearanceContext<'_>> {
return None;
}
Some(AppearanceContext {
doc,
dict: d,
subtype,
rect,
@@ -562,13 +577,12 @@ fn finish_appearance(context: &AppearanceContext<'_>, mut drawing: AppearanceDra
}
fn draw_markup(context: &AppearanceContext<'_>, drawing: &mut AppearanceDrawing) -> Option<()> {
let d = context.dict;
let subtype = context.subtype;
let stroke = context.stroke;
let c = &mut drawing.content;
let q = nums(d, b"QuadPoints").filter(|q| !q.is_empty() && q.len() % 8 == 0)?;
let q = quads(context)?;
let col = stroke?;
for quad in q.as_chunks::<8>().0.iter() {
for quad in q.iter() {
let p = |i: usize| (quad[2 * i], quad[2 * i + 1]);
let (p1, p2, p3, p4) = (p(0), p(1), p(2), p(3));
let h = (p1.0 - p3.0).hypot(p1.1 - p3.1);
@@ -609,14 +623,13 @@ fn draw_markup(context: &AppearanceContext<'_>, drawing: &mut AppearanceDrawing)
}
fn draw_redaction(context: &AppearanceContext<'_>, drawing: &mut AppearanceDrawing) -> Option<()> {
let d = context.dict;
let stroke = context.stroke;
let c = &mut drawing.content;
// While marked: the outline of each area (the fill comes when applied).
let q = nums(d, b"QuadPoints").filter(|q| !q.is_empty() && q.len() % 8 == 0)?;
let q = quads(context)?;
let col = stroke.unwrap_or([0.89, 0.13, 0.13]);
c.push_str(&format!("{}1 w\n", rg_stroke(col)));
for quad in q.as_chunks::<8>().0.iter() {
for quad in q.iter() {
c.push_str(&format!(
"{} {} m {} {} l {} {} l {} {} l h S\n",
n(quad[0]),
crates/annot/src/draw.rs
+7 -6
View File
@@ -51,9 +51,9 @@ pub(crate) fn can_draw(doc: &Document, d: &Dict) -> bool {
fn build_appearance(doc: &Document, d: &Dict, direct: &Dict) -> Option<Stream> {
// References followed first. Where that turns up something the builder doesn't draw (a
// cloudy `/BE` on a square), the drawing made without them stays, rather than none.
let mut stream = match appearance::build(direct) {
let mut stream = match appearance::build(doc, direct) {
Some(stream) => stream,
None if !std::ptr::eq(direct, d) => appearance::build(d)?,
None if !std::ptr::eq(direct, d) => appearance::build(doc, d)?,
None => return None,
};
// Picture-stamp restyling must retain the placement's page-axis correction.
@@ -92,10 +92,11 @@ pub(crate) const PASS_BUDGET: usize = 1 << 22;
/// `d` with the references among the values [`appearance::build`] reads followed. Any value may be
/// given by reference (ISO 32000-2 §7.3.10): a border style dictionary (`/BS 5 0 R`, Table 168), a
/// colour, `/RD`, `/CL`, a `/W` inside `/BS`. The builder reads only direct values, so it draws
/// from this copy. The entries in [`LINK_KEYS`] and streams stay references; past
/// [`INLINE_DEPTH`] levels or [`INLINE_BUDGET`] copied values a reference is left as it is, which
/// the builder treats as unreadable. `d` itself when nothing is a reference.
/// colour, `/RD`, `/CL`, a `/W` inside `/BS`. The builder reads only direct values (apart from
/// `/QuadPoints`, which it resolves itself), so it draws from this copy. The entries in
/// [`LINK_KEYS`] and streams stay references; past [`INLINE_DEPTH`] levels or [`INLINE_BUDGET`]
/// copied values a reference is left as it is, which the builder treats as unreadable. `d` itself
/// when nothing is a reference.
pub(crate) fn with_direct_values<'d>(doc: &Document, d: &'d Dict) -> std::borrow::Cow<'d, Dict> {
with_direct_values_within(doc, d, &mut INLINE_BUDGET.clone())
}
crates/annot/src/info.rs
+10 -1
View File
@@ -117,7 +117,16 @@ pub fn summaries(doc: &Document) -> Vec<Summary> {
color = Some([f(i - 3), f(i - 2), f(i - 1)]);
}
}
let quads = nums(b"QuadPoints").as_chunks::<8>().0.to_vec();
// Only usable quads, by the rule redaction apply reads them with, so a mark's outline
// is the area apply removes (and with none, the `/Rect` both fall back to).
let quads: Vec<[f32; 8]> = d
.get(b"QuadPoints")
.map(|o| doc.resolve(o))
.map(|a| {
let entries = a.as_array().into_iter().flatten().map(|x| doc.resolve(x).as_f64());
pdfcraft_geom::quad_points::usable_quads(entries).into_iter().map(|q| q.map(|v| v as f32)).collect()
})
.unwrap_or_default();
// Keep in step with `pdfcraft_render::inspect`.
let points = match subtype.as_str() {
"Line" => Some(nums(b"L")).filter(|l| l.len() == 4).map(|l| l.as_chunks::<2>().0.to_vec()).unwrap_or_default(),
crates/annot/src/tests.rs
+97 -11
View File
@@ -1350,6 +1350,12 @@ fn the_eraser_cuts_strokes_and_removes_empty_drawings() {
assert!(!list(&doc, 0).iter().any(|d| d.name(b"Subtype") == Some(b"Ink")));
}
/// A document holding only a catalog, for appearance builds from a bare dictionary whose
/// values are all direct.
fn bare_doc() -> Document {
document_of(&[b"<< /Type /Catalog /Pages << /Type /Pages /Kids [] /Count 0 >> >>"])
}
/// A one-page document from object bodies (object 1 is the catalog, 3 the page).
fn document_of(objs: &[&[u8]]) -> Document {
let mut out = b"%PDF-1.7\n".to_vec();
@@ -1445,7 +1451,7 @@ fn malformed_optional_interior_color_does_not_hide_shape_strokes() {
polyline.set(b"IC".to_vec(), malformed);
for (name, annotation) in [("Square", square), ("Circle", circle), ("Polygon", polygon), ("Line", line), ("PolyLine", polyline)] {
assert!(appearance::build(&annotation).is_some(), "malformed optional /IC must not hide a valid {name} stroke");
assert!(appearance::build(&bare_doc(), &annotation).is_some(), "malformed optional /IC must not hide a valid {name} stroke");
}
let doc = document_of(&[
@@ -1724,7 +1730,7 @@ fn a_stamp_without_an_appearance_draws_the_stamp_it_names() {
// A stamp's own artwork is never replaced, and the drawing is for display only: edits
// (restyling, flattening) still find nothing they may write.
assert!(appearance::display_stamp(&stamp_dict("/Name /Draft /AP << /N 9 0 R >>")).is_none());
assert!(appearance::build(&stamp_dict("/Name /Draft")).is_none());
assert!(appearance::build(&bare_doc(), &stamp_dict("/Name /Draft")).is_none());
assert!(appearance::display_stamp(&stamp_dict("/Name /Draft /PCStamp true")).is_none(), "PdfCraft's own stamps are drawn by `build`");
assert!(appearance::display_stamp(&stamp_dict("/Name /Draft /Rect [5 5 5 50]")).is_none(), "no width");
}
@@ -1827,8 +1833,8 @@ fn every_line_ending_is_drawn() {
assert_eq!(call.name(b"LE"), Some(b"Diamond".as_slice()));
assert!(ap_content(&doc, call).contains("h B"), "diamond callout");
assert!(appearance::build(&line_dict([f64::NAN, 0.0, 10.0, 0.0], "OpenArrow")).is_none(), "a non-finite endpoint is rejected");
assert!(appearance::build(&line_dict([0.0, 0.0, 10.0, 0.0], "Foo")).is_none(), "an unknown ending is not replaced");
assert!(appearance::build(&bare_doc(), &line_dict([f64::NAN, 0.0, 10.0, 0.0], "OpenArrow")).is_none(), "a non-finite endpoint is rejected");
assert!(appearance::build(&bare_doc(), &line_dict([0.0, 0.0, 10.0, 0.0], "Foo")).is_none(), "an unknown ending is not replaced");
}
#[test]
@@ -2369,7 +2375,7 @@ fn note_drawing(name: Option<&str>, color: [f64; 3]) -> Vec<u8> {
if let Some(n) = name {
d.set(b"Name".to_vec(), Object::name(n));
}
crate::appearance::build(&d).expect("a note is drawn").decoded().unwrap()
crate::appearance::build(&bare_doc(), &d).expect("a note is drawn").decoded().unwrap()
}
/// ISO 32000-2 §12.5.6.4, Table 175: a viewer shall provide icons for the seven standard names,
@@ -2417,7 +2423,7 @@ fn a_star_note_keeps_its_star_when_recoloured() {
fn content_of(src: &str) -> String {
let doc = document_of(&[src.as_bytes()]);
let d = doc.get(ObjRef::new(1, 0)).as_dict().cloned().expect("a dictionary");
let stream = appearance::build(&d).expect("drawn");
let stream = appearance::build(&bare_doc(), &d).expect("drawn");
String::from_utf8_lossy(&stream.decoded().unwrap()).into_owned()
}
@@ -2469,7 +2475,7 @@ fn bad_dash_arrays_draw_solid() {
for w in [f64::NAN, f64::INFINITY, -2.0] {
let dash = Object::Array([f64::NAN, 2.0].map(Object::Real).to_vec());
d.set(b"Border".to_vec(), Object::Array(vec![Object::Int(0), Object::Int(0), Object::Real(w), dash]));
let c = String::from_utf8_lossy(&appearance::build(&d).unwrap().decoded().unwrap()).into_owned();
let c = String::from_utf8_lossy(&appearance::build(&bare_doc(), &d).unwrap().decoded().unwrap()).into_owned();
assert!(c.contains("1 w") && !c.contains(" d\n") && !c.contains("NaN") && !c.contains("inf"), "{w}: {c}");
}
}
@@ -2491,7 +2497,7 @@ fn a_caret_with_a_paragraph_symbol_draws_the_pilcrow() {
let caret = |extra: &str| {
let doc = document_of(&[format!("<< /Subtype /Caret /Rect [30 30 60 60] /C [0.9 0.1 0.1] {extra} >>").as_bytes()]);
let d = doc.get(ObjRef::new(1, 0)).as_dict().cloned().unwrap();
let s = appearance::build(&d).unwrap();
let s = appearance::build(&bare_doc(), &d).unwrap();
(s.decoded().unwrap(), s.dict.get(b"Resources").and_then(Object::as_dict).is_some_and(|r| r.contains(b"Font")))
};
let has_pilcrow = |c: &[u8]| c.windows(5).any(|w| w == b"(\xb6) T");
@@ -2519,7 +2525,7 @@ fn a_caret_with_a_paragraph_symbol_draws_the_pilcrow() {
}
let doc = document_of(&[b"<< /Subtype /Caret /Rect [30 30 30 30] /Sy /P >>"]);
let d = doc.get(ObjRef::new(1, 0)).as_dict().cloned().unwrap();
assert!(appearance::build(&d).is_some());
assert!(appearance::build(&bare_doc(), &d).is_some());
}
// ── Line annotations: leader lines and captions (ISO 32000-2 §12.5.6.7, Table 178) ──────────
@@ -2539,7 +2545,7 @@ fn table_178_line(l: [f64; 4], extra: &[(&str, Object)]) -> Dict {
/// The built appearance's content.
fn line_drawing(d: &Dict) -> String {
let s = crate::appearance::build(d).expect("the line is drawn");
let s = crate::appearance::build(&bare_doc(), d).expect("the line is drawn");
String::from_utf8_lossy(&s.decoded().unwrap()).into_owned()
}
@@ -2692,7 +2698,7 @@ fn a_diagonal_lines_leaders_are_perpendicular_and_its_caption_turns_with_it() {
fn a_caption_from_rich_text_and_one_the_font_cant_draw() {
let rc = "<?xml version=\"1.0\"?><body xmlns=\"http://www.w3.org/1999/xhtml\"><p>Dim &amp; <b>5</b>0&#x20;mm &lt;x&gt;</p></body>";
let d = table_178_line([0.0, 0.0, 100.0, 0.0], &[("Cap", Object::Bool(true)), ("RC", Object::String(PdfString::text(rc)))]);
let s = crate::appearance::build(&d).unwrap().decoded().unwrap();
let s = crate::appearance::build(&bare_doc(), &d).unwrap().decoded().unwrap();
assert!(s.windows(17).any(|w| w == b"(Dim & 50 mm <x>)"), "{}", String::from_utf8_lossy(&s));
let cjk = table_178_line([0.0, 0.0, 100.0, 0.0], &[("Cap", Object::Bool(true)), ("Contents", Object::String(PdfString::text("長さ")))]);
assert_eq!(crate::appearance::undrawable_text(&cjk), Some(("the line's caption", '長')));
@@ -2963,3 +2969,83 @@ fn indirect_typewriter_names_keep_outer_whitespace_in_summaries() {
check(&doc);
check(&reopen(&doc));
}
const QUAD: [f64; 8] = [20.0, 110.0, 40.0, 110.0, 20.0, 95.0, 40.0, 95.0];
fn reals(v: &[f64]) -> Vec<Object> {
v.iter().map(|x| Object::Real(*x)).collect()
}
/// The appearance a `subtype` with `/QuadPoints` `quads` draws, as text.
fn quad_appearance(subtype: &str, quads: Vec<Object>) -> Option<String> {
quad_appearance_with(&bare_doc(), subtype, Object::Array(quads))
}
fn quad_appearance_with(doc: &Document, subtype: &str, quad_points: Object) -> Option<String> {
let mut d = Dict::new();
d.set(b"Subtype".to_vec(), Object::name(subtype));
d.set(b"Rect".to_vec(), Object::Array([20, 95, 40, 110].map(Object::Int).to_vec()));
d.set(b"C".to_vec(), Object::Array(vec![Object::Int(1), Object::Int(0), Object::Int(0)]));
d.set(b"QuadPoints".to_vec(), quad_points);
appearance::build(doc, &d).map(|s| String::from_utf8_lossy(&s.decoded().unwrap()).into_owned())
}
/// Appearance drawing reads `/QuadPoints` by the rule the other readers do, indirect objects
/// included: entries by reference, and the array itself by reference, draw the same quads as
/// the direct values.
#[test]
fn appearances_draw_indirect_quads_like_direct_ones() {
for subtype in ["Redact", "Highlight"] {
let clean = quad_appearance(subtype, reals(&QUAD)).unwrap();
let mut doc = bare_doc();
let by_ref: Vec<Object> = QUAD.iter().map(|v| Object::Ref(doc.add(Object::Real(*v)))).collect();
assert_eq!(quad_appearance_with(&doc, subtype, Object::Array(by_ref)).as_deref(), Some(clean.as_str()), "{subtype}: entries by reference");
let array_ref = Object::Ref(doc.add(Object::Array(reals(&QUAD))));
assert_eq!(quad_appearance_with(&doc, subtype, array_ref).as_deref(), Some(clean.as_str()), "{subtype}: the array by reference");
}
}
/// Redaction outlines and text markup draw the quads redaction apply would use, no more: a NaN
/// quad wrote `NaN` into the stream, and one non-number or a trailing partial quad left the other,
/// usable quads undrawn.
#[test]
fn appearances_draw_only_usable_quads() {
let mut one_nan = QUAD;
one_nan[3] = f64::NAN;
for subtype in ["Redact", "Highlight"] {
let clean = quad_appearance(subtype, reals(&QUAD)).unwrap();
assert!(!clean.contains("NaN"), "{clean}");
let mut name_first = reals(&[QUAD, QUAD].concat());
name_first[2] = Object::name("oops");
let same =
[reals(&[one_nan, QUAD].concat()), name_first, reals(&[&QUAD[..], &QUAD[..4]].concat()), reals(&[QUAD, [f64::INFINITY; 8]].concat())];
for (i, quads) in same.into_iter().enumerate() {
assert_eq!(quad_appearance(subtype, quads).as_deref(), Some(clean.as_str()), "{subtype} case {i}");
}
// No usable quad draws nothing, as with no /QuadPoints.
assert_eq!(quad_appearance(subtype, reals(&[f64::NAN; 8])), None, "{subtype}");
}
}
/// The comment list reads `/QuadPoints` by the rule redaction apply does: unusable quads are left
/// out, not read as zeros, and an indirect number is followed.
#[test]
fn summaries_list_only_usable_quads() {
let mut doc = fixture();
let forty = Object::Ref(doc.add(Object::Int(40)));
let page = page_refs(&doc).unwrap()[0];
let mut entries = annots(&doc, page);
let Object::Dict(annotation) = &mut entries[0] else { panic!("fixture annotation is inline") };
// A non-number in the first quad, an indirect 40 in the second, NaN in the third, and four
// entries of a fourth.
let mut quads = reals(&[QUAD, QUAD, QUAD, QUAD].concat());
quads[1] = Object::name("oops");
quads[10] = forty;
quads[17] = Object::Real(f64::NAN);
quads.truncate(28);
annotation.set(b"Subtype".to_vec(), Object::name("Redact"));
annotation.set(b"QuadPoints".to_vec(), Object::Array(quads));
set_annots(&mut doc, page, entries).unwrap();
let summary = summaries(&doc).into_iter().find(|a| a.page == 0 && a.index == 0).unwrap();
assert_eq!(summary.quads, vec![QUAD.map(|v| v as f32)]);
}
crates/audit/src/cover.rs
+2 -2
View File
@@ -283,7 +283,7 @@ impl<'a> Walker<'a> {
return Err(AuditError::TooManyForms);
}
// A form stream that cannot be decoded fails the audit: it might hide a cover.
let data = self.budget.decode_strict(stream)?;
let data = self.budget.decode_strict(self.doc, stream)?;
// The XObject matrix maps form space into the user space active at `Do`.
let mut sub_gs = gs.clone();
sub_gs.ctm = matrix.then(&gs.ctm);
@@ -326,7 +326,7 @@ pub fn filled_rects(doc: &Document, page: &Dict) -> Result<Vec<FilledRect>, Audi
let mut streams = Vec::with_capacity(contents.len());
for c in &contents {
if let Object::Stream(s) = &*doc.resolve(c) {
streams.push(budget.decode_strict(s)?);
streams.push(budget.decode_strict(doc, s)?);
}
}
let pieces = Pieces::join(&streams);
crates/automation/README.md
+29
View File
@@ -26,6 +26,11 @@ its undo/redo history.
`page_add_text`, `content_update` and `text_edit` take per-line `aligns`. An `"inherit"` entry
follows the box-level `align`; legacy `null` entries remain accepted with the same meaning.
`page_add_text` and `content_update` take one of `color` (`#RRGGBB` or a name, painted in
DeviceRGB), `gray` (0 black to 1 white, DeviceGray `g`) and `cmyk` (`[c, m, y, k]`, each 0 to 1,
DeviceCMYK `k`), so text added to a print file can be 100 K black. `content_list` reports `gray` or
`cmyk` for such text.
`Content` is `Json(Value)` or `Png { data, width, height }`. Errors are `UnknownTool`, `InvalidArgs` (the call didn't match the schema) or `Failed` (a readable message for the agent).
## Conventions
@@ -204,6 +209,30 @@ undoable and survive saving/reopening. `image_hit` shares the UI's curve and hol
The bounds, limits and font requirements are in [existing-text editing](../../docs/editing-existing-text.md)
and [image editing](../../docs/editing-images.md).
`text_fonts` also lists `builtin:thai` when the build includes the bundled Noto Sans Thai face
(the optional craft-fonts input). Passing it as `text_edit.font_resource` explicitly replaces a
paragraph or range with that face; the source font is never swapped for it silently. A build
without the face never lists it, and `text_edit` refuses it there. A page whose own fonts fill
the listing's 32 MiB preview budget can leave it out too; `text_edit` still accepts it then.
### Preparing grouped text
`text_prepare {doc, page}` makes supported grouped Form text editable on one page, as one click
on grouped text in the GUI's Edit text does. It keeps the group's original glyph operators and shared source
objects, records preparation and map repairs in the repair log, and is one undoable step. List
`text_paragraphs` afterwards; its indices belong to the prepared page. Besides the document
summary it returns what this edit did:
| Field | Type | Meaning |
| --- | --- | --- |
| `prepared_groups` | integer | Form placements drawn in place, so their text is now the page's own. |
| `repaired_fonts` | integer | Page fonts whose damaged legacy THSarabunPSK Thai map was repaired on a page-owned copy. |
| `refused_groups` | array of strings | One `name: reason` per group left as it was: unsafe compositing, tags, malformed content, or text grouped more than eight levels deep. |
The page's own fonts with that damaged map are repaired too, whether their text was grouped or
not. It fails, changing nothing, when the page has nothing to prepare or repair (the error gives
the first reason a group was left as it was) or the page is out of range.
### Superscript and subscript normalization
`text_script` changes characters to their Unicode superscript or subscript forms; it does not
crates/automation/src/batch.rs
+49 -2
View File
@@ -185,6 +185,7 @@ impl Automation {
let recipe = Recipe::parse(self, args)?;
let recursive = args.opt_bool("recursive")?.unwrap_or(false);
let stop = args.opt_bool("stop_on_error")?.unwrap_or(false);
let dry_run = args.opt_bool("dry_run")?.unwrap_or(false);
let paths = args.strs("paths")?;
if paths.is_empty() || paths.len() > 1000 {
return Err(Args::wrong("paths", "1 to 1000 files or folders"));
@@ -211,11 +212,41 @@ impl Automation {
let mut files = discovery.skipped;
let (mut completed, mut failed_count) = (0, 0);
let mut stopped = false;
let mut planned_outputs: HashSet<PathBuf> = HashSet::new();
for (path, relative) in discovery.files {
if stopped || !recipe.accepts(&path) {
files.push(json!({"path":path.to_string_lossy(),"status":"skipped","reason":if stopped {"stopped after error"} else {"unsupported input extension"}}));
continue;
}
if dry_run {
let planned: Result<PathBuf> = (|| {
let target = out.join(&relative).with_extension("pdf");
let target = self.resolve(target.to_str().ok_or_else(|| failed("output path is not UTF-8"))?, true)?;
if !target.starts_with(&out) {
return Err(failed("output path leaves out_dir"));
}
let dir = target.parent().ok_or_else(|| failed("output has no parent"))?;
let stem = target.file_stem().ok_or_else(|| failed("output has no stem"))?.to_string_lossy().into_owned();
for n in 1..=10_000 {
let candidate = if n == 1 { target.clone() } else { dir.join(format!("{stem} ({n}).pdf")) };
if candidate.symlink_metadata().is_ok() || planned_outputs.contains(&candidate) {
continue;
}
planned_outputs.insert(candidate.clone());
return Ok(candidate);
}
Err(failed("all 10000 output names are occupied"))
})();
match planned {
Ok(target) => files.push(json!({"path":path.to_string_lossy(),"status":"planned","output":target.to_string_lossy().replace('\\', "/"),"steps":recipe.steps})),
Err(e) => {
failed_count += 1;
files.push(json!({"path":path.to_string_lossy(),"status":"error","error":e.to_string()}));
stopped = stop;
}
}
continue;
}
// Recheck confinement and links before reading a discovered file.
let result: Result<_> = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
let current = self.resolve(path.to_str().ok_or_else(|| failed("input path is not UTF-8"))?, false)?;
@@ -244,8 +275,8 @@ impl Automation {
}
}
}
let skipped = files.len().saturating_sub(completed + failed_count);
Ok(json!({"completed":completed,"failed":failed_count,"skipped":skipped,"stopped":stopped,"files":files}))
let skipped = files.iter().filter(|file| file["status"] == "skipped").count();
Ok(json!({"dry_run":dry_run,"completed":completed,"failed":failed_count,"skipped":skipped,"stopped":stopped,"files":files}))
}
}
@@ -279,4 +310,20 @@ mod tests {
discovery.visited = 0;
assert!(discovery.walk(Path::new("absent"), Path::new(""), Path::new("output"), true, MAX_DEPTH + 1).unwrap_err().to_string().contains("64"));
}
#[test]
fn doc_batch_dry_run_lists_outputs_without_writing() {
let dir = tempfile::tempdir().unwrap();
std::fs::create_dir(dir.path().join("input")).unwrap();
let pdf = Session::new().create_from_text("Doc", "Hello").unwrap();
std::fs::write(dir.path().join("input/a.pdf"), &*pdf).unwrap();
let auto = Automation::new().with_root(dir.path()).unwrap();
let report = auto.doc_batch(&Args(&json!({"paths":["input"],"steps":["optimize"],"out_dir":"out","dry_run":true}))).unwrap();
assert_eq!(report["dry_run"], true, "{report}");
assert_eq!(report["completed"], 0, "{report}");
assert_eq!(report["failed"], 0, "{report}");
assert_eq!(report["files"][0]["status"], "planned", "{report}");
assert!(report["files"][0]["output"].as_str().unwrap().ends_with("out/a.pdf"), "{report}");
assert!(!dir.path().join("out").exists());
}
}
crates/automation/src/content.rs
+36 -5
View File
@@ -2,7 +2,7 @@
//! move/resize/retype/restyle it, delete it. Geometry is in points from the top-left of the
//! displayed page (y down), like every other tool.
use pdfcraft_engine::{ADDED_CHAR_SPACING_LIMIT, AddedContent, AddedText, Edit, FontFamily, TextAlign};
use pdfcraft_engine::{ADDED_CHAR_SPACING_LIMIT, AddedContent, AddedText, Edit, FontFamily, TextAlign, TextInk};
use serde_json::{Value, json};
use crate::comments::parse_color;
@@ -30,6 +30,27 @@ fn rect_arg(a: &Args, key: &str) -> Result<Option<[f64; 4]>> {
.transpose()
}
const INK_ARGS: &str = "give one of color (#RRGGBB or a name), gray (0 black to 1 white) and cmyk ([c, m, y, k], each 0 to 1)";
/// `gray` or `cmyk`: the text painted in DeviceGray or DeviceCMYK rather than RGB. At most one of
/// them and `color`.
fn ink_arg(a: &Args) -> Result<Option<TextInk>> {
if ["color", "gray", "cmyk"].iter().filter(|k| a.get(k).is_some()).count() > 1 {
return Err(bad(INK_ARGS));
}
let unit = |v: Option<f64>, key: &str| v.filter(|v| (0.0..=1.0).contains(v)).ok_or_else(|| bad(format!("{key} takes numbers from 0 to 1")));
if let Some(g) = a.get("gray") {
return Ok(Some(TextInk::Gray(unit(g.as_f64(), "gray")?)));
}
let Some(c) = a.get("cmyk") else { return Ok(None) };
let parts = c.as_array().filter(|p| p.len() == 4).ok_or_else(|| bad("cmyk must be [c, m, y, k]"))?;
let mut cmyk = [0.0; 4];
for (out, v) in cmyk.iter_mut().zip(parts) {
*out = unit(v.as_f64(), "cmyk")?;
}
Ok(Some(TextInk::Cmyk(cmyk)))
}
/// Apply the style arguments to `t`.
fn style(a: &Args, t: &mut AddedText) -> Result<()> {
if let Some(s) = a.opt_num("size")? {
@@ -49,8 +70,12 @@ fn style(a: &Args, t: &mut AddedText) -> Result<()> {
if let Some(i) = a.opt_bool("italic")? {
t.italic = i;
}
if let Some(c) = a.opt_str("color")? {
t.color = parse_color(c)?;
if let Some(ink) = ink_arg(a)? {
t.set_ink(Some(ink));
} else if let Some(c) = a.opt_str("color")? {
// An RGB colour paints in RGB, even one that is the ink's look.
t.set_ink(None);
t.set_color(parse_color(c)?);
}
if let Some(al) = a.opt_str("align")? {
t.align = parse_align(al)?;
@@ -124,6 +149,12 @@ impl Automation {
v["bold"] = json!(t.bold);
v["italic"] = json!(t.italic);
v["char_spacing"] = json!(t.char_spacing);
// Text painted in a print colour space says so; RGB text has neither.
match t.painted_ink() {
Some(TextInk::Gray(g)) => v["gray"] = json!(g),
Some(TextInk::Cmyk(c)) => v["cmyk"] = json!(c),
None => {}
}
}
AddedContent::Image(_) => v["type"] = json!("image"),
}
@@ -205,8 +236,8 @@ impl Automation {
if a.get("text").is_some() {
return Err(bad("images have no text"));
}
if ["font", "size", "bold", "italic", "color", "align", "aligns", "char_spacing"].iter().any(|k| a.get(k).is_some()) {
return Err(bad("font, size, bold, italic, color, align, aligns and char_spacing apply to text"));
if ["font", "size", "bold", "italic", "color", "gray", "cmyk", "align", "aligns", "char_spacing"].iter().any(|k| a.get(k).is_some()) {
return Err(bad("font, size, bold, italic, color, gray, cmyk, align, aligns and char_spacing apply to text"));
}
if let Some(deg) = a.opt_int("rotate")? {
if deg.rem_euclid(90) != 0 {
crates/automation/src/dispatch.rs
+5
View File
@@ -467,6 +467,7 @@ impl Automation {
"bring_forward" => ImageEdit::Arrange(pdfcraft_engine::ArrangeOrder::BringForward),
"send_backward" => ImageEdit::Arrange(pdfcraft_engine::ArrangeOrder::SendBackward),
"remove_crop" => ImageEdit::RemoveCrop,
"ungroup" => ImageEdit::Ungroup,
"replace" => {
let path = self.resolve(a.str("path")?, false)?;
let bytes = read_input(&path)?;
@@ -480,6 +481,10 @@ impl Automation {
self.apply(&a, Edit::EditPageImage { page, index, change })?
}
}
"text_prepare" => {
let page = self.page(&a)?;
self.prepare_text(&a, page)?
}
"text_paragraphs" => {
let page = self.page(&a)?;
let doc = self.doc_to_copy(&a)?;
crates/automation/src/documents.rs
+78 -3
View File
@@ -6,6 +6,10 @@ use super::{Arc, Automation, BlockKey, DocId, Document, Edit, Path, PathBuf, Tex
/// How far a watermark may be moved from the page centre, in points (a page is at most 14,400).
const WATERMARK_MAX_OFFSET: f64 = 20_000.0;
/// `doc_optimize` with `target_bytes`: (max target ppi, max JPEG quality) per extra attempt, mildest first.
/// The quality applies only to images the caller left on JPEG; zip and retain keep their encoding.
const TARGET_LADDER: [(f64, u8); 5] = [(110.0, 45), (96.0, 35), (72.0, 28), (54.0, 20), (36.0, 12)];
/// Bookmarks in one page of the bookmark list, by default and at most.
const BOOKMARK_PAGE: usize = 100;
@@ -380,7 +384,45 @@ impl Automation {
})
.collect::<Result<_>>()?,
};
let (bytes, r) = self.session.optimized_bytes(id, &settings, &discard).map_err(failed)?;
let target = match a.opt_int("target_bytes")? {
None => None,
Some(t) if t >= 1 => Some(usize::try_from(t).map_err(|_| ToolError::InvalidArgs(format!("target_bytes {t} is too large")))?),
Some(t) => return Err(ToolError::InvalidArgs(format!("target_bytes must be at least 1, not {t}"))),
};
let (mut bytes, mut r) = self.session.optimized_bytes(id, &settings, &discard).map_err(failed)?;
let mut attempts = 1;
let mut step_used = 0;
// Settings already optimized with; a rung that changes nothing is not run again.
let mut tried = vec![settings.clone()];
if let Some(t) = target {
// Stronger image settings, one rung per attempt, until the copy fits. Every attempt
// starts from the open document, so a failed one changes nothing; the number of
// attempts is fixed. The smallest copy is kept if none fits.
for (k, &(ppi, quality)) in TARGET_LADDER.iter().enumerate() {
if bytes.len() <= t {
break;
}
let mut s = settings.clone();
for img in [&mut s.color, &mut s.gray] {
img.downsample = true;
img.target_ppi = img.target_ppi.min(ppi);
img.above_ppi = img.above_ppi.min(ppi * 1.5).max(img.target_ppi);
// Only a caller who chose JPEG gets a lower quality; zip and retain stay as asked.
if let Compression::Jpeg(q) = img.compression {
img.compression = Compression::Jpeg(q.min(quality));
}
}
if tried.contains(&s) {
continue;
}
let (b2, r2) = self.session.optimized_bytes(id, &s, &discard).map_err(failed)?;
tried.push(s);
attempts += 1;
if b2.len() < bytes.len() {
(bytes, r, step_used) = (b2, r2, k + 1);
}
}
}
write_atomic(&path, &bytes)?;
fn valid_version(v: &[u8]) -> Option<&str> {
if matches!(v, [b'1'..=b'9', b'.', b'0'..=b'9']) { std::str::from_utf8(v).ok() } else { None }
@@ -400,7 +442,19 @@ impl Automation {
// entry). Both are "d.d", so text order is version order.
let pdf_version = catalog_version.as_deref().into_iter().chain(header_version).max();
let o = &r.optimize;
Ok(json!({
let mut extra = json!({});
if let Some(t) = target {
let (ppi, quality) = step_used.checked_sub(1).and_then(|k| TARGET_LADDER.get(k)).copied().unzip();
let lossy = matches!(settings.color.compression, Compression::Jpeg(_)) || matches!(settings.gray.compression, Compression::Jpeg(_));
let quality = quality.filter(|_| lossy);
extra = json!({
"target_bytes": t,
"target_reached": bytes.len() <= t,
"attempts": attempts,
"target_settings": { "step": step_used, "max_ppi": ppi, "max_jpeg_quality": quality },
});
}
let mut out = json!({
"path": path.to_string_lossy(),
"pdf_version": pdf_version,
"object_streams": settings.effective_object_streams(),
@@ -425,7 +479,11 @@ impl Automation {
"unused_xobjects": o.unused_xobjects,
"merged_objects": r.merged,
"discarded": r.discarded.iter().map(|(h, n)| json!({ "category": h.id(), "count": n })).collect::<Vec<_>>(),
}))
});
if let (Some(o), Some(e)) = (out.as_object_mut(), extra.as_object()) {
o.extend(e.iter().map(|(k, v)| (k.clone(), v.clone())));
}
Ok(out)
}
/// A watermark or background picture: `file` (an image or a PDF) and `file_page` (1-based).
@@ -444,6 +502,23 @@ impl Automation {
pub(crate) fn apply(&mut self, a: &Args, edit: Edit) -> Result<Value> {
let id = self.doc(a)?.id;
self.session.apply(id, edit).map_err(failed)?;
self.applied(a)
}
/// `text_prepare`: make a page's grouped text editable, with what that edit itself prepared,
/// repaired and left grouped.
pub(crate) fn prepare_text(&mut self, a: &Args, page: usize) -> Result<Value> {
let id = self.doc(a)?.id;
let prepared = self.session.prepare_text(id, page).map_err(failed)?;
let mut out = self.applied(a)?;
out["prepared_groups"] = json!(prepared.expanded);
out["repaired_fonts"] = json!(prepared.repaired_fonts);
out["refused_groups"] = json!(prepared.refused);
Ok(out)
}
/// The document's summary after an edit, with what it stored as typed but drew with `?`.
fn applied(&mut self, a: &Args) -> Result<Value> {
let doc = self.doc(a)?;
let mut out = summary(doc);
// Text the edit stored as typed but drew with `?` (its font can't draw it): said, not silent.
crates/automation/src/json.rs
+3
View File
@@ -375,6 +375,9 @@ pub(crate) fn info(d: &Document) -> Value {
})).collect::<Vec<_>>(),
"warnings": i.warnings,
"repairs": d.repair_log(),
// Objects that could not be read (each reads as null): nothing was repaired, what they held is
// lost. Last, so it includes what reading the fields above failed on.
"unreadable_objects": d.unreadable_objects(),
})
}
crates/automation/src/pages.rs
+50 -2
View File
@@ -20,6 +20,10 @@ pub(crate) const FIND_MAX_WARNINGS: usize = 100;
pub(crate) const FIND_MAX_WARNING_BYTES: usize = 1024;
/// HTML that one `text_extract` with `format: "html"` returns across all its pages; a page past
/// it gets none. `html_truncated` covers this and any page whose own copy hit its caps.
const MAX_EXTRACT_HTML_BYTES: usize = 32 << 20;
impl Automation {
// ---- pages and text ----------------------------------------------------------------------
@@ -201,6 +205,16 @@ impl Automation {
}
pub(crate) fn text_extract(&mut self, a: &Args) -> Result<Value> {
self.text_extract_within(a, MAX_EXTRACT_HTML_BYTES)
}
/// [`Self::text_extract`] with at most `html_budget` bytes of HTML across all pages.
pub(crate) fn text_extract_within(&mut self, a: &Args, html_budget: usize) -> Result<Value> {
let html = match a.opt_str("format")?.unwrap_or("text") {
"text" => false,
"html" => true,
other => return Err(ToolError::InvalidArgs(format!("format must be \"text\" or \"html\", not {other:?}"))),
};
let doc = self.doc_to_copy(a)?;
let id = doc.id;
let pages = match a.opt_ints("pages")? {
@@ -214,8 +228,42 @@ impl Automation {
Some(r) => t.column_text(&t.glyphs_in(r)),
None => t.plain_text(),
};
let out: Vec<Value> = pages.iter().zip(texts).map(|(p, t)| json!({ "page": p + 1, "text": text(&t) })).collect();
Ok(json!({ "pages": out }))
let doc = self.doc(a)?;
let mut out = Vec::with_capacity(pages.len());
// `spent`: the budget ran out, so later pages get no HTML. `cut`: some page's HTML is
// missing part of its text, by the budget or by the copy's own caps.
let (mut html_left, mut spent, mut cut) = (html_budget, false, false);
for (&p, t) in pages.iter().zip(texts) {
let mut page = json!({ "page": p + 1, "text": text(&t) });
if html {
// Copy with formatting: the same glyphs as an HTML fragment, within one budget for
// the whole call; a page past it gets none and the call says so.
let mut copy = pdfcraft_engine::RichCopy::default();
if !spent {
let glyphs: Vec<usize> = match rect {
Some(r) => t.glyphs_in(r),
None => (0..t.glyphs.len()).collect(),
};
copy = doc.rich_copy(p, &t, &glyphs).map_err(failed)?;
match html_left.checked_sub(copy.html.len()) {
Some(left) => html_left = left,
None => {
spent = true;
copy = pdfcraft_engine::RichCopy::default();
}
}
}
cut |= spent || copy.truncated;
page["html"] = json!(copy.html);
page["styles_incomplete"] = json!(copy.styles_incomplete || spent);
}
out.push(page);
}
let mut result = json!({ "pages": out });
if html {
result["html_truncated"] = json!(cut);
}
Ok(result)
}
/// Matches in document order, extracting page text in growing batches and stopping as soon as
crates/automation/src/redact.rs
+41 -6
View File
@@ -2,8 +2,8 @@
//! (removing what they cover for good) or clear them.
use pdfcraft_engine::{
Edit, Hidden, NewAnnotation, REDACT_MAX_WORDS, REDACTION_CODE_SETS, RedactPattern, RedactionCodeSet, Shape, Style, find_pattern, rect_quad,
redact_word_list,
Edit, Hidden, NewAnnotation, REDACT_MAX_WORDS, REDACT_VALIDATORS, REDACTION_CODE_SETS, RedactPattern, RedactRegex, RedactValidator,
RedactionCodeSet, Shape, Style, find_pattern, rect_quad, redact_word_list,
};
use serde_json::{Value, json};
@@ -59,16 +59,20 @@ impl Automation {
a.get("find").is_some(),
a.get("words").is_some(),
a.get("pattern").is_some(),
a.get("regex").is_some(),
a.opt_bool("whole_pages")?.unwrap_or(false),
];
if chosen.iter().filter(|c| **c).count() != 1 {
return Err(ToolError::InvalidArgs("pass exactly one of rect (with page), find, words, pattern or whole_pages: true".into()));
return Err(ToolError::InvalidArgs("pass exactly one of rect (with page), find, words, pattern, regex or whole_pages: true".into()));
}
let case_sensitive = a.opt_bool("case_sensitive")?.unwrap_or(false);
let whole_words = a.opt_bool("whole_words")?.unwrap_or(false);
if (case_sensitive || whole_words) && a.get("find").is_none() && a.get("words").is_none() {
return Err(ToolError::InvalidArgs("case_sensitive and whole_words apply only to find or words".into()));
}
if a.get("validate").is_some() && a.get("regex").is_none() {
return Err(ToolError::InvalidArgs("validate applies only to regex".into()));
}
// Per searched word or phrase: how many marks it made.
let mut matched_words: Vec<(String, usize)> = Vec::new();
if a.get("rect").is_some() {
@@ -89,6 +93,19 @@ impl Automation {
Some(p) => Some(RedactPattern::from_id(p).ok_or_else(|| ToolError::InvalidArgs(format!("unknown pattern {p:?}")))?),
None => None,
};
let user_re = match a.opt_str("regex")? {
Some(src) => {
let validator = match a.opt_str("validate")? {
Some(v) => Some(RedactValidator::from_id(v).ok_or_else(|| {
let ids: Vec<_> = REDACT_VALIDATORS.iter().map(|v| v.id()).collect();
ToolError::InvalidArgs(format!("unknown validate {v:?}; allowed: {}", ids.join(", ")))
})?),
None => None,
};
Some(RedactRegex::new(src, validator).map_err(|e| ToolError::InvalidArgs(e.to_string()))?)
}
None => None,
};
let needles: Vec<String> = match (a.opt_str("find")?, a.get("words")) {
(Some(f), _) => vec![f.to_owned()],
(None, Some(_)) => {
@@ -108,9 +125,27 @@ impl Automation {
let texts = self.page_texts(id, &pages)?;
let info = self.doc(a)?.info.clone();
for (&p, text) in pages.iter().zip(&texts) {
let hits: Vec<(Option<usize>, _)> = match pattern {
Some(pat) => text.find_with(|chars| find_pattern(pat, chars)).into_iter().map(|h| (None, h)).collect(),
None => needles
let hits: Vec<(Option<usize>, _)> = match (&user_re, pattern) {
(Some(re), _) => {
// A page with too many matches fails the call: marking only the first ones
// would leave the rest readable after redact_apply.
let too_many = std::cell::Cell::new(false);
let found = text.find_with(|chars| {
re.find(chars).unwrap_or_else(|_| {
too_many.set(true);
Vec::new()
})
});
if too_many.get() {
return Err(ToolError::InvalidArgs(format!(
"regex has more than 10000 matches on page {}; nothing was marked. Narrow the expression so that every match can be marked",
p + 1
)));
}
found.into_iter().map(|h| (None, h)).collect()
}
(None, Some(pat)) => text.find_with(|chars| find_pattern(pat, chars)).into_iter().map(|h| (None, h)).collect(),
(None, None) => needles
.iter()
.enumerate()
.flat_map(|(i, w)| text.find_opts(w, case_sensitive, whole_words).into_iter().map(move |h| (Some(i), h)))
crates/automation/src/signing.rs
+1
View File
@@ -119,6 +119,7 @@ impl Automation {
"page": s.page.map(|p| p + 1),
"rect": rect,
"visible": s.visible,
"verdict_shown": s.verdict_shown,
"revision": s.revision,
"sub_filter": s.sub_filter,
"algorithm": s.algorithm,
crates/automation/src/tests.rs
+32
View File
@@ -550,3 +550,35 @@ fn tool_file_reads_refuse_inputs_exceeding_512_mib() {
}
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn text_extract_html_has_one_budget_across_pages() {
let mut a = Automation::new();
let id = search_document(&mut a, 3);
let args = json!({ "doc": id.0, "format": "html" });
let full = a.text_extract(&Args(&args)).unwrap();
assert_eq!(full["html_truncated"], false);
let sizes: Vec<usize> = full["pages"].as_array().unwrap().iter().map(|p| p["html"].as_str().unwrap().len()).collect();
assert!(sizes.iter().all(|n| *n > 0), "{full}");
// Room for two pages: the third gets no HTML, and the call says so.
let capped = a.text_extract_within(&Args(&args), sizes[0] + sizes[1] + sizes[2] - 1).unwrap();
let pages = capped["pages"].as_array().unwrap();
assert_eq!(capped["html_truncated"], true);
assert_eq!(pages[1]["html"], full["pages"][1]["html"]);
assert_eq!(pages[2]["html"], "");
assert_eq!(pages[2]["styles_incomplete"], true);
assert_eq!(pages[2]["text"], "Page 3", "the plain text is not capped by the HTML budget");
// Plain text alone says nothing about HTML.
assert!(a.text_extract(&Args(&json!({ "doc": id.0 }))).unwrap().get("html_truncated").is_none());
// A smaller page after the one that overflowed still gets none: "Page 1" fits what "Page 10"
// did not.
let id = search_document(&mut a, 10);
let args = json!({ "doc": id.0, "pages": [10, 1], "format": "html" });
let full = a.text_extract(&Args(&args)).unwrap();
let (ten, one) = (full["pages"][0]["html"].as_str().unwrap().len(), full["pages"][1]["html"].as_str().unwrap().len());
assert!(one < ten, "{full}");
let capped = a.text_extract_within(&Args(&args), ten - 1).unwrap();
assert_eq!(capped["pages"][0]["html"], "");
assert_eq!(capped["pages"][1]["html"], "", "no HTML after the budget ran out: {capped}");
assert_eq!(capped["pages"][1]["styles_incomplete"], true);
}
crates/automation/src/tools.rs
+27 -16
View File
@@ -148,7 +148,7 @@ pub fn tools() -> Vec<ToolDef> {
"paths": {"type":"array","items":{"type":"string"},"minItems":1,"maxItems":1000},
"out_dir": {"type":"string"},
"steps": {"type":"array","items":{"type":"string","enum":["convert","ocr","optimize"]},"minItems":1,"maxItems":3,"uniqueItems":true},
"recursive": {"type":"boolean"}, "stop_on_error": {"type":"boolean"},
"recursive": {"type":"boolean"}, "stop_on_error": {"type":"boolean"}, "dry_run": {"type":"boolean"},
"ppi": {"type":"number","minimum":9,"maximum":1600},
"quality": {"type":"integer","minimum":1,"maximum":100},
"dpi": {"type":"number","minimum":72,"maximum":600},
@@ -164,7 +164,7 @@ pub fn tools() -> Vec<ToolDef> {
.cmd("file.open")
.with(schema(json!({ "path": { "type": "string" }, "password": { "type": "string", "description": "User or owner password for encrypted files." } }), &["path"])),
t("doc_list", "List open documents", "List the open documents with their ids, page counts and unsaved state.").ro().with(schema(json!({}), &[])),
t("doc_info", "Inspect a document", "Metadata, page sizes and labels, bookmarks, annotations, form fields, links, layers, page transitions and display times, the destination it opens at, attachments, fonts, security and repair notes.")
t("doc_info", "Inspect a document", "Metadata, page sizes and labels, bookmarks, annotations, form fields, links, layers, page transitions and display times, the destination it opens at, attachments, fonts, security, repair notes and the objects that could not be read.")
.ro()
.cmd("file.properties")
.with(schema(json!({ "doc": doc() }), &["doc"])),
@@ -216,13 +216,14 @@ pub fn tools() -> Vec<ToolDef> {
.with(schema(json!({ "doc": doc(), "pages": pages("to count (default: all)"),
"exclude_tables": { "type": "boolean" }, "exclude_code": { "type": "boolean" }, "exclude_images": { "type": "boolean" }
}), &["doc"])),
t("text_extract", "Extract text", "Extract the text of some or all pages, in reading order. With rect, only the text inside that rectangle on each page, as Column select takes it: one row per visual line, side-by-side pieces (table cells) separated by a tab.")
t("text_extract", "Extract text", "Extract the text of some or all pages, in reading order. With rect, only the text inside that rectangle on each page, as Column select takes it: one row per visual line, side-by-side pieces (table cells) separated by a tab. With format \"html\", each page also carries html, the same text as Copy with formatting puts on the clipboard (paragraphs, bold, italic, size, colour and font family), and styles_incomplete when some styles had to be left out. With rect, the html is that text in line order, not split into the column's rows and cells. The HTML of one call is capped at 32 MiB in total; pages past the cap carry empty html with styles_incomplete. html_truncated is set when that cap is reached or when a page's own copy was cut at its span or size cap.")
.ro()
.cmd("edit.column_select")
.with(schema(
json!({
"doc": doc(), "pages": pages("to extract (default: all)"),
"rect": { "type": "array", "items": { "type": "number" }, "minItems": 4, "maxItems": 4, "description": "Column select: [x0, y0, x1, y1] in points from the top-left of the displayed page; a glyph counts when its centre is inside." }
"rect": { "type": "array", "items": { "type": "number" }, "minItems": 4, "maxItems": 4, "description": "Column select: [x0, y0, x1, y1] in points from the top-left of the displayed page; a glyph counts when its centre is inside." },
"format": { "type": "string", "enum": ["text", "html"], "description": "text (default) or html: also return each page's text as formatted HTML." }
}),
&["doc"],
)),
@@ -619,7 +620,7 @@ pub fn tools() -> Vec<ToolDef> {
t(
"redact_mark",
"Mark for redaction",
"Mark content for redaction (nothing is removed until redact_apply). One of: rect [x0, y0, x1, y1] (points from the top-left of the displayed page) with page; find (text, every match); words (a list of words or phrases, every match of each; the result's matched_words says how many marks each made); pattern (phone, email, credit-card, ssn, date, iban: every match, Acrobat's Search & Redact patterns); whole_pages: true. find and words accept case_sensitive and whole_words (both default false). Select one page with page or multiple with pages (default all); do not pass both. rect requires page. overlay: text shown on the box once applied, or code_set (foia: U.S. FOIA (b)(1)(A)…(b)(9); privacy-act: U.S. Privacy Act (d)(5), (j)(1)…(k)(7)) with codes, shown as \"(b)(6), (b)(7)(C)\"; fill: box colour (default black). Undoable.",
"Mark content for redaction (nothing is removed until redact_apply). One of: rect [x0, y0, x1, y1] (points from the top-left of the displayed page) with page; find (text, every match); words (a list of words or phrases, every match of each; the result's matched_words says how many marks each made); pattern (phone, email, credit-card, ssn, date, iban: every match, Acrobat's Search & Redact patterns); regex (your own regular expression, every match, on the page text with words and lines separated by one space; Rust regex syntax: linear-time, no look-around or back-references, case-sensitive unless (?i), at most 1024 bytes, more than 10000 matches on one page fail the call, matches of the empty string are refused) with optional validate (luhn, mod97, iban: keep a match only if its checksum holds); whole_pages: true. find and words accept case_sensitive and whole_words (both default false). Select one page with page or multiple with pages (default all); do not pass both. rect requires page. overlay: text shown on the box once applied, or code_set (foia: U.S. FOIA (b)(1)(A)…(b)(9); privacy-act: U.S. Privacy Act (d)(5), (j)(1)…(k)(7)) with codes, shown as \"(b)(6), (b)(7)(C)\"; fill: box colour (default black). Undoable.",
)
.with(schema(
json!({
@@ -631,6 +632,8 @@ pub fn tools() -> Vec<ToolDef> {
"case_sensitive": { "type": "boolean", "description": "Match letter case for find or words (default false)." },
"whole_words": { "type": "boolean", "description": "Match whole words for find or words (default false)." },
"pattern": pattern_enum(),
"regex": { "type": "string", "minLength": 1, "maxLength": 1024, "description": "Regular expression (Rust regex syntax), at most 1024 bytes (maxLength counts characters, the limit is bytes); every match is marked." },
"validate": { "type": "string", "enum": ["luhn", "mod97", "iban"], "description": "With regex: keep a match only if its checksum is valid (spaces and hyphens ignored)." },
"whole_pages": { "type": "boolean" },
"pages": pages("to search or mark (default: all); mutually exclusive with page, and not valid with rect"),
"overlay": { "type": "string" },
@@ -718,13 +721,13 @@ pub fn tools() -> Vec<ToolDef> {
}),
&["doc"],
)),
t("content_list", "List added content", "Text and images added with page_add_text/page_add_image (or Edit a PDF ▸ Add content), per page with a 1-based index, rect (points from the top-left of the page), and text style.")
t("content_list", "List added content", "Text and images added with page_add_text/page_add_image (or Edit a PDF ▸ Add content), per page with a 1-based index, rect (points from the top-left of the page), and text style; text painted in gray or CMYK also has gray (its level) or cmyk ([c, m, y, k]).")
.ro()
.with(schema(json!({ "doc": doc() }), &["doc"])),
t(
"page_add_text",
"Add text to a page",
"Add text as page content (not a comment). Place it with at [x, y] (top-left, points from the top-left of the displayed page) and width (wrap width, default 200), or rect. Newlines start new lines; long lines wrap. Style: font helvetica/times/courier, size, bold, italic, color (#RRGGBB or a name), align left/center/right, char_spacing (points after each character). Text must be Western European (WinAnsi) characters, or Arabic, Hebrew or Thai when the build has their craft-fonts faces (drawn right to left where the text is); others are refused. It stays editable with content_update. Undoable.",
"Add text as page content (not a comment). Place it with at [x, y] (top-left, points from the top-left of the displayed page) and width (wrap width, default 200), or rect. Newlines start new lines; long lines wrap. Style: font helvetica/times/courier, size, bold, italic, color (#RRGGBB or a name, painted in RGB) or, for print, gray (0 black to 1 white, painted in DeviceGray) or cmyk ([c, m, y, k], each 0 to 1, painted in DeviceCMYK; [0, 0, 0, 1] is 100 K black), one of the three, align left/center/right, char_spacing (points after each character). Text must be Western European (WinAnsi) characters, or Arabic, Hebrew or Thai when the build has their craft-fonts faces (drawn right to left where the text is); others are refused. It stays editable with content_update. Undoable.",
)
.cmd("edit.text")
.with(schema(
@@ -735,6 +738,8 @@ pub fn tools() -> Vec<ToolDef> {
"rect": { "type": "array", "items": { "type": "number" }, "minItems": 4, "maxItems": 4 },
"font": { "type": "string", "enum": ["helvetica", "times", "courier"] }, "size": { "type": "number", "minimum": 1, "maximum": 500 },
"bold": { "type": "boolean" }, "italic": { "type": "boolean" }, "color": { "type": "string" },
"gray": { "type": "number", "minimum": 0, "maximum": 1, "description": "DeviceGray level, 0 black to 1 white (instead of color)." },
"cmyk": { "type": "array", "items": { "type": "number", "minimum": 0, "maximum": 1 }, "minItems": 4, "maxItems": 4, "description": "DeviceCMYK [c, m, y, k], each 0 to 1 (instead of color); [0, 0, 0, 1] is 100 K black." },
"align": { "type": "string", "enum": ["left", "center", "right", "justify"] },
"aligns": { "type": "array", "maxItems": 4096, "items": { "type": "string", "enum": ["left", "center", "right", "justify", "inherit"] }, "description": "Each paragraph's (line's) own alignment, by index; inherit (or legacy null) follows align." },
"char_spacing": { "type": "number", "minimum": -500, "maximum": 500, "description": "Points after each character." },
@@ -751,13 +756,15 @@ pub fn tools() -> Vec<ToolDef> {
json!({ "doc": doc(), "page": { "type": "integer", "minimum": 1 }, "path": { "type": "string" }, "rect": { "type": "array", "items": { "type": "number" }, "minItems": 4, "maxItems": 4 } }),
&["doc", "page", "path"],
)),
t("content_update", "Edit added content", "Move/resize (rect), retype (text) or restyle (font, size, bold, italic, color, align, char_spacing in points) an added item (page, index from content_list). Images: rotate (degrees, multiple of 90, counter-clockwise), flip_h / flip_v (toggle), crop [left, bottom, right, top] as fractions trimmed, image (a file that replaces the picture). Undoable.")
t("content_update", "Edit added content", "Move/resize (rect), retype (text) or restyle (font, size, bold, italic, color or gray or cmyk as page_add_text takes them, align, char_spacing in points) an added item (page, index from content_list). Images: rotate (degrees, multiple of 90, counter-clockwise), flip_h / flip_v (toggle), crop [left, bottom, right, top] as fractions trimmed, image (a file that replaces the picture). Undoable.")
.with(schema(
json!({
"doc": doc(), "page": { "type": "integer", "minimum": 1 }, "index": { "type": "integer", "minimum": 1 },
"rect": { "type": "array", "items": { "type": "number" }, "minItems": 4, "maxItems": 4 }, "text": { "type": "string" },
"font": { "type": "string", "enum": ["helvetica", "times", "courier"] }, "size": { "type": "number", "minimum": 1, "maximum": 500 },
"bold": { "type": "boolean" }, "italic": { "type": "boolean" }, "color": { "type": "string" },
"gray": { "type": "number", "minimum": 0, "maximum": 1, "description": "DeviceGray level, 0 black to 1 white (instead of color)." },
"cmyk": { "type": "array", "items": { "type": "number", "minimum": 0, "maximum": 1 }, "minItems": 4, "maxItems": 4, "description": "DeviceCMYK [c, m, y, k], each 0 to 1 (instead of color); [0, 0, 0, 1] is 100 K black." },
"align": { "type": "string", "enum": ["left", "center", "right", "justify"] },
"aligns": { "type": "array", "maxItems": 4096, "items": { "type": "string", "enum": ["left", "center", "right", "justify", "inherit"] }, "description": "Each paragraph's (line's) own alignment, by index; inherit (or legacy null) follows align." },
"char_spacing": { "type": "number", "minimum": -500, "maximum": 500, "description": "Points after each character." },
@@ -891,7 +898,7 @@ pub fn tools() -> Vec<ToolDef> {
comment_ref(json!({ "status": { "type": "string", "enum": ["none", "accepted", "rejected", "cancelled", "completed"] }, "author": { "type": "string" } })),
&["doc", "status"],
)),
t("sign_list", "List and validate signatures", "Every signature field with its validation: status (valid, unknown = intact but the signer isn't trusted, invalid, unsigned), signer and certificate, date, reason, location, certification level, page and rect, the revision it covers and changes made after signing (none, allowed, disallowed), with Acrobat-style explanations.")
t("sign_list", "List and validate signatures", "Every signature field with its validation: status (valid, unknown = intact but the signer isn't trusted, invalid, unsigned), signer and certificate, date, reason, location, certification level, page and rect, whether PdfCraft draws that state into the signature's own appearance as displayed (verdict_shown; never written to the file), the revision it covers and changes made after signing (none, allowed, disallowed), with Acrobat-style explanations.")
.ro()
.with(schema(json!({ "doc": doc() }), &["doc"])),
t("sign_id_create", "Create a digital ID", "Create a self-signed digital ID and save it as a password-protected .p12 file (Acrobat: Configure a new digital ID ▸ Create a new digital ID ▸ Save to file). Default key: 2048-bit RSA, valid 5 years.").with(schema(
@@ -1285,7 +1292,7 @@ pub fn tools() -> Vec<ToolDef> {
"property": { "type": "string", "enum": ["type", "title", "actual_text", "alternate_text", "language", "summary", "scope"] },
"value": { "type": "string" },
}), &["doc", "object", "generation", "property"])),
t("accessibility_mark_tag_artifact", "Mark tag content as artifact", "Mark a reachable leaf structure tag's direct marked content as an artifact and remove the tag from the reading structure. Use object and generation from accessibility_tags. Tags with child structure elements, no direct marked content, or content not found on its page are rejected. Undoable.")
t("accessibility_mark_tag_artifact", "Mark tag content as artifact", "Mark a reachable leaf structure tag's direct marked content as an artifact and remove the tag from the reading structure. Use object and generation from accessibility_tags. Tags with child structure elements, no direct marked content, content not found on its page, or content that a damaged or undecodable content stream may hide or change are rejected. Undoable.")
.with(schema(json!({
"doc": doc(), "object": { "type": "integer", "minimum": 1 },
"generation": { "type": "integer", "minimum": 0, "maximum": 65535 },
@@ -1320,7 +1327,7 @@ pub fn tools() -> Vec<ToolDef> {
json!({ "doc": doc(), "field": { "type": "string" }, "event": { "type": "string", "enum": ["keystroke", "format", "validate", "calculate", "mouse_up"] }, "script": { "type": "string" } }),
&["doc", "field", "event"],
)),
t("doc_export_office", "Export to Word, PowerPoint, HTML, RTF or Markdown", "Export a PDF ▸ Word (.docx), PowerPoint (.pptx, one slide per page), HTML (.html, one file with images inline), RTF (.rtf) or Markdown (.md or .markdown: pipe tables, images left out), chosen by path's extension: text, headings, bold and italic. HTML uses rich styling by default; set html_mode to simple for CSS-free headings, paragraphs, emphasis and tables. Simple HTML keeps text and inline images but omits colour and layout styling. Word uses borderless inferred tables and positioned editable shapes on pages with artwork, rotated or large text; each source page retains its size. Font matching and vector graphics remain limited. Export reads forms 50 deep and up to 50 000 a page, counting forms the page draws itself. Pages that go past either limit, or that have pictures past what one export may hold or decode, are listed in truncated_pages, with a warning; what was left out is missing.")
t("doc_export_office", "Export to Word, PowerPoint, HTML, RTF or Markdown", "Export a PDF ▸ Word (.docx), PowerPoint (.pptx, one slide per page), HTML (.html, one file with images inline), RTF (.rtf) or Markdown (.md or .markdown: pipe tables, images left out), chosen by path's extension: text, headings, bold and italic. HTML uses rich styling by default; set html_mode to simple for CSS-free headings, paragraphs, emphasis and tables. Simple HTML keeps text and inline images but omits colour and layout styling. Word uses inferred tables, bordered only along the grid lines the page draws, and positioned editable shapes on pages with artwork, rotated or large text; each source page retains its size. Font matching and vector graphics remain limited. Export reads forms 50 deep and up to 50 000 a page, counting forms the page draws itself. Pages that go past either limit, or that have pictures past what one export may hold or decode, are listed in truncated_pages, with a warning; what was left out is missing.")
.cmd("export.docx")
.with(schema(json!({ "doc": doc(), "path": { "type": "string" }, "html_mode": { "type": "string", "enum": ["rich", "simple"], "default": "rich" } }), &["doc", "path"])),
t("pdfa_verify", "Verify PDF/A", "Standards ▸ Verify PDF/A compliance: the PDF/A-2b or 3b rules the document breaks (ISO 19005 clause, message, page, whether Save as PDF/A can fix it), plus what it declares.")
@@ -1555,14 +1562,14 @@ pub fn tools() -> Vec<ToolDef> {
.ro()
.cmd("edit.edit_text")
.with(schema(json!({ "doc": doc(), "page": { "type": "integer", "minimum": 1 } }), &["doc", "page"])),
t("image_edit", "Edit an image", "Change one of a page's images (number from page_images): action move (rect: new full placement box in top-left-origin points, based on page_images.rect), crop (rect: the part to keep, top-left-origin points; the pixels stay in place and page_images.visible_rect shows the result; a later move carries the crop along), remove_crop (remove a crop created by PdfCraft while preserving clipping paths from the original document), rotate (quarters clockwise, default 1), rotate_by (degrees: any angle clockwise about the visible centre; the box in page_images then becomes the turned image's bounding box), transform (matrix: 6 numbers [a, b, c, d, e, f] in top-left-origin points, applied after the current placement so a point (x, y) of the image moves to (a·x + c·y + e, b·x + d·y + f); finite, with a·d − b·c not 0; the box in page_images becomes the transformed image's bounding box), flip_horizontal (reflect left/right on the displayed page), flip_vertical (reflect top/bottom on the displayed page; both flip actions follow the page rotation), replace (path: an image file, drawn in the same place), delete, or arrange: bring_to_front, send_to_back, bring_forward, send_backward (reorder stacking order among content on the page; errors when the image is already at the requested limit; arranging an added image converts it into an ordinary page image). Undoable. The selected image crop transforms with its pixels, isolating shared path scopes while retaining neighboring painting. Text-outline clips return a clear error. Rotation/flip use the visible centre. Form artwork supports move/resize/rotate/flip/delete as a group; replace and image_save only support raster images.")
t("image_edit", "Edit an image", "Change one of a page's images (number from page_images): action move (rect: new full placement box in top-left-origin points, based on page_images.rect), crop (rect: the part to keep, top-left-origin points; the pixels stay in place and page_images.visible_rect shows the result; a later move carries the crop along), remove_crop (remove a crop created by PdfCraft while preserving clipping paths from the original document), rotate (quarters clockwise, default 1), rotate_by (degrees: any angle clockwise about the visible centre; the box in page_images then becomes the turned image's bounding box), transform (matrix: 6 numbers [a, b, c, d, e, f] in top-left-origin points, applied after the current placement so a point (x, y) of the image moves to (a·x + c·y + e, b·x + d·y + f); finite, with a·d − b·c not 0; the box in page_images becomes the transformed image's bounding box), flip_horizontal (reflect left/right on the displayed page), flip_vertical (reflect top/bottom on the displayed page; both flip actions follow the page rotation), replace (path: an image file, drawn in the same place), delete, ungroup (Form artwork only: its content is drawn in place, so its text becomes editable with text_paragraphs/text_edit and its pictures separate; refused for knockout groups and for transparency groups that blend inside, are drawn translucent or blend in a colour space other than the page's, which would composite differently, and for forms whose /DefaultGray, /DefaultRGB or /DefaultCMYK the page doesn't share, which would stop remapping their colours), or arrange: bring_to_front, send_to_back, bring_forward, send_backward (reorder stacking order among content on the page; errors when the image is already at the requested limit; arranging an added image converts it into an ordinary page image). Undoable. The selected image crop transforms with its pixels, isolating shared path scopes while retaining neighboring painting. Text-outline clips return a clear error. Rotation/flip use the visible centre. Form artwork supports move/resize/rotate/flip/delete as a group; replace and image_save only support raster images.")
.cmd("edit.edit_text")
.with(schema(
json!({
"doc": doc(),
"page": { "type": "integer", "minimum": 1 },
"image": { "type": "integer", "minimum": 1 },
"action": { "type": "string", "enum": ["move", "crop", "rotate", "rotate_by", "transform", "flip_horizontal", "flip_vertical", "replace", "delete", "bring_to_front", "send_to_back", "bring_forward", "send_backward", "remove_crop"] },
"action": { "type": "string", "enum": ["move", "crop", "rotate", "rotate_by", "transform", "flip_horizontal", "flip_vertical", "replace", "delete", "bring_to_front", "send_to_back", "bring_forward", "send_backward", "remove_crop", "ungroup"] },
"rect": { "type": "array", "items": { "type": "number" }, "minItems": 4, "maxItems": 4 },
"quarters": { "type": "integer" },
"degrees": { "type": "number" },
@@ -1581,9 +1588,12 @@ pub fn tools() -> Vec<ToolDef> {
t("text_font_import", "Load document font", "Load an explicitly selected, openly licensed full TrueType/OpenType face into a page. The program's family, embedding permissions and licence declaration are validated. Completes source subsets of the same face without replacing existing glyphs. Returns a font resource usable with text_edit.font_resource. Undoable; respects root confinement. No system-interface fallback is exported.")
.cmd("edit.edit_text")
.with(schema(json!({"doc":doc(), "page":{"type":"integer","minimum":1}, "path":{"type":"string"}}), &["doc","page","path"])),
t("text_fonts", "List document fonts", "List validated embedded source faces and approved full preview faces for a page. Full registered resources can be selected by text_edit.font_resource; preview_only faces are not PDF resource names.")
t("text_fonts", "List document fonts", "List validated embedded source faces and approved full preview faces for a page. Full registered resources and bundled choices can be selected by text_edit.font_resource. builtin:thai explicitly replaces a paragraph or range with the bundled Noto Sans Thai; a build without that face never lists it and text_edit refuses it there (a page whose fonts fill the listing budget may also omit it, but text_edit still accepts it). preview_only faces cannot be selected.")
.ro().cmd("edit.edit_text")
.with(schema(json!({"doc":doc(), "page":{"type":"integer","minimum":1}}), &["doc","page"])),
t("text_prepare", "Prepare grouped text", "Make existing text inside supported Form XObjects directly editable on one page. Replaces placements with their original operators and resources, preserving position, clipping and other pages. Refuses unsafe transparency, tags or malformed content. Adds no text or cover boxes. Returns prepared_groups (placements made editable), repaired_fonts (damaged legacy Thai maps repaired) and refused_groups ('name: reason' for each group left as it was). Also repairs the page's own damaged legacy Thai maps, grouped or not. Fails when the page has nothing to prepare or repair. Undoable; list text_paragraphs again afterwards.")
.cmd("edit.edit_text")
.with(schema(json!({"doc":doc(), "page":{"type":"integer","minimum":1}}), &["doc","page"])),
t("text_paragraphs", "List paragraphs", "The paragraphs on a page (adjacent source runs grouped into aligned visual rows, including mixed fonts, sizes and colours): number, text, editor_text (visual rows separated by newlines), rows and source line numbers, box (top-left-origin points), font and size. Use the number with text_edit's paragraph: until you list the page again, a number names the paragraph listed here, even after edits regroup the page.")
.ro()
.cmd("edit.edit_text")
@@ -1597,7 +1607,7 @@ pub fn tools() -> Vec<ToolDef> {
.with(schema(
json!({ "doc": doc(), "page": { "type": "integer", "minimum": 1 }, "line": { "type": "integer", "minimum": 1 },
"paragraph": { "type": "integer", "minimum": 1 },
"font_resource": { "type": "string", "description": "Explicit source/full page font resource from text_fonts or text_font_import." },
"font_resource": { "type": "string", "description": "Paragraphs: source/full page resource or bundled document-font choice from text_fonts (builtin:thai selects the bundled Thai face); also accepts text_font_import resources." },
"ranges": { "type": "array", "maxItems": 1024, "items": { "type": "object", "properties": {"range":{"type":"array","items":{"type":"integer","minimum":0},"minItems":2,"maxItems":2},"font":{"type":"string","enum":["helvetica","times","courier"]},"bold":{"type":"boolean"},"italic":{"type":"boolean"},"font_resource":{"type":"string"},"size":{"type":"number","exclusiveMinimum":0},"color":{"type":"string"},"underline":{"type":"boolean"},"char_spacing":{"type":"number"},"scale":{"type":"number"}},"required":["range"]}, "description":"Disjoint UTF-8 byte ranges in replacement text; unspecified source formatting survives." },
"text": { "type": "string" },
"font": { "type": "string", "enum": ["helvetica", "times", "courier"] },
@@ -1628,7 +1638,7 @@ pub fn tools() -> Vec<ToolDef> {
.with(schema(json!({ "doc": doc() }), &["doc"])),
t("doc_open_revision", "Open a revision", "Open saved revision `revision` (1 = the oldest) of a document as a new, unsaved document, to see the file as it was then.")
.with(schema(json!({ "doc": doc(), "revision": { "type": "integer", "minimum": 1 } }), &["doc", "revision"])),
t("doc_optimize", "Optimize PDF", "Write an optimized copy to `path` (Acrobat's PDF Optimizer). preset: standard (default; Reduce File Size's choices) or mobile (images to 96 ppi above 144, JPEG high quality); the options below change it. compatibility: retain (default; keeps the document's version, or 1.5 when object streams are written), 1.4, 1.5, 1.6, 1.7 or 2.0. merge_fragments (default true) first puts a page picture drawn as many stacked image strips (as some PDF printers write it) back together into one image, so it can be resampled (merged_images, fragments_merged). color / gray: { downsample, ppi, above_ppi, compression: jpeg|zip|retain, quality 1–100 } (defaults: downsample to 150 ppi above 225, JPEG 60). mono: { downsample, ppi, above_ppi, compression: ccitt_g4|jbig2|retain } for black-and-white images and image masks (default: averaged down to 300 ppi above 450 ppi, counted in images_resampled; stored losslessly as CCITT Group 4 or JBIG2 (or 1-bit Flate, once resampled or packed from more bits) when that is smaller, up to 16 JBIG2 images sharing one symbol dictionary (/JBIG2Globals) where that is smaller still (image masks and DeviceGray images named directly, of at most 64 Mi pixels, which redaction can store again without it), Group 3 fax images, JBIG2 images, black-and-white RGB and indexed images using only palette entries 0 and 1 among them, counted in images_recompressed; retain resamples only, a CCITT image staying CCITT, a JBIG2 image JBIG2 and any other becoming Flate). Images past what one run may decode stay as they are, counted in images_refused; streams it only stores with Flate past that stay LZW, uncounted. Images are measured where pages draw them and replaced only when smaller. discard_*: thumbnails (default true), alternate_images (true), tags, print_settings; flate_unencoded (true); object_streams (default true; always off for 1.4); lzw_to_flate (true) decodes LZW streams and stores them with Flate instead, leaving any it cannot decode unchanged (lzw_streams counts the streams converted); remove_invalid_links and remove_unreferenced_dests (true); images and forms the pages list but never draw are always dropped (unused_xobjects). discard: Remove Hidden Information categories (metadata, attachments, comments, form-fields, hidden-text, hidden-layers, bookmarks, links-actions-scripts, private-data). Signed documents are refused. The open document is unchanged.")
t("doc_optimize", "Optimize PDF", "Write an optimized copy to `path` (Acrobat's PDF Optimizer). preset: standard (default; Reduce File Size's choices) or mobile (images to 96 ppi above 144, JPEG high quality); the options below change it. compatibility: retain (default; keeps the document's version, or 1.5 when object streams are written), 1.4, 1.5, 1.6, 1.7 or 2.0. merge_fragments (default true) first puts a page picture drawn as many stacked image strips (as some PDF printers write it) back together into one image, so it can be resampled (merged_images, fragments_merged). color / gray: { downsample, ppi, above_ppi, compression: jpeg|zip|retain, quality 1–100 } (defaults: downsample to 150 ppi above 225, JPEG 60). mono: { downsample, ppi, above_ppi, compression: ccitt_g4|jbig2|retain } for black-and-white images and image masks (default: averaged down to 300 ppi above 450 ppi, counted in images_resampled; stored losslessly as CCITT Group 4 or JBIG2 (or 1-bit Flate, once resampled or packed from more bits) when that is smaller, up to 16 JBIG2 images sharing one symbol dictionary (/JBIG2Globals) where that is smaller still (image masks and DeviceGray images named directly, of at most 64 Mi pixels, which redaction can store again without it), Group 3 fax images, JBIG2 images, black-and-white RGB and indexed images using only palette entries 0 and 1 among them, counted in images_recompressed; retain resamples only, a CCITT image staying CCITT, a JBIG2 image JBIG2 and any other becoming Flate). Images past what one run may decode stay as they are, counted in images_refused; streams it only stores with Flate past that stay LZW, uncounted. Images are measured where pages draw them and replaced only when smaller. discard_*: thumbnails (default true), alternate_images (true), tags, print_settings; flate_unencoded (true); object_streams (default true; always off for 1.4); lzw_to_flate (true) decodes LZW streams and stores them with Flate instead, leaving any it cannot decode unchanged (lzw_streams counts the streams converted); remove_invalid_links and remove_unreferenced_dests (true); images and forms the pages list but never draw are always dropped (unused_xobjects). discard: Remove Hidden Information categories (metadata, attachments, comments, form-fields, hidden-text, hidden-layers, bookmarks, links-actions-scripts, private-data). Signed documents are refused. target_bytes (optional): if the copy is larger, retry with up to 5 progressively stronger image settings (downsample to at most 110, 96, 72, 54, 36 ppi, applied to colour and grey images; images set to JPEG also get JPEG quality at most 45, 35, 28, 20, 12, while zip and retain images keep their encoding and only get the lower ppi) and stop at the first copy that fits; reports target_reached, attempts and target_settings, and writes the smallest copy with target_reached false when none fits. Without it nothing changes. The open document is unchanged.")
.cmd("optimize.advanced")
.with(schema(
json!({
@@ -1640,6 +1650,7 @@ pub fn tools() -> Vec<ToolDef> {
"color": { "type": "object" },
"gray": { "type": "object" },
"mono": { "type": "object" },
"target_bytes": { "type": "integer", "minimum": 1 },
"discard_thumbnails": { "type": "boolean" },
"discard_alternate_images": { "type": "boolean" },
"discard_tags": { "type": "boolean" },
crates/automation/tests/it/automation.rs
+603 -21
View File
@@ -2538,6 +2538,7 @@ fn protecting_through_tools() {
// Copying wasn't allowed either (#568): no text or images come out, though search works.
for (tool, args) in [
("text_extract", json!({ "doc": re })),
("text_extract", json!({ "doc": re, "format": "html" })),
("text_word_count", json!({ "doc": re })),
("text_paragraphs", json!({ "doc": re, "page": 1 })),
("doc_export_text", json!({ "doc": re, "path": "out.txt" })),
@@ -4817,6 +4818,34 @@ fn redacting_through_tools() {
assert!(matches!(a.call("redact_mark", &json!({ "doc": doc })), Err(ToolError::InvalidArgs(_))));
}
#[test]
fn redacting_with_a_caller_regex_through_tools() {
let dir = workdir("redact-regex");
std::fs::write(
dir.join("ids.txt"),
"Ref 1 85 05 78 006 084 36 end\nRef 2 91 12 34 567 890 12 end\nCard 4539 1488 0343 6467 and 4539 1488 0343 6468",
)
.unwrap();
let mut a = auto(&dir);
let doc = ok(&mut a, "doc_create", json!({ "from": "text", "path": "ids.txt" }))["doc"].as_u64().unwrap();
let bad = |a: &mut Automation, args: Value| matches!(a.call("redact_mark", &args), Err(ToolError::InvalidArgs(_)));
assert!(bad(&mut a, json!({ "doc": doc, "regex": "(unclosed" })));
assert!(bad(&mut a, json!({ "doc": doc, "regex": "a*" })));
assert!(bad(&mut a, json!({ "doc": doc, "regex": "x".repeat(2000) })));
assert!(bad(&mut a, json!({ "doc": doc, "regex": "x", "pattern": "ssn" })));
assert!(bad(&mut a, json!({ "doc": doc, "regex": "x", "validate": "crc" })));
assert!(bad(&mut a, json!({ "doc": doc, "find": "x", "validate": "luhn" })));
assert!(matches!(a.call("redact_mark", &json!({ "doc": doc, "regex": "Z{3}" })), Err(ToolError::Failed(_))));
let r = ok(&mut a, "redact_mark", json!({ "doc": doc, "regex": r"\d \d\d \d\d \d\d \d{3} \d{3} \d\d" }));
assert_eq!(r["marked"].as_u64(), Some(2));
let r = ok(&mut a, "redact_mark", json!({ "doc": doc, "regex": r"\d{4}( \d{4}){3}", "validate": "luhn" }));
assert_eq!(r["marked"].as_u64(), Some(1), "only the number with a valid checksum");
ok(&mut a, "redact_apply", json!({ "doc": doc }));
let text = page_text(&mut a, doc)[0].clone();
assert!(!text.contains("85 05") && !text.contains("91 12") && !text.contains("6467"), "{text}");
assert!(text.contains("6468"), "{text}");
}
#[test]
/// Mark, apply and save remove the complete international number while retaining labels and controls.
fn redacting_italian_international_slash_numbers_through_tools() {
@@ -5214,6 +5243,83 @@ fn adding_content_through_tools() {
assert!(matches!(a.call("content_delete", &json!({ "doc": doc, "page": 1, "index": 5 })), Err(ToolError::Failed(_))));
}
/// #6440: `page_add_text` paints text in CMYK or gray for a print file, so its black is 100 K
/// (`0 0 0 1 k`) rather than DeviceRGB; `content_update` restyles it either way.
#[test]
fn added_text_takes_a_cmyk_or_gray_ink_through_tools() {
let dir = workdir("content-ink");
let mut a = auto(&dir);
let doc = ok(&mut a, "doc_open", json!({ "path": "a.pdf" }))["doc"].as_u64().unwrap();
ok(&mut a, "page_add_text", json!({ "doc": doc, "page": 1, "text": "PROOF", "at": [20, 20], "cmyk": [0, 0, 0, 1] }));
ok(&mut a, "page_add_text", json!({ "doc": doc, "page": 1, "text": "Note", "at": [20, 80], "gray": 0.5 }));
let items = ok(&mut a, "content_list", json!({ "doc": doc }))["items"].clone();
assert_eq!((&items[0]["cmyk"], &items[1]["gray"]), (&json!([0.0, 0.0, 0.0, 1.0]), &json!(0.5)));
// The item streams of the saved file.
let saved_items = |a: &mut Automation, name: &str| -> Vec<String> {
ok(a, "doc_save", json!({ "doc": doc, "path": name }));
let saved = pdfcraft_cos::Document::open(std::sync::Arc::new(std::fs::read(dir.join(name)).unwrap())).unwrap();
let page = &pdfcraft_model::pages(&saved)[0];
let list = saved.resolve(page.dict.get(b"Contents").unwrap()).as_array().cloned().unwrap();
list.iter()
.filter_map(|c| match &*saved.resolve(c) {
pdfcraft_cos::Object::Stream(s) if s.dict.name(b"PCMark") == Some(&b"Added"[..]) => {
Some(String::from_utf8_lossy(&s.decoded().unwrap()).into_owned())
}
_ => None,
})
.collect()
};
let streams = saved_items(&mut a, "ink.pdf");
assert!(streams[0].contains("0 0 0 1 k") && !streams[0].contains("rg"), "{}", streams[0]);
assert!(streams[1].contains("0.5 g") && !streams[1].contains("rg"), "{}", streams[1]);
// Restyled: to RGB with color, and back to CMYK.
ok(&mut a, "content_update", json!({ "doc": doc, "page": 1, "index": 2, "color": "#000000" }));
ok(&mut a, "content_update", json!({ "doc": doc, "page": 1, "index": 1, "cmyk": [1, 0, 0, 0] }));
let items = ok(&mut a, "content_list", json!({ "doc": doc }))["items"].clone();
assert_eq!((&items[0]["cmyk"], items[1].get("gray")), (&json!([1.0, 0.0, 0.0, 0.0]), None));
let streams = saved_items(&mut a, "restyled.pdf");
assert!(streams[0].contains("1 0 0 0 k") && streams[1].contains("0 0 0 rg"), "{streams:?}");
// One colour at a time, four numbers from 0 to 1.
for bad in [
json!({ "color": "red", "cmyk": [0, 0, 0, 1] }),
json!({ "gray": 0.5, "cmyk": [0, 0, 0, 1] }),
json!({ "gray": 1.5 }),
json!({ "cmyk": [0, 0, 1] }),
json!({ "cmyk": [0, 0, 0, 1, 0] }),
json!({ "cmyk": [0, "x", 0, 1] }),
json!({ "cmyk": [0, 0, 0, -0.1] }),
] {
let mut args = json!({ "doc": doc, "page": 1, "text": "x", "at": [20, 200] });
args.as_object_mut().unwrap().extend(bad.as_object().unwrap().clone());
assert!(matches!(a.call("page_add_text", &args), Err(ToolError::InvalidArgs(_))), "{bad}");
}
assert_eq!(ok(&mut a, "content_list", json!({ "doc": doc }))["count"], 2, "nothing refused was added");
// content_update refuses them too, an image takes no ink, and a refused update changes nothing.
let mut png = Vec::new();
{
let mut enc = png::Encoder::new(&mut png, 40, 20);
enc.set_color(png::ColorType::Rgb);
let mut w = enc.write_header().unwrap();
w.write_image_data(&[200u8; 2400]).unwrap();
}
std::fs::write(dir.join("logo.png"), png).unwrap();
ok(&mut a, "page_add_image", json!({ "doc": doc, "page": 1, "path": "logo.png", "rect": [100, 200, 180, 240] }));
let before = ok(&mut a, "content_list", json!({ "doc": doc }))["items"].clone();
for (index, bad) in [
(1, json!({ "gray": 0.5, "cmyk": [0, 0, 0, 1] })),
(2, json!({ "color": "red", "gray": 0.5 })),
(1, json!({ "size": 30, "cmyk": [0, 0, 0, 2] })),
(2, json!({ "bold": true, "gray": -1 })),
(3, json!({ "gray": 0.5 })),
(3, json!({ "cmyk": [0, 0, 0, 1] })),
] {
let mut args = json!({ "doc": doc, "page": 1, "index": index });
args.as_object_mut().unwrap().extend(bad.as_object().unwrap().clone());
assert!(matches!(a.call("content_update", &args), Err(ToolError::InvalidArgs(_))), "{bad}");
}
assert_eq!(ok(&mut a, "content_list", json!({ "doc": doc }))["items"], before, "a refused update changed nothing");
}
/// Edit ▸ Copy and Paste headlessly (#2636): an added text box or image, as it is, onto its own
/// page beside it, onto a point, and into another document.
#[test]
@@ -6821,6 +6927,43 @@ fn a_dynamic_xfa_forms_signature_box_is_signed_under_its_som_path() {
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn named_layer_verdicts_are_reported_and_rendered_through_tools() {
use pdfcraft_cos::{Dict, Document, Lexer, ObjRef, Object, SaveOptions, Stream};
let dir = workdir("named-layer-verdict");
let mut cos = Document::open(std::sync::Arc::new(fixture(1))).unwrap();
let dict = |text: &str| Lexer::new(text.as_bytes(), 0).object().unwrap().as_dict().unwrap().clone();
let form = |resources: Dict, ops: &str| {
let mut d = dict("<< /Type /XObject /Subtype /Form /BBox [0 0 100 100] >>");
d.set(b"Resources".to_vec(), resources);
Stream::from_raw(d, ops.as_bytes().to_vec())
};
let blank = cos.add(form(Dict::new(), ""));
let layers = dict(&format!("<< /XObject << /n1 {blank} /n3 {blank} >> >>"));
let layer_form = cos.add(form(layers, "/n1 Do /n3 Do"));
let normal = cos.add(form(dict(&format!("<< /XObject << /FRM {layer_form} >> >>")), "/FRM Do"));
// Deliberately malformed CMS: an invalid signature still gets the redrawn invalid state.
let value = cos.add(dict("<< /Type /Sig /SubFilter /ETSI.CAdES.detached /ByteRange [0 0 0 0] /Contents <00> >>"));
let widget = cos
.add(dict(&format!("<< /FT /Sig /T (Signature1) /Subtype /Widget /F 4 /Rect [50 50 150 150] /P 4 0 R /V {value} /AP << /N {normal} >> >>")));
cos.update_dict(ObjRef::new(4, 0), |p| p.set(b"Annots".to_vec(), vec![Object::Ref(widget)])).unwrap();
cos.update_dict(cos.root().unwrap(), |root| root.set(b"AcroForm".to_vec(), dict(&format!("<< /Fields [{widget}] >>")))).unwrap();
let original = pdfcraft_cos::write_full(&cos, &SaveOptions::default()).unwrap();
std::fs::write(dir.join("named.pdf"), &original).unwrap();
let mut a = auto(&dir);
let doc = ok(&mut a, "doc_open", json!({"path": "named.pdf"}))["doc"].as_u64().unwrap();
let list = ok(&mut a, "sign_list", json!({"doc": doc}));
assert_eq!(list["signatures"][0]["status"], "invalid");
assert_eq!(list["signatures"][0]["verdict_shown"], true);
let rendered = a.call("page_render", &json!({"doc": doc, "page": 1, "dpi": 72})).unwrap();
let Content::Png { data, .. } = &rendered[0] else { panic!("expected a PNG") };
let pixels = image::load_from_memory_with_format(data, image::ImageFormat::Png).unwrap().to_rgba8();
let red = pixels.pixels().filter(|p| p.0.iter().zip([0xD7, 0x37, 0x3F]).all(|(a, b)| a.abs_diff(b) < 30)).count();
assert!(red > 100, "the invalid state is drawn: {red} red pixels");
ok(&mut a, "doc_save", json!({"doc": doc, "path": "saved.pdf", "full": false}));
assert_eq!(std::fs::read(dir.join("saved.pdf")).unwrap(), original, "the drawn state never enters saves");
}
#[test]
fn digital_ids_signing_and_validation_through_tools() {
let dir = workdir("signing");
@@ -6879,6 +7022,8 @@ fn sign_document_and_verify_trust(a: &mut Automation, dir: &Path) -> u64 {
assert_eq!(t["trusted"].as_array().unwrap().len(), 1);
let list = ok(a, "sign_list", json!({ "doc": doc }));
assert_eq!((list["all_valid"].as_bool(), list["signatures"][0]["status"].as_str()), (Some(true), Some("valid")));
// PdfCraft's own appearance has no status layers to draw into: only the page badge shows it.
assert_eq!(list["signatures"][0]["verdict_shown"], false);
ok(a, "comment_add", json!({ "doc": doc, "page": 1, "type": "note", "at": [20, 20], "contents": "ok" }));
let s = &ok(a, "sign_list", json!({ "doc": doc }))["signatures"][0];
assert_eq!((s["status"].as_str(), s["modification"].as_str()), (Some("valid"), Some("allowed")));
@@ -7487,6 +7632,63 @@ fn a_page_printed_in_strips_gets_small_through_tools() {
assert!(matches!(a.call("doc_optimize", &json!({ "doc": doc, "path": "x.pdf", "preset": "tiny" })), Err(ToolError::InvalidArgs(_))));
}
#[test]
fn optimizing_to_a_target_size() {
// Dropped at the end of the test, also when an assertion fails.
let tmp = tempfile::TempDir::new().unwrap();
let dir = tmp.path().to_path_buf();
let (w, h) = (2400u32, 1600u32);
let px: Vec<u8> = (0..w * h)
.flat_map(|i| {
let (x, y) = (i % w, i / w);
[(x * 255 / w) as u8 ^ ((x * y) & 15) as u8, (y * 255 / h) as u8, ((x + y) & 255) as u8]
})
.collect();
let mut jpeg = Vec::new();
let mut enc = image::codecs::jpeg::JpegEncoder::new_with_quality(&mut jpeg, 95);
enc.set_pixel_density(image::codecs::jpeg::PixelDensity::dpi(600));
enc.encode(&px, w, h, image::ExtendedColorType::Rgb8).unwrap();
std::fs::write(dir.join("photo.jpg"), &jpeg).unwrap();
let mut a = auto(&dir);
let doc = ok(&mut a, "doc_create", json!({ "from": "images", "paths": ["photo.jpg"] }))["doc"].as_u64().unwrap();
// Without a target nothing changes: no target fields.
let base = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "base.pdf" }));
assert!(base.get("target_reached").is_none(), "{base}");
let base_len = base["bytes_after"].as_u64().unwrap();
// A target below the default result is met by a stronger step.
let r = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "fit.pdf", "target_bytes": base_len - 1 }));
assert_eq!(r["target_reached"], true, "{r}");
assert!(r["bytes_after"].as_u64().unwrap() < base_len, "{r}");
assert!(r["target_settings"]["step"].as_u64().unwrap() >= 1, "{r}");
assert_eq!(std::fs::metadata(dir.join("fit.pdf")).unwrap().len(), r["bytes_after"].as_u64().unwrap());
// A target already met by the first attempt tries nothing more.
let r = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "easy.pdf", "target_bytes": base_len }));
assert_eq!((r["target_reached"].as_bool(), r["attempts"].as_u64()), (Some(true), Some(1)), "{r}");
// An impossible target: bounded attempts, smallest copy written, said plainly.
let r = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "tiny.pdf", "target_bytes": 1 }));
assert_eq!((r["target_reached"].as_bool(), r["attempts"].as_u64()), (Some(false), Some(6)), "{r}");
assert!(r["bytes_after"].as_u64().unwrap() < base_len, "{r}");
assert!(std::fs::metadata(dir.join("tiny.pdf")).is_ok());
// The open document is untouched by the failed attempts, and bad targets are refused.
let again = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "again.pdf" }));
assert_eq!(again["bytes_after"].as_u64().unwrap(), base_len);
assert!(matches!(a.call("doc_optimize", &json!({ "doc": doc, "path": "x.pdf", "target_bytes": 0 })), Err(ToolError::InvalidArgs(_))));
// A caller who asked for lossless images keeps them: retries only tighten the ppi, no JPEG quality.
let lossless = json!({ "compression": "zip" });
let r = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "zip.pdf", "color": lossless, "gray": lossless, "target_bytes": 1 }));
assert_eq!(r["attempts"].as_u64(), Some(6), "{r}");
assert!(r["target_settings"]["max_ppi"].is_number() && r["target_settings"]["max_jpeg_quality"].is_null(), "{r}");
// A JPEG caller gets the quality step.
let r = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "jpg.pdf", "target_bytes": 1 }));
assert!(r["target_settings"]["max_jpeg_quality"].is_number(), "{r}");
// Settings already at the ladder's floor leave every rung equal to the first run: no retries.
let floor = json!({ "ppi": 9, "above_ppi": 9, "compression": "jpeg", "quality": 1 });
let r = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "floor.pdf", "color": floor, "gray": floor, "target_bytes": 1 }));
assert_eq!((r["target_reached"].as_bool(), r["attempts"].as_u64()), (Some(false), Some(1)), "{r}");
// Confinement still applies.
assert!(a.call("doc_optimize", &json!({ "doc": doc, "path": "../out.pdf", "target_bytes": 1000 })).is_err());
}
#[test]
fn initial_view_through_tools() {
let dir = workdir("initial-view");
@@ -8209,6 +8411,52 @@ BT /F1 11 Tf 20 20 Td (Total due: 150.00) Tj ET";
assert!(rendered.iter().any(|c| matches!(c, Content::Png { .. })));
}
/// #5149: a two-column, two-row table drawn with ruled lines reaches Word as a table with its
/// rules, not as four paragraphs one under another.
#[test]
fn exporting_a_ruled_two_by_two_table_to_word_keeps_the_table() {
let dir = workdir("ruled-table-word");
let content = "0.5 w 20 270 m 300 270 l S 20 245 m 300 245 l S 20 220 m 300 220 l S \
20 220 m 20 270 l S 160 220 m 160 270 l S 300 220 m 300 270 l S \
BT /F1 11 Tf 25 253 Td (Name) Tj 140 0 Td (Value) Tj -140 -25 Td (Alpha) Tj 140 0 Td (One) Tj ET \
BT /F1 11 Tf 20 150 Td (A paragraph after the table.) Tj ET";
let stream = format!("<< /Length {} >>\nstream\n{content}\nendstream", content.len());
let objs = [
"<< /Type /Catalog /Pages 2 0 R >>",
"<< /Type /Pages /Kids [3 0 R] /Count 1 >>",
"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 320 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >>",
stream.as_str(),
"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>",
];
std::fs::write(dir.join("table.pdf"), pdf_of(&objs)).unwrap();
let mut a = auto(&dir);
let doc = ok(&mut a, "doc_open", json!({ "path": "table.pdf" }))["doc"].as_u64().unwrap();
ok(&mut a, "doc_export_office", json!({ "doc": doc, "path": "table.docx" }));
let zip = std::fs::read(dir.join("table.docx")).unwrap();
let mut at = 0;
let mut xml = String::new();
while zip[at..].starts_with(b"PK\x03\x04") {
let size = u32::from_le_bytes(zip[at + 18..at + 22].try_into().unwrap()) as usize;
let len = u16::from_le_bytes(zip[at + 26..at + 28].try_into().unwrap()) as usize;
let data = at + 30 + len;
if &zip[at + 30..data] == b"word/document.xml" {
std::io::Read::read_to_string(&mut flate2::read::DeflateDecoder::new(&zip[data..data + size]), &mut xml).unwrap();
break;
}
at = data + size;
}
assert_eq!(xml.matches("<w:tbl>").count(), 1, "the ruled grid is a table: {xml}");
let table = xml.split_once("<w:tbl>").and_then(|(_, t)| t.split_once("</w:tbl>")).map(|(t, _)| t).unwrap();
assert_eq!(table.matches("<w:tr>").count(), 2, "{table}");
for word in ["Name", "Value", "Alpha", "One"] {
assert_eq!(table.matches(&format!(">{word}</w:t>")).count(), 1, "{word} is a cell: {xml}");
assert_eq!(xml.matches(&format!(">{word}</w:t>")).count(), 1, "{word} is written once");
}
assert!(table.contains("w:val=\"single\""), "the drawn rules become borders: {table}");
assert!(!table.contains("A paragraph after the table."));
assert_eq!(xml.matches(">A paragraph after the table.</w:t>").count(), 1);
}
/// #4158: a word set in italic inside a sentence ("Before *Chevron* after.") reaches Word as an
/// italic run of the sentence's paragraph, not as a paragraph of its own between two halves.
#[test]
@@ -9311,27 +9559,15 @@ fn a_deleted_image_is_not_kept_by_reduce_or_optimize() {
/// page is turned with /Rotate 90.
fn table_pdf() -> Vec<u8> {
let body = "BT /F1 12 Tf 20 250 Td (Name) Tj 130 0 Td (Qty) Tj -130 -20 Td (Apple) Tj 130 0 Td (12) Tj -130 -20 Td (Pear) Tj 130 0 Td (7) Tj ET";
let objs = [
"<< /Type /Catalog /Pages 2 0 R >>".to_string(),
"<< /Type /Pages /Kids [4 0 R 5 0 R] /Count 2 /MediaBox [0 0 300 300] >>".to_string(),
"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>".to_string(),
"<< /Type /Page /Parent 2 0 R /Contents 6 0 R /Resources << /Font << /F1 3 0 R >> >> >>".to_string(),
"<< /Type /Page /Parent 2 0 R /Rotate 90 /Contents 6 0 R /Resources << /Font << /F1 3 0 R >> >> >>".to_string(),
format!("<< /Length {} >>\nstream\n{body}\nendstream", body.len()),
];
let mut out = b"%PDF-1.7\n".to_vec();
let mut offsets = Vec::new();
for (i, o) in objs.iter().enumerate() {
offsets.push(out.len());
out.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes());
}
let xref = out.len();
out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes());
for o in offsets {
out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes());
}
out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes());
out
let stream = format!("<< /Length {} >>\nstream\n{body}\nendstream", body.len());
pdf_of(&[
"<< /Type /Catalog /Pages 2 0 R >>",
"<< /Type /Pages /Kids [4 0 R 5 0 R] /Count 2 /MediaBox [0 0 300 300] >>",
"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>",
"<< /Type /Page /Parent 2 0 R /Contents 6 0 R /Resources << /Font << /F1 3 0 R >> >> >>",
"<< /Type /Page /Parent 2 0 R /Rotate 90 /Contents 6 0 R /Resources << /Font << /F1 3 0 R >> >> >>",
&stream,
])
}
/// Issue #740: `text_extract` with `rect` takes what Column select takes, in the displayed-page
@@ -9372,6 +9608,56 @@ fn text_extract_with_a_rect_takes_one_column() {
}
}
/// Copy with formatting, headless: `format: "html"` adds each page's text as the HTML fragment
/// Copy puts on the clipboard, for the whole page or only what `rect` takes.
#[test]
fn text_extract_as_html_keeps_the_formatting() {
let dir = workdir("rich-copy");
std::fs::write(dir.join("table.pdf"), table_pdf()).unwrap();
let mut a = auto(&dir);
let doc = ok(&mut a, "doc_open", json!({ "path": "table.pdf" }))["doc"].as_u64().unwrap();
let page = &ok(&mut a, "text_extract", json!({ "doc": doc, "pages": [1], "format": "html" }))["pages"][0];
let html = page["html"].as_str().unwrap();
assert!(html.starts_with("<p style=\"margin:0\"><span style=\"font-family:'Helvetica';font-size:12pt\">Name"), "{html}");
assert!(html.contains("Pear") && html.contains("Qty"), "{html}");
assert_eq!(page["text"], "Name\nApple\nPear\nQty\n12\n7", "the plain text is unchanged");
assert_eq!(page["styles_incomplete"], false);
// Plain text by default: no html key.
assert!(ok(&mut a, "text_extract", json!({ "doc": doc, "pages": [1] }))["pages"][0].get("html").is_none());
// Only what the rect takes; nothing inside gives empty HTML.
let column = ok(&mut a, "text_extract", json!({ "doc": doc, "pages": [1], "rect": [120, 0, 300, 300], "format": "html" }));
let html = column["pages"][0]["html"].as_str().unwrap();
assert!(html.contains("Qty") && !html.contains("Name"), "{html}");
let none = ok(&mut a, "text_extract", json!({ "doc": doc, "pages": [1], "rect": [0, 0, 5, 5], "format": "html" }));
assert_eq!(none["pages"][0]["html"], "");
// An unknown format is refused.
for bad in [json!("rtf"), json!(1)] {
let e = a.call("text_extract", &json!({ "doc": doc, "format": bad })).unwrap_err();
assert!(matches!(e, ToolError::InvalidArgs(_)), "{bad}: {e}");
}
// Bold, italic, colour and size come through end to end.
let body = "BT /F1 18 Tf 20 250 Td (Big) Tj ET BT /F2 12 Tf 20 200 Td (Strong) Tj ET \
BT /F3 12 Tf 20 150 Td (Slanted) Tj ET BT 0 0 1 rg /F1 12 Tf 20 100 Td (Blue) Tj ET";
let stream = format!("<< /Length {} >>\nstream\n{body}\nendstream", body.len());
let styled = pdf_of(&[
"<< /Type /Catalog /Pages 2 0 R >>",
"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 300 300] >>",
"<< /Type /Page /Parent 2 0 R /Contents 4 0 R /Resources << /Font << /F1 5 0 R /F2 6 0 R /F3 7 0 R >> >> >>",
&stream,
"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>",
"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica-Bold >>",
"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica-Oblique >>",
]);
std::fs::write(dir.join("styled.pdf"), styled).unwrap();
let doc = ok(&mut a, "doc_open", json!({ "path": "styled.pdf" }))["doc"].as_u64().unwrap();
let page = &ok(&mut a, "text_extract", json!({ "doc": doc, "format": "html" }))["pages"][0];
let html = page["html"].as_str().unwrap();
assert!(html.contains("font-size:18pt\">Big"), "{html}");
assert!(html.contains("<strong>Strong</strong>"), "{html}");
assert!(html.contains("<em>Slanted</em>"), "{html}");
assert!(html.contains("color:#0000FF\">Blue"), "{html}");
}
#[test]
fn mixed_visual_region_edit_preview_save_reopen_and_undo_through_tools() {
use pdfcraft_cos::{Dict, Document, Object, Stream, write_incremental};
@@ -10882,6 +11168,154 @@ fn simple_rule_scan_drawn(content: &[u8]) -> Vec<u8> {
write_full(&doc, &SaveOptions::default()).unwrap()
}
/// #3005: a table and a small signature drawn inside one transparency-group Form XObject (how
/// HTML-to-PDF converters write a block with a translucent picture). Edit a PDF sees one piece
/// of artwork and no text; `image_edit ungroup` draws it in place with the same pixels, its
/// text becomes paragraphs that `text_edit` changes, and undo brings the form back.
fn table_in_a_form_pdf() -> Vec<u8> {
let table = b"BT /F1 12 Tf 1 0 0 1 20 120 Tm (Revenue) Tj 1 0 0 1 120 120 Tm (153,300.00) Tj 1 0 0 1 20 100 Tm (Costs) Tj 1 0 0 1 120 100 Tm (382,110.00) Tj ET 0.6 G 0.5 w 15 115 m 220 115 l S";
// Two pictures with soft masks overlapping each other, the second also over the table's
// last cell: a translucent wash, and a picture whose corner pixels are see-through.
let inner = [&table[..], b" q 40 0 0 20 30 40 cm /Sig Do Q q 60 0 0 40 50 30 cm /Wash Do Q q 100 0 0 30 110 90 cm /Sig Do Q"].concat();
let rgb: Vec<u8> = std::iter::repeat_n([20u8, 60, 220], 16).flatten().collect();
let alpha: Vec<u8> = (0..16).map(|i| if i % 3 == 0 { 255 } else { 0 }).collect();
let page = b"q /Fm1 Do Q";
let stream = |dict: &str, data: &[u8]| [format!("{dict} /Length {} >>\nstream\n", data.len()).as_bytes(), data, b"\nendstream"].concat();
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 240 160] /Contents 4 0 R /Resources << /XObject << /Fm1 5 0 R >> >> >>".to_vec(),
stream("<<", page),
stream(
"<< /Type /XObject /Subtype /Form /BBox [0 0 240 160] /Group << /S /Transparency /CS /DeviceRGB >> /Resources << /Font << /F1 6 0 R >> /XObject << /Sig 7 0 R /Wash 9 0 R >> >>",
&inner,
),
b"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>".to_vec(),
stream("<< /Type /XObject /Subtype /Image /Width 4 /Height 4 /ColorSpace /DeviceRGB /BitsPerComponent 8 /SMask 8 0 R", &rgb),
stream("<< /Type /XObject /Subtype /Image /Width 4 /Height 4 /ColorSpace /DeviceGray /BitsPerComponent 8", &alpha),
stream(
"<< /Type /XObject /Subtype /Image /Width 2 /Height 2 /ColorSpace /DeviceRGB /BitsPerComponent 8 /SMask 10 0 R",
&[230, 40, 40].repeat(4),
),
stream("<< /Type /XObject /Subtype /Image /Width 2 /Height 2 /ColorSpace /DeviceGray /BitsPerComponent 8", &[128, 96, 160, 64]),
];
let mut out = b"%PDF-1.7\n".to_vec();
let mut offsets = Vec::new();
for (i, o) in objs.iter().enumerate() {
offsets.push(out.len());
out.extend_from_slice(format!("{} 0 obj\n", i + 1).as_bytes());
out.extend_from_slice(o);
out.extend_from_slice(b"\nendobj\n");
}
let xref = out.len();
out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes());
for o in offsets {
out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes());
}
out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes());
out
}
#[test]
fn ungrouping_a_table_drawn_in_a_form_makes_its_text_editable() {
let dir = workdir("ungroup");
std::fs::write(dir.join("table.pdf"), table_in_a_form_pdf()).unwrap();
let mut a = auto(&dir);
let doc = ok(&mut a, "doc_open", json!({ "path": "table.pdf" }))["doc"].as_u64().unwrap();
let paragraphs = |a: &mut Automation| ok(a, "text_paragraphs", json!({ "doc": doc, "page": 1 }))["paragraphs"].as_array().unwrap().len();
let objects = |a: &mut Automation| ok(a, "object_list", json!({ "doc": doc, "page": 1 }))["objects"].as_array().unwrap().len();
assert_eq!((paragraphs(&mut a), objects(&mut a)), (0, 1), "one piece of artwork, no text");
let (w, h, before) = a.render_pixels(doc, 1, 2.0).unwrap();
ok(&mut a, "image_edit", json!({ "doc": doc, "page": 1, "image": 1, "action": "ungroup" }));
let (w2, h2, after) = a.render_pixels(doc, 1, 2.0).unwrap();
assert_eq!((w, h), (w2, h2));
let differing = before.as_ref().as_chunks::<4>().0.iter().zip(after.as_ref().as_chunks::<4>().0).filter(|(p, q)| p != q).count();
assert_eq!(differing, 0, "ungrouping draws the same pixels");
assert!(paragraphs(&mut a) >= 4, "the table's text is page text now");
let images = ok(&mut a, "page_images", json!({ "doc": doc, "page": 1 }));
let kinds: Vec<_> = images["images"].as_array().unwrap().iter().map(|i| i["kind"].clone()).collect();
assert_eq!(kinds, vec![json!("image"); 3], "the pictures are separate pictures now: {images}");
let list = ok(&mut a, "text_paragraphs", json!({ "doc": doc, "page": 1 }));
let target =
list["paragraphs"].as_array().unwrap().iter().find(|p| p["text"].as_str().is_some_and(|t| t.contains("382,110.00"))).unwrap()["paragraph"]
.clone();
ok(&mut a, "text_edit", json!({ "doc": doc, "page": 1, "paragraph": target, "text": "400,000.00" }));
let text = ok(&mut a, "text_extract", json!({ "doc": doc, "pages": [1] }));
let s = text.to_string();
assert!(s.contains("400,000.00") && !s.contains("382,110.00"), "{s}");
ok(&mut a, "edit_undo", json!({ "doc": doc }));
ok(&mut a, "edit_undo", json!({ "doc": doc }));
assert_eq!((paragraphs(&mut a), objects(&mut a)), (0, 1), "undo brings the form back");
}
/// One 300 × 300 page that draws form `Fm1` with `page_do`; the form (with `matrix`) fills with
/// pattern `/P1` (`pattern`) in `content`, and a red outline on the page marks where it lands.
fn pattern_in_a_form_pdf(page_do: &str, matrix: &str, content: &str, pattern: &str) -> Vec<u8> {
let objs = [
"<< /Type /Catalog /Pages 2 0 R >>".to_string(),
"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_string(),
"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /XObject << /Fm1 5 0 R >> >> >>".to_string(),
stream_object("<<", &format!("{page_do} 1 0 0 RG 1 w 60 60 180 180 re S")),
stream_object(&format!("<< /Type /XObject /Subtype /Form /BBox [0 0 300 300] {matrix} /Resources << /Pattern << /P1 6 0 R >> >>"), content),
pattern.to_string(),
];
let mut out = b"%PDF-1.7\n".to_vec();
let mut offsets = Vec::new();
for (i, o) in objs.iter().enumerate() {
offsets.push(out.len());
out.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes());
}
let xref = out.len();
out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes());
for o in offsets {
out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes());
}
out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes());
out
}
/// #3067 review: a pattern's matrix maps pattern space to the default space of the stream that
/// uses it (ISO 32000-2 §8.7.2), so a pattern filled inside a form moved by its `/Matrix` or by a
/// `cm` before the `Do` must stay where it was when the form is drawn inline. Each case draws
/// differently in Acrobat when the placement is lost (tiles out of phase or half size, the
/// shading outside its box); ungrouped, every one paints the same pixels.
#[test]
fn ungrouping_keeps_patterns_where_the_forms_placement_put_them() {
let tiling = stream_object(
"<< /Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 20 20] /XStep 20 /YStep 20 /Resources << >>",
"0 0 0 rg 0 0 10 10 re f 10 10 10 10 re f",
);
let shading = "<< /Type /Pattern /PatternType 2 /Shading << /ShadingType 2 /ColorSpace /DeviceRGB /Coords [0 0 100 0] /Function << /FunctionType 2 /Domain [0 1] /C0 [1 0 0] /C1 [0 0 1] /N 1 >> /Extend [false false] >> >>";
let cases = [
("no placement", "q /Fm1 Do Q", "", "/Pattern cs /P1 scn 60 60 180 180 re f", tiling.as_str()),
("form Matrix, half a tile", "q /Fm1 Do Q", "/Matrix [1 0 0 1 10 0]", "/Pattern cs /P1 scn 50 60 180 180 re f", tiling.as_str()),
("cm doubling the form", "q 2 0 0 2 0 0 cm /Fm1 Do Q", "", "/Pattern cs /P1 scn 30 30 90 90 re f", tiling.as_str()),
("shading moved by the form Matrix", "q /Fm1 Do Q", "/Matrix [1 0 0 1 100 100]", "/Pattern cs /P1 scn 0 0 100 100 re f", shading),
];
for (what, page_do, matrix, content, pattern) in cases {
let dir = workdir("ungroup_patterns");
std::fs::write(dir.join("p.pdf"), pattern_in_a_form_pdf(page_do, matrix, content, pattern)).unwrap();
let mut a = auto(&dir);
let doc = ok(&mut a, "doc_open", json!({ "path": "p.pdf" }))["doc"].as_u64().unwrap();
let (w, h, before) = a.render_pixels(doc, 1, 1.0).unwrap();
ok(&mut a, "image_edit", json!({ "doc": doc, "page": 1, "image": 1, "action": "ungroup" }));
ok(&mut a, "doc_save", json!({ "doc": doc, "path": "out.pdf" }));
let saved = ok(&mut a, "doc_open", json!({ "path": "out.pdf" }))["doc"].as_u64().unwrap();
let (w2, h2, after) = a.render_pixels(saved, 1, 1.0).unwrap();
assert_eq!((w, h), (w2, h2), "{what}");
let differing = before.as_ref().as_chunks::<4>().0.iter().zip(after.as_ref().as_chunks::<4>().0).filter(|(p, q)| p != q).count();
assert_eq!(differing, 0, "{what}: ungrouping draws the same pixels, saved and reopened");
}
}
/// An indirect stream object: `dict` (without its closing `>>`) and `data`.
fn stream_object(dict: &str, data: &str) -> String {
format!("{dict} /Length {} >>\nstream\n{data}\nendstream", data.len())
}
/// Reviewing headlessly: corrections keep the boxes, a stale or foreign review is refused, and
/// applying consumes the handle (the pages already accepted were applied once).
#[test]
@@ -11428,3 +11862,151 @@ fn comments_copy_and_paste_between_documents_through_tools() {
assert!(matches!(a.call("comments_copy", &json!({ "doc": src, "comments": [[2]] })), Err(ToolError::InvalidArgs(_))));
assert_eq!(ok(&mut a, "comment_list", json!({ "doc": src }))["comments"].as_array().unwrap().len(), 2, "the source is untouched");
}
#[test]
fn explicit_bundled_thai_font_choice_layout_edit_undo_and_reopen() {
let temp = tempfile::tempdir().unwrap();
let dir = temp.path();
std::fs::write(dir.join("source.pdf"), fixture(1)).unwrap();
let mut a = auto(dir);
let doc = ok(&mut a, "doc_open", json!({"path":"source.pdf"}))["doc"].as_u64().unwrap();
let before = page_text(&mut a, doc);
let fonts = ok(&mut a, "text_fonts", json!({"doc":doc,"page":1}));
let choice = fonts["fonts"].as_array().unwrap().iter().find(|f| f["resource"] == "builtin:thai");
let args = json!({"doc":doc,"page":1,"paragraph":1,"text":"น้ำ กิ่ กี้ ผู้ใช้","font_resource":"builtin:thai"});
if pdfcraft_fonts::document_thai_font().is_none() {
assert!(choice.is_none());
assert!(a.call("text_edit", &args).is_err());
assert_eq!(page_text(&mut a, doc), before);
return;
}
assert_eq!(choice.unwrap()["preview_only"], false);
ok(&mut a, "text_edit_layout", args.clone());
assert_eq!(page_text(&mut a, doc), before);
ok(&mut a, "text_edit", args);
let edited = page_text(&mut a, doc);
assert!(edited[0].contains("น้ำ กิ่ กี้ ผู้ใช้"));
ok(&mut a, "edit_undo", json!({"doc":doc}));
assert_eq!(page_text(&mut a, doc), before);
ok(&mut a, "edit_redo", json!({"doc":doc}));
ok(&mut a, "doc_save", json!({"doc":doc,"path":"edited.pdf"}));
let reopened = ok(&mut a, "doc_open", json!({"path":"edited.pdf"}))["doc"].as_u64().unwrap();
assert_eq!(page_text(&mut a, reopened), edited);
let Err(ToolError::Failed(m)) =
a.call("text_edit", &json!({"doc":reopened,"page":1,"paragraph":1,"text":"น้ำ 😀","font_resource":"builtin:thai"}))
else {
panic!("Noto Sans Thai has no emoji")
};
assert!(m.contains("Noto Sans Thai") && m.contains("U+1F600"), "the error names the font and the character: {m}");
assert_eq!(page_text(&mut a, reopened), edited);
let image = a.call("page_render", &json!({"doc":reopened,"page":1,"dpi":144})).unwrap();
assert!(image.iter().any(|c| matches!(c, Content::Png { .. })));
}
/// `text_prepare` refuses what it can't act on, changing nothing: a page with no grouped text
/// (with the reason), and a page the document doesn't have.
#[test]
fn text_prepare_refuses_pages_without_grouped_text() {
let temp = tempfile::tempdir().unwrap();
let dir = temp.path();
std::fs::write(dir.join("source.pdf"), fixture(1)).unwrap();
let mut a = auto(dir);
let doc = ok(&mut a, "doc_open", json!({"path":"source.pdf"}))["doc"].as_u64().unwrap();
let Err(ToolError::Failed(m)) = a.call("text_prepare", &json!({"doc":doc,"page":1})) else { panic!("nothing to prepare") };
assert!(m.contains("no supported grouped text"), "{m}");
let Err(ToolError::InvalidArgs(m)) = a.call("text_prepare", &json!({"doc":doc,"page":2})) else { panic!("no page 2") };
assert!(m.contains("out of range"), "{m}");
assert!(a.call("edit_undo", &json!({"doc":doc})).is_err(), "nothing was recorded to undo");
assert_eq!(ok(&mut a, "doc_info", json!({"doc":doc}))["document"]["dirty"], false);
}
#[test]
fn grouped_existing_thai_body_edits_saves_reopens_and_undo_restores_original_glyphs() {
use pdfcraft_cos::{Dict, Document, ObjRef, Object, SaveOptions, Stream, write_full};
use std::sync::Arc;
let temp = tempfile::tempdir().unwrap();
let dir = temp.path();
let mut source = Document::open(Arc::new(fixture(2))).unwrap();
let old = "ข้อความเดิม น้ำ ผู้ใช้";
let new = "ข้อความแก้ไข น้ำ ผู้ใช้";
let mut cmap = String::from("begincmap 1 begincodespacerange <00> <ff> endcodespacerange\n");
cmap.push_str(&format!("{} beginbfchar\n", old.chars().count()));
for (i, ch) in old.chars().enumerate() {
cmap.push_str(&format!("<{:02x}> <{:04x}>\n", 65 + i, u32::from(ch)));
}
cmap.push_str("endbfchar endcmap");
let unicode = source.add(Object::Stream(Stream::flate(Dict::new(), cmap.as_bytes())));
let mut font = Dict::new();
font.set(b"Type".to_vec(), Object::name("Font"));
font.set(b"Subtype".to_vec(), Object::name("Type1"));
font.set(b"BaseFont".to_vec(), Object::name("ABCDEF+LegacyThai"));
font.set(b"FirstChar".to_vec(), Object::Int(65));
font.set(b"LastChar".to_vec(), Object::Int(65 + old.chars().count() as i64 - 1));
font.set(b"Widths".to_vec(), Object::Array(vec![Object::Int(300); old.chars().count()]));
font.set(b"ToUnicode".to_vec(), Object::Ref(unicode));
let font = source.add(Object::Dict(font));
let mut fonts = Dict::new();
fonts.set(b"F1".to_vec(), Object::Ref(font));
let mut resources = Dict::new();
resources.set(b"Font".to_vec(), Object::Dict(fonts));
let mut form = Dict::new();
form.set(b"Type".to_vec(), Object::name("XObject"));
form.set(b"Subtype".to_vec(), Object::name("Form"));
form.set(b"BBox".to_vec(), Object::Array(vec![Object::Int(0), Object::Int(0), Object::Int(200), Object::Int(300)]));
form.set(b"Resources".to_vec(), Object::Dict(resources));
let codes: String = (0..old.chars().count()).map(|i| format!("{:02x}", 65 + i)).collect();
let form = source.add(Object::Stream(Stream::flate(form, format!("BT /F1 14 Tf 20 220 Td <{codes}> Tj ET").as_bytes())));
for page in [ObjRef::new(4, 0), ObjRef::new(6, 0)] {
let mut xobjects = Dict::new();
xobjects.set(b"Body".to_vec(), Object::Ref(form));
let mut resources = Dict::new();
resources.set(b"XObject".to_vec(), Object::Dict(xobjects));
let content = source.add(Object::Stream(Stream::flate(Dict::new(), b"q /Body Do Q")));
source
.update_dict(page, |d| {
d.set(b"Resources".to_vec(), Object::Dict(resources));
d.set(b"Contents".to_vec(), Object::Ref(content));
})
.unwrap();
}
std::fs::write(dir.join("source.pdf"), write_full(&source, &SaveOptions::default()).unwrap()).unwrap();
let mut a = auto(dir);
let id = ok(&mut a, "doc_open", json!({"path":"source.pdf"}))["doc"].as_u64().unwrap();
let before = a.call("page_render", &json!({"doc":id,"page":1,"dpi":72})).unwrap();
assert_eq!(ok(&mut a, "text_paragraphs", json!({"doc":id,"page":1}))["count"], 0);
let prepared = ok(&mut a, "text_prepare", json!({"doc":id,"page":1}));
assert_eq!(
[&prepared["prepared_groups"], &prepared["repaired_fonts"], &prepared["refused_groups"]],
[&json!(1), &json!(0), &json!([])],
"{prepared}"
);
assert_eq!(a.call("page_render", &json!({"doc":id,"page":1,"dpi":72})).unwrap(), before, "preparing must not change pixels");
assert_eq!(ok(&mut a, "text_paragraphs", json!({"doc":id,"page":1}))["paragraphs"][0]["text"], old);
// The bundled Thai face is an optional build input. Without it, `builtin:thai` is refused and
// the grouped edit, undo, redo and save checks run on Latin text in Helvetica instead.
let thai = json!({"doc":id,"page":1,"paragraph":1,"text":new,"font_resource":"builtin:thai"});
let (new, args) = if pdfcraft_fonts::document_thai_font().is_some() {
(new, thai)
} else {
let Err(ToolError::Failed(m)) = a.call("text_edit", &thai) else { panic!("builtin:thai needs the bundled Thai face") };
assert!(m.contains("builtin:thai"), "{m}");
let latin = "Edited body";
(latin, json!({"doc":id,"page":1,"paragraph":1,"text":latin,"font":"helvetica"}))
};
ok(&mut a, "text_edit", args);
assert_eq!(ok(&mut a, "text_lines", json!({"doc":id,"page":1}))["lines"][0]["text"].as_str().unwrap().trim(), new);
assert_eq!(page_text(&mut a, id)[1], old, "a shared form's other page must not change");
let after = a.call("page_render", &json!({"doc":id,"page":1,"dpi":72})).unwrap();
assert_ne!(after, before);
ok(&mut a, "edit_undo", json!({"doc":id}));
assert_eq!(a.call("page_render", &json!({"doc":id,"page":1,"dpi":72})).unwrap(), before);
ok(&mut a, "edit_undo", json!({"doc":id}));
assert_eq!(ok(&mut a, "text_paragraphs", json!({"doc":id,"page":1}))["count"], 0);
ok(&mut a, "edit_redo", json!({"doc":id}));
ok(&mut a, "edit_redo", json!({"doc":id}));
ok(&mut a, "doc_save", json!({"doc":id,"path":"edited.pdf"}));
let reopened = ok(&mut a, "doc_open", json!({"path":"edited.pdf"}))["doc"].as_u64().unwrap();
assert_eq!(a.call("page_render", &json!({"doc":reopened,"page":1,"dpi":72})).unwrap(), after);
assert_eq!(page_text(&mut a, reopened), [new, old]);
assert_eq!(ok(&mut a, "content_list", json!({"doc":reopened}))["count"], 0, "editing existing text must not create Add Text objects");
}
crates/automation/tests/it/batch.rs
+58
View File
@@ -238,3 +238,61 @@ fn mcp_batch_errors_keep_the_structured_per_file_report() {
assert_eq!(report["result"]["isError"], true, "{report}");
assert_eq!(report["result"]["structuredContent"]["failed"], 1);
}
#[test]
fn dry_run_planned_files_are_not_counted_as_skipped() {
let dir = tempfile::tempdir().unwrap();
std::fs::create_dir_all(dir.path().join("input/sub")).unwrap();
std::fs::create_dir_all(dir.path().join("out")).unwrap();
let original = pdf();
std::fs::write(dir.path().join("input/a.pdf"), &original).unwrap();
std::fs::write(dir.path().join("input/sub/a.pdf"), &original).unwrap();
std::fs::write(dir.path().join("out/a.pdf"), b"precious").unwrap();
let mut auto = Automation::new().with_root(dir.path()).unwrap();
let report = result(&mut auto, json!({"paths":["input","input/sub"],"steps":["optimize"],"out_dir":"out","dry_run":true}));
assert_eq!(report["dry_run"], true, "{report}");
assert_eq!(report["completed"], 0, "{report}");
assert_eq!(report["failed"], 0, "{report}");
assert_eq!(report["skipped"], 0, "planned files are not skipped: {report}");
let files = report["files"].as_array().unwrap();
assert_eq!(files.len(), 2, "{report}");
assert!(files.iter().all(|file| file["status"] == "planned"), "{report}");
assert!(files.iter().all(|file| file["steps"] == json!(["optimize"])), "planned files list the ordered steps: {report}");
assert!(files[0]["output"].as_str().unwrap().ends_with("out/a (2).pdf"), "existing output gets the publish collision name: {report}");
assert!(files[1]["output"].as_str().unwrap().ends_with("out/a (3).pdf"), "later planned files reserve their names: {report}");
assert_eq!(std::fs::read(dir.path().join("out/a.pdf")).unwrap(), b"precious");
assert!(!dir.path().join("out/a (2).pdf").exists(), "dry run writes nothing: {report}");
#[cfg(unix)]
{
// An output subfolder that resolves outside out_dir is a per-file error:
// it counts as failed, honors stop_on_error, and later files are skipped.
for outside_root in [false, true] {
let dir = tempfile::tempdir().unwrap();
let outside = tempfile::tempdir().unwrap();
std::fs::create_dir_all(dir.path().join("input/sub")).unwrap();
std::fs::create_dir_all(dir.path().join("out")).unwrap();
std::fs::write(dir.path().join("input/a.pdf"), pdf()).unwrap();
std::fs::write(dir.path().join("input/sub/page.pdf"), pdf()).unwrap();
std::fs::write(dir.path().join("input/z.pdf"), pdf()).unwrap();
let target = if outside_root { outside.path().to_path_buf() } else { dir.path().join("outside") };
if !outside_root {
std::fs::create_dir(&target).unwrap();
}
std::os::unix::fs::symlink(&target, dir.path().join("out/sub")).unwrap();
let mut auto = Automation::new().with_root(dir.path()).unwrap();
let report = result(
&mut auto,
json!({"paths":["input"],"steps":["optimize"],"out_dir":"out","recursive":true,"dry_run":true,"stop_on_error":true}),
);
assert_eq!(report["failed"], 1, "preview target failure is per-file: {report}");
assert_eq!(report["stopped"], true, "{report}");
assert_eq!(report["skipped"], 1, "only the unprocessed file is skipped: {report}");
let files = report["files"].as_array().unwrap();
assert_eq!(files.len(), 3, "{report}");
assert_eq!(files[0]["status"], "planned", "{report}");
assert_eq!(files[1]["status"], "error", "{report}");
assert_eq!(files[2]["status"], "skipped", "stop_on_error leaves later files unplanned: {report}");
assert!(!dir.path().join("out/a.pdf").exists(), "dry run writes nothing: {report}");
}
}
}
crates/automation/tests/it/main.rs
+1
View File
@@ -17,3 +17,4 @@ mod optimize_content_boundaries;
mod redact_after_stray_q_form;
mod redact_text_after_q;
mod render_warnings;
mod unreadable_objects;
crates/automation/tests/it/unreadable_objects.rs
+64
View File
@@ -0,0 +1,64 @@
//! `doc_info` lists the objects of a file that could not be read: each reads as null, so the file
//! opens, but what it held is lost and nothing was repaired.
use pdfcraft_automation::{Automation, Content};
use serde_json::{Value, json};
/// A one-page PDF whose document information dictionary (object 4) is `info`.
fn with_info(info: &str) -> Vec<u8> {
let bodies = [
"<< /Type /Catalog /Pages 2 0 R >>",
"<< /Type /Pages /Kids [3 0 R] /Count 1 >>",
"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] >>",
info,
];
let mut out = b"%PDF-1.7\n".to_vec();
let mut offsets = Vec::new();
for (i, body) in bodies.iter().enumerate() {
offsets.push(out.len());
out.extend_from_slice(format!("{} 0 obj\n{body}\nendobj\n", i + 1).as_bytes());
}
let xref = out.len();
out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", bodies.len() + 1).as_bytes());
for o in offsets {
out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes());
}
out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R /Info 4 0 R >>\nstartxref\n{xref}\n%%EOF\n", bodies.len() + 1).as_bytes());
out
}
fn info_of(a: &mut Automation, path: &str) -> Value {
let doc = match a.call("doc_open", &json!({ "path": path })).unwrap().remove(0) {
Content::Json(v) => v["doc"].as_u64().unwrap(),
other => panic!("expected JSON: {other:?}"),
};
match a.call("doc_info", &json!({ "doc": doc })).unwrap().remove(0) {
Content::Json(v) => v,
other => panic!("expected JSON: {other:?}"),
}
}
/// An unparseable `/Info` is listed with its number and why, the rest of the file reads, and
/// nothing is reported as repaired.
#[test]
fn doc_info_lists_an_object_that_could_not_be_read() {
let dir = std::env::temp_dir().join(format!("pdfcraft-unreadable-objects-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
std::fs::write(dir.join("broken.pdf"), with_info("<< /Title (Report) /Author ) >>")).unwrap();
std::fs::write(dir.join("sound.pdf"), with_info("<< /Title (Report) >>")).unwrap();
let mut a = Automation::new().with_root(&dir).unwrap();
let info = info_of(&mut a, "broken.pdf");
let lines = info["unreadable_objects"].as_array().unwrap();
assert_eq!(lines.len(), 1, "{info}");
let line = lines[0].as_str().unwrap();
assert!(line.starts_with("object 4 0 could not be read: syntax error"), "{line}");
assert_eq!(info["repairs"], json!([]), "nothing was repaired");
assert_eq!(info["pages"].as_array().unwrap().len(), 1, "the rest of the file reads");
let info = info_of(&mut a, "sound.pdf");
assert_eq!(info["unreadable_objects"], json!([]), "{info}");
assert_eq!(info["title"], "Report");
let _ = std::fs::remove_dir_all(dir);
}
crates/cos/src/budget.rs
+189 -36
View File
@@ -11,7 +11,9 @@ use crate::{CosError, Document, ObjRef, Object, Stream};
/// 16 ns, so this is 2 to 5 minutes, and about as long again for what an operation does with the
/// pixels. A page scanned at 300 dpi takes about 17 million, 8.4 for its pixels and that again to
/// decode them, so this alone reads about 1,000 of them. A byte into or out of a filter is a step
/// too, and takes less: 3 ns at most, or up to 19 ns through a predictor on samples of a bit.
/// too, and takes less: 3 ns at most, or up to 19 ns through a predictor on samples of a bit. A
/// byte Flate reads is [`pdfcraft_filters::FLATE_READ_WORK`] steps besides, since inflating blocks
/// that make nothing takes up to 21 ns a byte: about 7 ns a step.
const IMAGE_WORK: u64 = 1 << 34;
/// The steps each byte of the file adds to [`IMAGE_WORK`], so that a long scan is read whole: such a
/// page pays for itself with about 2 KB of the file, about the smallest a page of text is stored
@@ -38,9 +40,9 @@ const JPEG_WORK: u64 = 2;
/// and images can share one, so reading it again for each has to count.
const JPEG_READ_WORK: u64 = 4;
/// A stream decode within a byte bound, with the bytes its filter stages decoded: the data or
/// the error, each with that count.
type Decoder = fn(&Stream, usize) -> Result<(Vec<u8>, usize), (FilterError, usize)>;
/// A stream decode within a byte bound, with the bytes its filter stages decoded: the data and
/// whether it stopped at damage, or the error, each with that count.
type Decoder = fn(&Stream, usize) -> Result<(Vec<u8>, bool, usize), (FilterError, usize)>;
/// What one operation over a document's images may still spend decoding them: exporting them,
/// optimizing them, and storing LZW streams with Flate as it does, clearing them for redaction.
@@ -263,45 +265,57 @@ impl DecodeBudget {
}
/// `stream` decoded like [`Stream::decoded`], charged to the allowance. Fails, without
/// decoding, once the allowance is spent.
/// decoding, once the allowance is spent. A stream damaged partway gives what decoded before
/// the damage, so code that writes the stream back reads it with
/// [`DecodeBudget::decode_with_damage`] instead.
pub fn decode(&mut self, stream: &Stream) -> Result<Vec<u8>, CosError> {
self.charge(stream, usize::MAX, Stream::decoded_within_counted)
self.charge(stream, usize::MAX, Stream::decoded_within_counted).map(|(data, _)| data)
}
/// Like [`DecodeBudget::decode`], for a stream with a tighter bound of its own: decoding
/// past `max` bytes is an error that charges `max`, the work it took to find out.
pub fn decode_within(&mut self, stream: &Stream, max: usize) -> Result<Vec<u8>, CosError> {
self.charge(stream, max, Stream::decoded_within_counted)
self.charge(stream, max, Stream::decoded_within_counted).map(|(data, _)| data)
}
/// [`DecodeBudget::decode_within`] (`usize::MAX` for no bound of its own), with the stream's
/// `/DecodeParms` given by reference read in `doc`, also saying whether the stream is damaged:
/// `true` when a filter stopped at corrupt data, so the bytes are only what decoded before it.
/// A stream written back from them loses everything after the damage, which other readers may
/// still read.
pub fn decode_with_damage(&mut self, doc: &Document, stream: &Stream, max: usize) -> Result<(Vec<u8>, bool), CosError> {
self.charge(&doc.decoding_view(stream), max, Stream::decoded_within_counted)
}
/// `stream` decoded like [`Stream::decoded_strict`] (any corruption is an error), charged to
/// the allowance.
pub fn decode_strict(&mut self, stream: &Stream) -> Result<Vec<u8>, CosError> {
self.charge(stream, usize::MAX, Stream::decoded_strict_within_counted)
/// the allowance, with its `/DecodeParms` given by reference read in `doc`.
pub fn decode_strict(&mut self, doc: &Document, stream: &Stream) -> Result<Vec<u8>, CosError> {
let strict: Decoder = |s, max| s.decoded_strict_within_counted(max).map(|(data, spent)| (data, false, spent));
self.charge(&doc.decoding_view(stream), usize::MAX, strict).map(|(data, _)| data)
}
/// Decodes `stream` within `max` and the allowance, charging what every filter stage
/// decoded: a stage's output counts even when a later stage shrinks it, and a corrupt stream
/// pays for the bytes decoded before the error, so referencing a stream again and again
/// spends the allowance by the work each decode does.
fn charge(&mut self, stream: &Stream, max: usize, decode: Decoder) -> Result<Vec<u8>, CosError> {
fn charge(&mut self, stream: &Stream, max: usize, decode: Decoder) -> Result<(Vec<u8>, bool), CosError> {
if self.bytes == 0 && !stream.raw.is_empty() {
return Err(CosError::Filter(FilterError::LimitExceeded(self.limit).to_string()));
}
if max < self.bytes {
let result = decode(stream, max);
let spent = match &result {
Ok((_, spent)) => *spent,
Ok((_, _, spent)) => *spent,
Err((FilterError::LimitExceeded(_), _)) => max,
Err((_, spent)) => *spent,
};
self.bytes = self.bytes.saturating_sub(spent);
return result.map(|(data, _)| data).map_err(|(e, _)| CosError::Filter(e.to_string()));
return result.map(|(data, damaged, _)| (data, damaged)).map_err(|(e, _)| CosError::Filter(e.to_string()));
}
match decode(stream, self.bytes) {
Ok((data, spent)) => {
Ok((data, damaged, spent)) => {
self.bytes = self.bytes.saturating_sub(spent);
Ok(data)
Ok((data, damaged))
}
Err((FilterError::LimitExceeded(_), _)) => {
self.bytes = 0;
@@ -332,17 +346,24 @@ impl DecodeBudget {
self.seen.insert(r)
}
/// Each stream `contents` names, decoded against the allowance, in order. `contents` is a
/// page's `/Contents`: one stream, or an array whose non-stream entries are left out.
/// A stream that fails to decode, or arrives after the allowance is spent, is an `Err` in its
/// place, so the caller chooses between skipping it and giving up.
/// Each stream `contents` names, decoded against the allowance, in order, its `/DecodeParms`
/// given by reference read in `doc`. `contents` is a page's `/Contents`: one stream, or an
/// array whose non-stream entries are left out. A stream that fails to decode, or arrives
/// after the allowance is spent, is an `Err` in its place, so the caller chooses between
/// skipping it and giving up.
pub fn decode_contents(&mut self, doc: &Document, contents: &Object) -> Vec<Result<Vec<u8>, CosError>> {
self.decode_contents_with_damage(doc, contents).into_iter().map(|read| read.map(|(data, _)| data)).collect()
}
/// [`DecodeBudget::decode_contents`], each stream also saying whether it is damaged partway
/// ([`DecodeBudget::decode_with_damage`], which reads `/DecodeParms` given by reference).
pub fn decode_contents_with_damage(&mut self, doc: &Document, contents: &Object) -> Vec<Result<(Vec<u8>, bool), CosError>> {
match &*doc.resolve(contents) {
Object::Stream(s) => vec![self.decode(s)],
Object::Stream(s) => vec![self.decode_with_damage(doc, s, usize::MAX)],
Object::Array(list) => list
.iter()
.filter_map(|entry| match &*doc.resolve(entry) {
Object::Stream(s) => Some(self.decode(s)),
Object::Stream(s) => Some(self.decode_with_damage(doc, s, usize::MAX)),
_ => None,
})
.collect(),
@@ -367,6 +388,11 @@ mod image_tests {
Stream::from_raw(dict, pdfcraft_filters::encode_jbig2_generic(&[0xFF; 4], 16, 2).unwrap())
}
/// What a Flate stage pays to read `n` bytes, besides what it makes.
fn read(n: usize) -> u64 {
pdfcraft_filters::FLATE_READ_WORK * n as u64
}
/// Each image spends what it took from one allowance, and those past it are refused and
/// counted, whatever their filters. What can't be decoded here costs nothing, and fails
/// without being one of these.
@@ -413,7 +439,7 @@ mod image_tests {
let mut dict = Dict::new();
dict.set(b"Filter".to_vec(), Object::Array(vec![Object::name("FlateDecode"), Object::name("FlateDecode")]));
let bomb = Stream::from_raw(dict, pdfcraft_filters::encode_flate(&inner));
let cost = (bomb.raw.len() + inner.len() + samples.len()) as u64;
let cost = (bomb.raw.len() + inner.len() + samples.len()) as u64 + read(bomb.raw.len()) + read(inner.len());
let mut budget = ImageBudget::with_work(2 * cost + 1);
for left in [cost + 1, 1] {
assert_eq!(budget.decode(&doc, &bomb, 1 << 20).unwrap(), samples);
@@ -442,17 +468,17 @@ mod image_tests {
let image = behind(pdfcraft_filters::encode_flate(&alone.raw));
let mut budget = ImageBudget::with_work(u64::MAX);
assert_eq!(budget.decode(&doc, &image, 1 << 20).unwrap(), samples);
assert_eq!(u64::MAX - budget.work_left(), decoding + (image.raw.len() + alone.raw.len()) as u64);
assert_eq!(u64::MAX - budget.work_left(), decoding + (image.raw.len() + alone.raw.len()) as u64 + read(image.raw.len()));
// Past this image's limit: an error, with what finding out took paid and the rest left.
let bomb = behind(pdfcraft_filters::encode_flate(&vec![0; 1 << 16]));
let mut budget = ImageBudget::with_work(u64::MAX);
assert!(budget.decode(&doc, &bomb, 1000).is_err());
assert_eq!(u64::MAX - budget.work_left(), bomb.raw.len() as u64 + 1000);
assert_eq!(u64::MAX - budget.work_left(), bomb.raw.len() as u64 + read(bomb.raw.len()) + 1000);
assert_eq!(budget.refused(), 0, "that image's limit isn't the run's");
// A small allowance is a limit too: it stops there, and the image is refused.
let mut budget = ImageBudget::with_work(bomb.raw.len() as u64 + 10);
let mut budget = ImageBudget::with_work(bomb.raw.len() as u64 + read(bomb.raw.len()) + 10);
assert!(budget.decode(&doc, &bomb, 1 << 20).is_err());
assert_eq!((budget.work_left(), budget.refused()), (0, 1));
}
@@ -492,7 +518,7 @@ mod image_tests {
let doc = Document::open(std::sync::Arc::new(file)).unwrap();
let image = doc.get(doc.root().unwrap()).as_dict().unwrap().reference(b"Image").unwrap();
let s = stream(&doc, image);
let filters = (raw.len() + skipped.len()) as u64;
let filters = (raw.len() + skipped.len()) as u64 + read(raw.len());
assert_eq!(spent(&doc, &s), first, "the first read undoes them");
assert_eq!(spent(&doc, &s), first - filters, "kept");
@@ -507,7 +533,7 @@ mod image_tests {
let image = jbig2(Some(Object::Dict(parms)));
let mut budget = ImageBudget::with_work(u64::MAX);
assert!(budget.decode(&doc, &image, 1 << 20).unwrap_err().to_string().contains("globals"));
assert_eq!(u64::MAX - budget.work_left(), (over.len() + crate::MAX_JBIG2_GLOBALS) as u64);
assert_eq!(u64::MAX - budget.work_left(), (over.len() + crate::MAX_JBIG2_GLOBALS) as u64 + read(over.len()));
assert_eq!(budget.refused(), 0);
}
@@ -528,7 +554,7 @@ mod image_tests {
dict.set(b"DecodeParms".to_vec(), Object::Array(vec![Object::Null, Object::Dict(parms)]));
let image = Stream::from_raw(dict, pdfcraft_filters::encode_flate(&coded));
let pixels = u64::from(w) * u64::from(h);
let cost = (image.raw.len() + coded.len() + 4 * coded.len() + 512 * h as usize) as u64 + pixels;
let cost = (image.raw.len() + coded.len() + 4 * coded.len() + 512 * h as usize) as u64 + read(image.raw.len()) + pixels;
let mut budget = ImageBudget::with_work(cost);
assert_eq!(budget.decode(&doc, &image, 1 << 20).unwrap().len(), 512 * h as usize);
assert_eq!(budget.work_left(), 0, "exactly that");
@@ -553,7 +579,7 @@ mod image_tests {
};
let rows = vec![0xFFu8; 512 * 1024];
let flate = blank(Some("FlateDecode"), pdfcraft_filters::encode_flate(&rows));
let cost = (flate.raw.len() + rows.len()) as u64 + 4096 * 1024;
let cost = (flate.raw.len() + rows.len()) as u64 + read(flate.raw.len()) + 4096 * 1024;
let mut budget = ImageBudget::with_work(cost);
assert_eq!(budget.decode(&doc, &flate, 1 << 24).unwrap(), rows);
assert_eq!((budget.work_left(), budget.refused()), (0, 0), "exactly that");
@@ -590,6 +616,36 @@ mod image_tests {
assert_eq!((budget.work_left(), budget.refused()), (0, 1));
}
/// What Flate reads is priced for blocks that make nothing, and an image an encoder made pays
/// little for it: a scan-like picture stored in under half its size costs no more than half
/// again its bytes, samples and pixels. Redaction decodes a picture once for each page that draws it, so
/// a letterhead on hundreds of pages has to fit about as often as it did unpriced (here, at
/// most half again the cost).
#[test]
fn an_image_an_encoder_made_pays_little_for_its_reading() {
let doc = Document::new_empty();
let mut seed = 0x2545_F491_4F6C_DD1Du64;
let samples: Vec<u8> = (0..256 * 256usize)
.map(|i| {
seed ^= seed << 13;
seed ^= seed >> 7;
seed ^= seed << 17;
((i % 256) as u8 / 16) ^ (seed as u8 & 3)
})
.collect();
let mut dict = Dict::new();
dict.set(b"Width".to_vec(), Object::Int(256));
dict.set(b"Height".to_vec(), Object::Int(256));
let image = Stream::flate(dict, &samples);
assert!(image.raw.len() * 2 < samples.len(), "stored smaller: {}", image.raw.len());
let unpriced = (image.raw.len() + samples.len()) as u64 + 256 * 256;
let mut budget = ImageBudget::with_work(u64::MAX);
budget.decode(&doc, &image, 1 << 20).unwrap();
let paid = u64::MAX - budget.work_left();
assert_eq!(paid, unpriced + read(image.raw.len()));
assert!(2 * paid <= 3 * unpriced, "{paid} steps against {unpriced}");
}
/// An image pays for the pixels its data holds, not those its size declares: one claiming
/// 131072 by 131072 pixels with a few bytes of Flate costs what those bytes give, and the
/// images after it are decoded as they would be without it.
@@ -609,8 +665,8 @@ mod image_tests {
budget.decode(&doc, s, 1 << 24).unwrap();
u64::MAX - budget.work_left()
};
assert_eq!(cost(&claims), (claims.raw.len() + 16 + 16 * 8) as u64, "its bytes, and a pixel a bit of them");
assert_eq!(cost(&image), (image.raw.len() + 64 * 64) as u64 + 64 * 64, "its bytes, and its pixels");
assert_eq!(cost(&claims), (claims.raw.len() + 16 + 16 * 8) as u64 + read(claims.raw.len()), "its bytes, and a pixel a bit of them");
assert_eq!(cost(&image), (image.raw.len() + 64 * 64) as u64 + read(image.raw.len()) + 64 * 64, "its bytes, and its pixels");
for decode in [ImageBudget::decode, ImageBudget::decode_filters] {
let mut budget = ImageBudget::with_work(cost(&claims) + cost(&image));
assert_eq!(decode(&mut budget, &doc, &claims, 1 << 24).unwrap(), [0; 16]);
@@ -627,7 +683,7 @@ mod image_tests {
let samples = vec![7u8; 1000];
let raw = pdfcraft_filters::encode_flate(&samples);
let chain = [(Filter::Flate, Params::default())];
let cost = (raw.len() + samples.len()) as u64;
let cost = (raw.len() + samples.len()) as u64 + read(raw.len());
let mut budget = ImageBudget::with_work(cost + 10);
assert_eq!(budget.decode_chain(&chain, &raw, 1 << 20).unwrap(), samples);
assert_eq!(budget.work_left(), 10);
@@ -664,7 +720,7 @@ mod image_tests {
assert_eq!(budget.decode_filters(&doc, &image, 1 << 20).unwrap(), rows);
let cost = u64::MAX - budget.work_left();
// The tag byte of each of its 100 rows counts, which the predictor takes off.
assert_eq!(cost, (image.raw.len() + rows.len() + 100) as u64);
assert_eq!(cost, (image.raw.len() + rows.len() + 100) as u64 + read(image.raw.len()));
let mut budget = ImageBudget::with_work(cost - 1);
assert!(budget.decode_filters(&doc, &image, 1 << 20).is_err());
assert_eq!((budget.work_left(), budget.refused()), (0, 1));
@@ -769,7 +825,7 @@ mod tests {
assert_eq!(budget.remaining(), 0);
let mut budget = DecodeBudget::new((1 << 20) - 1, 1);
let Object::Stream(stream) = &*doc.get(bomb) else { panic!("not a stream") };
assert!(budget.decode_strict(stream).is_err());
assert!(budget.decode_strict(&doc, stream).is_err());
assert_eq!(budget.remaining(), 0);
let mut budget = DecodeBudget::new(10 << 20, 1);
@@ -793,14 +849,111 @@ mod tests {
let corrupt = Stream::from_raw(whole.dict.clone(), whole.raw.get(..cut).unwrap_or_default().to_vec());
let partial = corrupt.decoded().map(|d| d.len()).unwrap_or(0);
assert!(partial > 0, "a cut flate stream decodes in part");
let doc = Document::new_empty();
let mut budget = DecodeBudget::new(10 << 20, 1);
assert!(budget.decode_strict(&corrupt).is_err());
assert!(budget.decode_strict(&doc, &corrupt).is_err());
assert_eq!(budget.remaining(), (10 << 20) - partial);
let mut budget = DecodeBudget::new(10 << 20, 1);
assert!(budget.decode_within(&corrupt, 1 << 30).is_ok());
assert_eq!(budget.remaining(), (10 << 20) - partial);
}
/// A stream damaged partway decodes to what came before the damage, and says so, so that code
/// writing it back can leave it as it is. What every reader takes is no damage: a missing
/// checksum, no filter at all.
#[test]
fn a_stream_damaged_partway_says_so() {
let doc = Document::new_empty();
let read = |budget: &mut DecodeBudget, s: &Stream, max| budget.decode_with_damage(&doc, s, max).map(|(d, damaged)| (d.len(), damaged)).ok();
let whole = Stream::flate(Dict::new(), &vec![b' '; 1 << 20]);
let cut = Stream::from_raw(whole.dict.clone(), whole.raw.get(..whole.raw.len() / 2).unwrap_or_default().to_vec());
let partial = cut.decoded().map(|d| d.len()).unwrap_or(0);
assert!(partial > 0, "a cut flate stream decodes in part");
let mut budget = DecodeBudget::new(10 << 20, 1);
assert_eq!(read(&mut budget, &cut, usize::MAX), Some((partial, true)));
assert_eq!(budget.remaining(), (10 << 20) - partial);
// Within a bound of its own, under what is left.
assert_eq!(read(&mut budget, &cut, 4 << 20), Some((partial, true)));
assert_eq!(budget.decode(&cut).map(|d| d.len()).ok(), Some(partial));
let mut budget = DecodeBudget::new(10 << 20, 1);
assert_eq!(read(&mut budget, &whole, usize::MAX), Some((1 << 20, false)));
assert_eq!(read(&mut budget, &whole, 4 << 20), Some((1 << 20, false)));
let unchecked = Stream::from_raw(whole.dict.clone(), whole.raw.get(..whole.raw.len() - 4).unwrap_or_default().to_vec());
assert_eq!(read(&mut budget, &unchecked, usize::MAX), Some((1 << 20, false)));
assert_eq!(read(&mut budget, &Stream::from_raw(Dict::new(), b"0 0 m".to_vec()), usize::MAX), Some((5, false)));
// A later stage's damage is the stream's.
let mut dict = Dict::new();
dict.set(b"Filter".to_vec(), Object::Array(vec![Object::name("FlateDecode"), Object::name("ASCIIHexDecode")]));
let hex = Stream::from_raw(dict, Stream::flate(Dict::new(), b"4142zz4344>").raw.to_vec());
assert_eq!(budget.decode_with_damage(&doc, &hex, usize::MAX).ok(), Some((b"AB".to_vec(), true)));
// So is an earlier stage's, though the stage after it reads all it was given: a stored
// block of whole hex, then a block of the reserved type (RFC 1951 §3.2.3).
let readable = b"4142>";
let len = u16::try_from(readable.len()).unwrap_or_default();
let mut raw = vec![0x78, 0x01, 0x00];
raw.extend_from_slice(&len.to_le_bytes());
raw.extend_from_slice(&(!len).to_le_bytes());
raw.extend_from_slice(readable);
raw.push(0x07);
let flate_damaged = Stream::from_raw(hex.dict.clone(), raw);
assert_eq!(budget.decode_with_damage(&doc, &flate_damaged, usize::MAX).ok(), Some((b"AB".to_vec(), true)));
}
/// `/DecodeParms` given by reference, or a parameter given by reference in them, is read: a
/// predicted stream decodes as with them direct. Of the stream, decoding copies no more than
/// its filters and their parameters.
#[test]
fn parameters_given_by_reference_are_read() {
let content = b"q 10 0 0 10 0 0 cm /Im Do Q";
// PNG Up, one column: each byte a row of its own, tagged 2.
let mut rows = Vec::new();
let mut above = 0u8;
for &b in content {
rows.extend([2, b.wrapping_sub(above)]);
above = b;
}
let raw = Stream::flate(Dict::new(), &rows).raw;
let mut doc = Document::new_empty();
let direct: Dict = [(b"Predictor".to_vec(), Object::Int(12))].into_iter().collect();
let by_reference = Object::Ref(doc.add(Object::Dict(direct.clone())));
let predictor = doc.add(Object::Int(12));
let inner = Object::Dict([(b"Predictor".to_vec(), Object::Ref(predictor))].into_iter().collect());
for parms in [Object::Dict(direct), by_reference, inner] {
let mut dict = Dict::new();
dict.set(b"Filter".to_vec(), Object::name("FlateDecode"));
dict.set(b"DecodeParms".to_vec(), parms.clone());
dict.set(b"Other".to_vec(), Object::Array(vec![Object::Int(0); 1000]));
let s = Stream::from_raw(dict, raw.to_vec());
assert_eq!(DecodeBudget::page().decode_with_damage(&doc, &s, usize::MAX).ok(), Some((content.to_vec(), false)), "{parms:?}");
assert_eq!(DecodeBudget::page().decode_strict(&doc, &s).ok(), Some(content.to_vec()), "{parms:?}");
let r = doc.add(Object::Stream(s.clone()));
let read = DecodeBudget::page().decode_contents_with_damage(&doc, &Object::Array(vec![Object::Ref(r)]));
assert_eq!(read, [Ok((content.to_vec(), false))], "{parms:?}");
assert_eq!(DecodeBudget::page().decode_contents(&doc, &Object::Ref(r)), [Ok(content.to_vec())], "{parms:?}");
let view = doc.decoding_view(&s);
assert!(matches!(view, std::borrow::Cow::Borrowed(_)) || view.dict.len() == 2, "{:?}", view.dict);
}
}
/// A page's content streams each say whether they are damaged, in their places.
#[test]
fn a_pages_streams_each_say_whether_they_are_damaged() {
let mut doc = Document::new_empty();
let whole = doc.add(Object::Stream(Stream::flate(Dict::new(), b"0 0 m 1 1 l S")));
let Object::Stream(s) = &*doc.get(whole) else { panic!("not a stream") };
let cut = Stream::from_raw(s.dict.clone(), s.raw.get(..s.raw.len() - 6).unwrap_or_default().to_vec());
let cut = doc.add(Object::Stream(cut));
let contents = Object::Array(vec![Object::Ref(whole), Object::Null, Object::Ref(cut)]);
let mut budget = DecodeBudget::page();
let read: Vec<_> = budget.decode_contents_with_damage(&doc, &contents).into_iter().map(|r| r.map(|(_, damaged)| damaged).ok()).collect();
assert_eq!(read, [Some(false), Some(true)]);
let one: Vec<_> =
budget.decode_contents_with_damage(&doc, &Object::Ref(cut)).into_iter().map(|r| r.map(|(_, damaged)| damaged).ok()).collect();
assert_eq!(one, [Some(true)]);
assert_eq!(budget.decode_contents(&doc, &contents).iter().filter(|r| r.is_ok()).count(), 2, "read up to the damage");
}
/// A stream that fails its reading as zlib, and then as raw deflate after decoding all of it,
/// gives nothing back, and each reference still pays for what it decoded.
#[test]
@@ -832,7 +985,7 @@ mod tests {
assert_eq!(budget.remaining(), (10 << 20) - charged);
assert!(budget.decode_within(&chained, 2 << 20).is_ok());
assert_eq!(budget.remaining(), (10 << 20) - 2 * charged);
assert!(budget.decode_strict(&chained).is_ok());
assert!(budget.decode_strict(&Document::new_empty(), &chained).is_ok());
assert_eq!(budget.remaining(), (10 << 20) - 3 * charged);
}
}
crates/cos/src/document.rs
+9 -3
View File
@@ -24,11 +24,11 @@ pub(crate) use objects::Held;
pub(crate) use reader::find;
#[cfg(test)]
use reader::rfind;
pub(crate) use state::ObjectReader;
use state::{ObjStm, Slot};
use state::{ObjStm, Slot, Unreadable};
pub(crate) use state::{ObjectReader, held_bytes};
pub use state::{ReadState, SourceIdentity};
#[cfg(test)]
use state::{SECTION_READS, held_bytes, objstm_member};
use state::{SECTION_READS, objstm_member};
use xref::XrefIndex;
pub use xref::{Revision, XrefEntry};
@@ -48,8 +48,14 @@ pub struct Document {
/// `fill_xref_gaps`.
located: Arc<BTreeMap<u32, (u64, u16)>>,
repair_log: Arc<Vec<String>>,
/// What loading each file object gave, a failure as well: a damaged object is otherwise
/// loaded again on every read, and a member of an object stream that can't be decoded
/// decodes the stream again (`Document::load`).
cache: Arc<Mutex<state::ObjectCache>>,
objstms: Arc<Mutex<HashMap<u32, Arc<ObjStm>>>>,
/// See [`Self::unreadable_objects`]. Shared by clones, like the caches: it describes the
/// source, and is reset wherever they are.
unreadable: Arc<Mutex<Unreadable>>,
/// What `/JBIG2Globals` streams of the file decode to, by object number: the images of a scan
/// share them, and each would otherwise decode them again (see `Document::jbig2_globals`).
jbig2_globals: Arc<Mutex<HashMap<u32, Arc<Vec<u8>>>>>,
crates/cos/src/document/objects.rs
+123 -27
View File
@@ -1,7 +1,9 @@
use std::collections::HashSet;
use super::reader::find;
use super::state::{LENGTHS, LoadGuard, ObjStm, ObjectReader, ReadState, Slot, SourceIdentity, held_bytes, objstm_member};
use super::state::{
LENGTHS, LoadGuard, ObjStm, ObjectReader, ReadState, Slot, SourceIdentity, loaded_bytes, note_overlay_read, objstm_member, overlay_reads,
};
use super::xref::{Revision, XrefEntry, XrefIndex};
use super::{
Arc, CosError, Cow, Dict, Document, Lexer, MAX_DECODED, MAX_JBIG2_GLOBALS, ObjRef, Object, Stream, is_whitespace, parse_indirect_shared,
@@ -88,6 +90,39 @@ impl Document {
&self.repair_log
}
/// Objects of the file that could not be read, lowest number first, as lines such as
/// "object 11 0 could not be read: syntax error at byte 512: ...". [`Self::get`] reads each
/// as null (ISO 32000-2 §7.3.10), so a document with one opens, but what it held is lost:
/// unlike [`Self::repair_log`], nothing was repaired. Lists the objects read so far, by this
/// document, its clones or a scan of it, that have not since been edited; past 100, a last
/// line says there were more. Opening again, authenticating or rebuilding the index starts
/// the list afresh, as it does the object cache.
pub fn unreadable_objects(&self) -> Vec<String> {
let Ok(unreadable) = self.unreadable.lock() else { return Vec::new() };
unreadable.lines(|num| self.overlay.contains_key(&num))
}
/// Record `num` as unreadable. `reason` is built only for an object not yet listed, so a file
/// whose broken object is asked for again and again pays for it once.
fn note_unreadable(&self, num: u32, reason: impl FnOnce() -> String) {
// A poisoned lock only means the object goes unlisted; the read itself still fails.
let Ok(mut unreadable) = self.unreadable.lock() else { return };
let hidden = |n: u32| self.overlay.contains_key(&n);
if unreadable.wants(num, hidden) {
unreadable.note(num, self.generation(num), &reason(), hidden);
}
}
/// Why an object listed at `offset` could not be read: what parsing it there, without resolving
/// an indirect `/Length`, found. A best account, not the only one: the reader also tried
/// elsewhere.
fn why_unreadable(&self, offset: usize) -> String {
match parse_indirect_shared(&self.data, offset, &|_| None, &self.endstreams) {
Err(e) => e.to_string(),
Ok(_) => "no readable copy was found at its cross-reference offset or elsewhere in the file".into(),
}
}
/// Add a line to [`Self::repair_log`]: a leniency applied while reading that the user should
/// know about (fidelity: never silent).
pub fn note_repair(&mut self, line: String) {
@@ -288,7 +323,29 @@ impl Document {
/// ([`pdfcraft_filters::MAX_CHAIN`]). Only small values are copied, so a file can't have a
/// large object copied once for each key and filter.
pub fn with_direct_parms<'s>(&self, s: &'s Stream) -> Cow<'s, Stream> {
let Some(p) = s.dict.get(b"DecodeParms") else { return Cow::Borrowed(s) };
let Some(made) = self.direct_parms(s) else { return Cow::Borrowed(s) };
let mut s = s.clone();
s.dict.set(b"DecodeParms".to_vec(), made);
Cow::Owned(s)
}
/// [`Document::with_direct_parms`] for decoding alone: where that copies `s`, this makes a
/// stream of `s`'s data, shared, and a dictionary of its `/Filter` and the parameters made
/// direct, all its filters read. A stream named again and again, as a page's `/Contents` may
/// list one, isn't copied whole each time.
pub(crate) fn decoding_view<'s>(&self, s: &'s Stream) -> Cow<'s, Stream> {
let Some(made) = self.direct_parms(s) else { return Cow::Borrowed(s) };
let mut dict = Dict::new();
if let Some(f) = s.dict.get(b"Filter") {
dict.set(b"Filter".to_vec(), f.clone());
}
dict.set(b"DecodeParms".to_vec(), made);
Cow::Owned(Stream { dict, raw: s.raw.clone() })
}
/// The `/DecodeParms` [`Document::with_direct_parms`] gives `s`, or `None` when it keeps `s`.
fn direct_parms(&self, s: &Stream) -> Option<Object> {
let p = s.dict.get(b"DecodeParms")?;
let crypt = |f: &Object| matches!(f.as_name(), Some(b"Crypt"));
let filters: Vec<bool> = match s.dict.get(b"Filter") {
Some(Object::Array(a)) => a.iter().map(crypt).collect(),
@@ -296,7 +353,7 @@ impl Document {
None => Vec::new(),
};
if filters.iter().filter(|c| !**c).count() > pdfcraft_filters::MAX_CHAIN {
return Cow::Borrowed(s);
return None;
}
let mut changed = matches!(p, Object::Ref(_));
// Its numbers and flags kept, resolved; anything else is no filter's parameters (null).
@@ -323,12 +380,7 @@ impl Document {
}
_ => one(p, &mut changed),
};
if !changed {
return Cow::Borrowed(s);
}
let mut s = s.clone();
s.dict.set(b"DecodeParms".to_vec(), made);
Cow::Owned(s)
changed.then_some(made)
}
/// `o`, or the object it refers to, without copying either.
@@ -497,27 +549,38 @@ impl Document {
}
pub fn try_get(&self, num: u32) -> Result<Arc<Object>, CosError> {
match self.overlay.get(&num) {
Some(Slot::Set(_, o) | Slot::RepairedSet(_, _, o)) => return Ok(o.clone()),
Some(Slot::Freed(_)) => return Ok(Arc::new(Object::Null)),
None => {}
if let Some(slot) = self.overlay.get(&num) {
note_overlay_read();
return Ok(match slot {
Slot::Set(_, o) | Slot::RepairedSet(_, _, o) => o.clone(),
Slot::Freed(_) => Arc::new(Object::Null),
});
}
if let Some(o) = self.cache.lock().map_err(|_| CosError::Poisoned)?.get(&num) {
return Ok(o.clone());
if let Some(loaded) = self.cache.lock().map_err(|_| CosError::Poisoned)?.get(&num) {
return loaded.clone();
}
// Loading can re-enter (indirect stream `/Length`, object streams); damaged files make
// cycles such as `6 0 obj << /Length 6 0 R >>`. Bound the nesting on every path.
let _guard =
LoadGuard::enter(num).ok_or_else(|| CosError::Syntax { offset: 0, detail: format!("reference cycle while loading object {num}") })?;
let _scope = crate::security::FollowScope::enter();
let obj = self.load(num, 0)?;
let obj = Arc::new(obj);
// The editing cache is bounded like the scan reader's: re-parsing a file-backed view
// is cheap, and returned objects outlive eviction.
let owned = held_bytes(&obj, &self.data, 0);
self.cache.lock().map_err(|_| CosError::Poisoned)?.insert(num, obj.clone(), owned);
self.evict_editor_cache_if_over_budget(num)?;
Ok(obj)
let edits_read = overlay_reads();
let loaded = self.load(num, 0).map(Arc::new);
if let Err(e) = &loaded
&& !matches!(e, CosError::Poisoned)
{
self.note_unreadable(num, || e.to_string());
}
// A failure is kept like an object, or every read of a damaged object loads it again. A
// poisoned lock says nothing about the object. Clones share the cache, so a load that read
// this document's edits (an edited object stream or `/Length`) is not kept.
if overlay_reads() == edits_read && !matches!(loaded, Err(CosError::Poisoned)) {
// The editing cache is bounded like the scan reader's: re-parsing a file-backed view
// is cheap, and returned objects outlive eviction.
self.cache.lock().map_err(|_| CosError::Poisoned)?.insert(num, loaded.clone(), loaded_bytes(&loaded, &self.data));
self.evict_editor_cache_if_over_budget(num)?;
}
loaded
}
/// Bounds for the editing cache: roomier than the scan reader's (128 objects / 8 MB)
@@ -629,11 +692,20 @@ impl Document {
None | Some(XrefEntry::Free { .. }) => Ok(Object::Null),
Some(XrefEntry::InFile { offset, .. }) => {
let resolve = |r: ObjRef| {
if let Some(known) = LENGTHS.with(|m| m.borrow().get(&r).copied()) {
if self.overlay.contains_key(&r.num) {
note_overlay_read();
}
// A length resolved through this document's edits counts as an edit read
// on every reuse.
if let Some((known, edits)) = LENGTHS.with(|m| m.borrow().get(&r).copied()) {
if edits {
note_overlay_read();
}
return known;
}
let edits_read = overlay_reads();
let length = self.try_get_reference(r).ok().and_then(|o| o.as_int());
LENGTHS.with(|m| m.borrow_mut().insert(r, length));
LENGTHS.with(|m| m.borrow_mut().insert(r, (length, overlay_reads() != edits_read)));
length
};
let off = offset as usize;
@@ -646,7 +718,10 @@ impl Document {
.or_else(|| if self.header_offset == 0 { None } else { at(off.saturating_add(self.header_offset)) })
.or_else(|| self.located.get(&num).and_then(|(o, _)| usize::try_from(*o).ok()).and_then(at))
.or_else(|| self.scan_for(num).map(|(id, o)| self.decrypted(id, o)))
.ok_or(CosError::MissingObject(num))
.ok_or_else(|| {
self.note_unreadable(num, || self.why_unreadable(off));
CosError::MissingObject(num)
})
}
Some(XrefEntry::InStream { stream, index }) => {
let stm = self.objstm(stream)?;
@@ -659,7 +734,9 @@ impl Document {
}
pub(super) fn objstm(&self, num: u32) -> Result<Arc<ObjStm>, CosError> {
if let Some(s) = self.objstms.lock().map_err(|_| CosError::Poisoned)?.get(&num) {
// An edited or freed stream is this document's alone: clones share the parsed streams.
let edited = self.overlay.contains_key(&num);
if !edited && let Some(s) = self.objstms.lock().map_err(|_| CosError::Poisoned)?.get(&num) {
return Ok(s.clone());
}
// A file entry that cannot place the stream (one that says it is itself compressed
@@ -667,6 +744,7 @@ impl Document {
// header, as a plain object with a wrong offset is. An edited or freed stream, in the
// overlay or in a saved revision's table, is what that says it is: the file's copy
// must not come back.
let edits_read = overlay_reads();
let loaded = match self.try_get(num) {
Ok(o) if matches!(*o, Object::Stream(_)) => o,
_ if self.overlay.contains_key(&num) || matches!(self.xref_entry(num), Some(XrefEntry::Free { .. })) => {
@@ -674,6 +752,8 @@ impl Document {
}
_ => Arc::new(self.scan_for(num).map(|(id, o)| self.decrypted(id, o)).ok_or(CosError::MissingObject(num))?),
};
// A stream whose load read this document's edits (its `/Length`) is as much its own as an edited one.
let own = edited || overlay_reads() != edits_read;
let Object::Stream(s) = &*loaded else { return Err(CosError::MissingObject(num)) };
let n = s.dict.int(b"N").unwrap_or(0).clamp(0, 1_000_000) as usize;
let first = usize::try_from(s.dict.int(b"First").unwrap_or(0).max(0)).ok();
@@ -681,6 +761,8 @@ impl Document {
// the file's object streams comes here without a read around it.
let _scope = crate::security::FollowScope::enter();
let readable = crate::security::following(Cow::Borrowed(s), || self.with_direct_parms(s));
#[cfg(test)]
super::tests::OBJSTM_DECODES.with(|decodes| decodes.set(decodes.get() + 1));
let data = readable.decoded_within(self.stream_limit)?;
let mut lx = Lexer::new(&data, 0);
let mut index = Vec::with_capacity(n);
@@ -689,6 +771,9 @@ impl Document {
index.push(objstm_member(onum, off, first, data.len()));
}
let stm = Arc::new(ObjStm { index, data: Arc::new(data) });
if own {
return Ok(stm);
}
// Same bound as the object cache: decoded object streams can be megabytes each.
// Insert first so the retained `num` is actually in the map.
self.objstms.lock().map_err(|_| CosError::Poisoned)?.insert(num, stm.clone());
@@ -745,6 +830,7 @@ impl Document {
pub fn set(&mut self, r: ObjRef, obj: impl Into<Object>) {
let obj = self.adopt(obj.into());
self.overlay.insert(r.num, Slot::Set(r.generation, Arc::new(obj)));
self.forget_member_failures(r.num);
if r.num >= self.next_num {
self.next_num = r.num.saturating_add(1);
}
@@ -804,6 +890,16 @@ impl Document {
/// Delete an object (its number becomes free with the next generation).
pub fn free(&mut self, r: ObjRef) {
self.overlay.insert(r.num, Slot::Freed(r.generation.saturating_add(1)));
self.forget_member_failures(r.num);
}
/// Forget cached failures of the members `stream` held in the file: they were failures of
/// the file's copy, and an edit of `stream` may read. Clones share the cache, so theirs are
/// searched again too, which costs time only. A poisoned lock leaves the cache as it is.
fn forget_member_failures(&self, stream: u32) {
if let Ok(mut cache) = self.cache.lock() {
cache.forget_failures(|n| matches!(self.entries.lookup(*n, &self.data), Some(XrefEntry::InStream { stream: s, .. }) if s == stream));
}
}
/// Free what was added since the file was read and nothing refers to any more: the objects
crates/cos/src/document/open.rs
+3
View File
@@ -23,6 +23,7 @@ impl Document {
repair_log: Arc::new(Vec::new()),
cache: Arc::default(),
objstms: Arc::default(),
unreadable: Arc::default(),
jbig2_globals: Arc::default(),
stream_limit: crate::object::MAX_DECODED,
overlay: BTreeMap::new(),
@@ -110,6 +111,7 @@ impl Document {
repair_log: Arc::new(Vec::new()),
cache: Arc::default(),
objstms: Arc::default(),
unreadable: Arc::default(),
jbig2_globals: Arc::default(),
stream_limit: stream_limit.min(crate::object::MAX_DECODED),
overlay: BTreeMap::new(),
@@ -216,6 +218,7 @@ impl Document {
// Objects read while locating the catalog were read before decryption was possible.
self.cache = Arc::default();
self.objstms = Arc::default();
self.unreadable = Arc::default();
Ok(())
}
crates/cos/src/document/reader.rs
+43 -3
View File
@@ -1,5 +1,6 @@
#[cfg(test)]
use super::state::SECTION_READS;
use super::state::Unreadable;
use super::xref::{
MAX_COMPACT_XREF_BYTES, MAX_COMPACT_XREF_ROWS, Revision, XrefEntry, XrefIndex, XrefRange, XrefRevisionSection, XrefSection, XrefSectionData,
};
@@ -634,10 +635,20 @@ impl Document {
// Offsets of objects that may be cross-reference streams: their dictionaries are the
// trailers of files that have no `trailer` keyword (§7.5.8).
let mut xref_streams = Vec::new();
for (num, offset, generation, body) in object_headers(&data) {
let headers = object_headers(&data);
// Where `/XRef` occurs in the file, found in one pass.
let mut xref_marks = Vec::new();
let mut mark_from = 0;
while let Some(p) = find(&data, b"/XRef", mark_from, data.len()) {
xref_marks.push(p);
mark_from = p + 1;
}
for &(num, offset, generation, body) in &headers {
entries.insert(num, XrefEntry::InFile { offset, generation });
objstms.push(num);
if find(&data, b"/XRef", body, body.saturating_add(1024)).is_some()
// `/XRef` within the 1024 bytes after the header (the first one that is there).
let first = xref_marks.partition_point(|&p| p < body);
if xref_marks.get(first).is_some_and(|&p| p + 5 <= body.saturating_add(1024).min(data.len()))
&& let Ok(at) = usize::try_from(offset)
{
xref_streams.push(at);
@@ -648,6 +659,7 @@ impl Document {
}
self.entries = Arc::new(XrefIndex::Materialized(Arc::new(entries.clone()), Arc::default()));
self.cache.lock().map_err(|_| CosError::Poisoned)?.clear();
*self.unreadable.lock().map_err(|_| CosError::Poisoned)? = Unreadable::default();
// Objects inside object streams.
for num in objstms {
if let Ok(o) = self.try_get(num)
@@ -664,6 +676,8 @@ impl Document {
}
let object_count = entries.len();
self.entries = Arc::new(XrefIndex::Materialized(Arc::new(entries), Arc::default()));
// Failures from the scan stand: the members added change no scanned entry, and a
// `/Length` found only now cannot make a stream unreadable.
self.cache.lock().map_err(|_| CosError::Poisoned)?.clear();
// Trailer: the last `trailer` dictionary or cross-reference stream dictionary with a
// usable /Root (last in the file, whichever kind), else the catalog itself. One whose
@@ -823,7 +837,33 @@ pub(crate) fn find(hay: &[u8], needle: &[u8], from: usize, to: usize) -> Option<
if from >= to || needle.is_empty() || to - from < needle.len() {
return None;
}
hay[from..to].windows(needle.len()).position(|w| w == needle).map(|p| p + from)
let hay = &hay[from..to];
// Look for the needle's rarest byte first, a plain byte scan, and compare the rest only there.
const COMMON: &[u8] = b" etaoinsrhldcumfpgwybvkxjqz";
let (k, rare) =
needle.iter().copied().enumerate().max_by_key(|&(_, b)| COMMON.iter().position(|&c| c == b).unwrap_or(COMMON.len())).unwrap_or((0, 0));
let mut i = k;
while let Some(p) = hay.get(i..).and_then(|rest| position_of(rest, rare)) {
let start = i + p - k;
if hay.get(start..start + needle.len()) == Some(needle) {
return Some(start + from);
}
i += p + 1;
}
None
}
/// The first position of `byte` in `hay`. Whole 32-byte blocks are tested without branching, which
/// the compiler turns into vector compares; only a block that holds the byte is searched.
fn position_of(hay: &[u8], byte: u8) -> Option<usize> {
const BLOCK: usize = 32;
let (blocks, rest) = hay.as_chunks::<BLOCK>();
for (i, block) in blocks.iter().enumerate() {
if block.iter().fold(false, |any, &b| any | (b == byte)) {
return block.iter().position(|&b| b == byte).map(|p| i * BLOCK + p);
}
}
rest.iter().position(|&b| b == byte).map(|p| blocks.len() * BLOCK + p)
}
pub(super) fn rfind(hay: &[u8], needle: &[u8], from: usize) -> Option<usize> {
crates/cos/src/document/state.rs
+88 -9
View File
@@ -6,8 +6,20 @@ thread_local! {
static LOADING: std::cell::RefCell<Vec<u32>> = const { std::cell::RefCell::new(Vec::new()) };
/// Indirect `/Length` values resolved during the current outermost load: a damaged file
/// retries a parse several ways, and each retry would resolve the same lengths again
/// (exponential in the chain of streams whose lengths point at further streams).
pub(super) static LENGTHS: std::cell::RefCell<HashMap<ObjRef, Option<i64>>> = std::cell::RefCell::new(HashMap::new());
/// (exponential in the chain of streams whose lengths point at further streams). Each is
/// kept with whether resolving it read unsaved edits.
pub(super) static LENGTHS: std::cell::RefCell<HashMap<ObjRef, (Option<i64>, bool)>> = std::cell::RefCell::new(HashMap::new());
/// Unsaved edits read on this thread (`Document::try_get`, and a `/Length` in `Document::load`).
static OVERLAY_READS: std::cell::Cell<u64> = const { std::cell::Cell::new(0) };
}
/// Unsaved edits read so far on this thread: a load during which this changes depended on one.
pub(super) fn overlay_reads() -> u64 {
OVERLAY_READS.with(std::cell::Cell::get)
}
pub(super) fn note_overlay_read() {
OVERLAY_READS.with(|c| c.set(c.get().wrapping_add(1)));
}
/// Marks an object as being loaded on this thread. Loading can re-enter (indirect stream
@@ -30,6 +42,14 @@ impl LoadGuard {
}
}
/// What loading a file object gave, kept in the document's cache either way.
pub(super) type Loaded = Result<Arc<Object>, CosError>;
/// The bytes a load result holds of its own ([`held_bytes`]); a failure holds none.
pub(super) fn loaded_bytes(loaded: &Loaded, data: &Arc<Vec<u8>>) -> usize {
loaded.as_ref().map_or(0, |o| held_bytes(o, data, 0))
}
impl Drop for LoadGuard {
fn drop(&mut self) {
let outermost = LOADING.with(|l| {
@@ -61,6 +81,60 @@ pub(super) fn objstm_member(num: i64, offset: i64, first: Option<usize>, len: us
(start < len).then_some((num, start))
}
/// Objects the reader could not read, each with the reason (see `Document::unreadable_objects`).
/// Bounded: a document records past [`Unreadable::MAX`] of the objects it shows only the fact that
/// there were more. Entries a clone has edited are hidden from it and do not count, so the whole
/// list holds at most `MAX` plus the objects edited in clones.
#[derive(Debug, Default)]
pub(super) struct Unreadable {
pub(super) objects: BTreeMap<u32, (u16, String)>,
pub(super) more: bool,
}
impl Unreadable {
pub(super) const MAX: usize = 100;
/// Characters of a reason kept: a parser's message can quote the bytes it stopped at.
const REASON: usize = 160;
/// Whether `num` is to be recorded: not listed yet and there is room among the entries the
/// asking document still shows (`hidden` are the ones it has edited, as in [`Self::lines`]),
/// so a clone's edits never use up its own allowance. A full list notes that there were more,
/// so a caller asks this before building the reason.
pub(super) fn wants(&mut self, num: u32, hidden: impl Fn(u32) -> bool) -> bool {
if self.objects.contains_key(&num) {
return false;
}
if self.objects.keys().filter(|n| !hidden(**n)).count() >= Self::MAX {
self.more = true;
return false;
}
true
}
/// Record `num` once; the first reason given is kept.
pub(super) fn note(&mut self, num: u32, generation: u16, reason: &str, hidden: impl Fn(u32) -> bool) {
if self.wants(num, hidden) {
self.objects.insert(num, (generation, reason.chars().take(Self::REASON).collect()));
}
}
/// One line per recorded object not `hidden`, then one saying there were more, if there were.
pub(super) fn lines(&self, hidden: impl Fn(u32) -> bool) -> Vec<String> {
// Clones share the list and each hides its own edits, so the list may hold more than
// `MAX` entries in all; each document still shows at most `MAX`.
let mut shown = self.objects.iter().filter(|(num, _)| !hidden(**num));
let mut lines: Vec<String> = shown
.by_ref()
.take(Self::MAX)
.map(|(num, (generation, reason))| format!("object {num} {generation} could not be read: {reason}"))
.collect();
if self.more || shown.next().is_some() {
lines.push(format!("more objects could not be read; only the first {} are listed", Self::MAX));
}
lines
}
}
/// Where an overlay slot came from.
#[derive(Clone, Debug, PartialEq)]
pub(super) enum Slot {
@@ -182,7 +256,7 @@ thread_local! {
pub(super) static SECTION_READS: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
}
/// Parsed objects by number, each with what it holds ([`held_bytes`]) measured once, when it goes
/// What loading each object gave (a failure as well), by number, each with what it holds ([`held_bytes`]) measured once, when it goes
/// in, and their sum kept beside them.
///
/// Both caches are bounded by those bytes. Summing them on every read walked every cached object
@@ -191,12 +265,12 @@ thread_local! {
/// (#5873).
#[derive(Default)]
pub(crate) struct ObjectCache {
entries: HashMap<u32, (Arc<Object>, usize)>,
entries: HashMap<u32, (Loaded, usize)>,
owned: usize,
}
impl ObjectCache {
pub(crate) fn get(&self, num: &u32) -> Option<&Arc<Object>> {
pub(crate) fn get(&self, num: &u32) -> Option<&Loaded> {
self.entries.get(num).map(|(object, _)| object)
}
@@ -205,8 +279,8 @@ impl ObjectCache {
self.entries.contains_key(num)
}
/// Keep `object` as `num`, which holds `owned` bytes (what [`held_bytes`] says of it).
pub(crate) fn insert(&mut self, num: u32, object: Arc<Object>, owned: usize) {
/// Keep what loading `num` gave, which holds `owned` bytes ([`loaded_bytes`]).
pub(crate) fn insert(&mut self, num: u32, object: Loaded, owned: usize) {
if let Some((_, before)) = self.entries.insert(num, (object, owned)) {
self.owned = self.owned.saturating_sub(before);
}
@@ -232,6 +306,11 @@ impl ObjectCache {
self.owned = 0;
}
/// Drop the cached failures `stale` picks. A failure holds no bytes, so the total stands.
pub(crate) fn forget_failures(&mut self, mut stale: impl FnMut(&u32) -> bool) {
self.entries.retain(|num, (loaded, _)| loaded.is_ok() || !stale(num));
}
pub(crate) fn retain(&mut self, mut keep: impl FnMut(&u32) -> bool) {
self.entries.retain(|num, _| keep(num));
self.owned = self.entries.values().fold(0usize, |n, (_, owned)| n.saturating_add(*owned));
@@ -239,7 +318,7 @@ impl ObjectCache {
#[cfg(test)]
pub(crate) fn values(&self) -> impl Iterator<Item = &Arc<Object>> {
self.entries.values().map(|(object, _)| object)
self.entries.values().filter_map(|(object, _)| object.as_ref().ok())
}
}
@@ -256,7 +335,7 @@ const OWNED_BYTES_DEPTH: usize = 256;
/// InDesign's `/PieceInfo` -- thousands of small numeric arrays, parsed into tens of thousands of
/// values -- weighed nothing, so the caches kept a thousand of them: a 35 MB file held 1.5 GB
/// (#5873).
pub(super) fn held_bytes(object: &Object, data: &Arc<Vec<u8>>, depth: usize) -> usize {
pub(crate) fn held_bytes(object: &Object, data: &Arc<Vec<u8>>, depth: usize) -> usize {
if depth > OWNED_BYTES_DEPTH {
return usize::MAX;
}
crates/cos/src/document/tests.rs
+302 -8
View File
@@ -3,6 +3,8 @@ use super::*;
std::thread_local! {
/// How many times `Document::scan_for` read the file on this thread.
pub(super) static FILE_SCANS: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
/// How many times `Document::objstm` decoded an object stream on this thread.
pub(super) static OBJSTM_DECODES: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
}
/// A scan that doesn't find an object isn't made again: each reads the whole file, and a file can
@@ -730,6 +732,163 @@ fn an_entry_that_misses_its_object_is_found_by_searching_but_is_not_exact() {
assert!(doc.xref_entry_is_exact(1) && doc.xref_entry_is_exact(2));
}
/// An object that can't be read at its entry or anywhere else is searched for in the whole file
/// once: every later read, in a clone too, gives the same error without searching again. An
/// edit of that number still answers for it.
#[test]
fn an_unreadable_object_is_searched_for_once() {
let bodies = ["<< /Type /Catalog /Pages 2 0 R >>", "<< /Type /Pages /Kids [] /Count 0 >>", "<< /Odd Key(x) >>"];
let doc = Document::open(Arc::new(build(&bodies, "/Root 1 0 R /Info 3 0 R"))).unwrap();
FILE_SCANS.with(|scans| scans.set(0));
let first = doc.try_get(3);
assert!(first.is_err(), "{first:?}");
for _ in 0..10 {
assert_eq!(doc.try_get(3), first);
assert_eq!(*doc.get(ObjRef::new(3, 0)), Object::Null);
}
let mut edited = doc.clone();
assert_eq!(edited.try_get(3), first);
assert_eq!(FILE_SCANS.with(std::cell::Cell::get), 1, "searches for an object that isn't there");
let report = doc.unreadable_objects();
assert_eq!(report.len(), 1, "cached failures must stay listed exactly once: {report:?}");
assert_eq!(edited.unreadable_objects(), report, "clones retain the cached failure diagnostic");
edited.set(ObjRef::new(3, 0), Object::Int(7));
assert_eq!(edited.try_get(3).as_deref(), Ok(&Object::Int(7)));
assert_eq!(doc.try_get(3), first, "the edit is the clone's alone");
edited.free(ObjRef::new(3, 0));
assert_eq!(edited.try_get(3).as_deref(), Ok(&Object::Null));
}
/// A member of an object stream that can't be decoded (here, one that decodes past the document's
/// stream limit) is decoded once: every later read, in a clone too, gives the same error without
/// decoding the stream again. The stream itself loads, so no search of the file is involved.
#[test]
fn a_member_of_an_undecodable_object_stream_is_decoded_once() {
let file = object_stream_file("3 0 ", &format!("<< /Hidden ({}) >>", "x".repeat(4096)), 0);
let doc = Document::open_with_stream_limit(Arc::new(file), None, 1024).unwrap();
OBJSTM_DECODES.with(|decodes| decodes.set(0));
let first = doc.try_get(3);
assert!(matches!(first, Err(CosError::Filter(_))), "{first:?}");
for _ in 0..10 {
assert_eq!(doc.try_get(3), first);
}
assert_eq!(doc.clone().try_get(3), first);
assert_eq!(OBJSTM_DECODES.with(std::cell::Cell::get), 1, "decodes an object stream that can't be decoded");
}
/// Clones share the cache, and an edit is one clone's alone: a member of an object stream that
/// one clone freed or replaced still reads, in the others, from the file's stream.
#[test]
fn an_edited_object_stream_leaves_other_clones_members_alone() {
let file = Arc::new(object_stream_file("3 0 ", "<< /Hidden (yes) >>", 0));
let doc = Document::open(file.clone()).unwrap();
let mut freed = doc.clone();
freed.free(ObjRef::new(4, 0));
assert_eq!(freed.try_get(3), Err(CosError::MissingObject(4)));
assert_eq!(text_of(&doc, 3, b"Hidden").as_deref(), Some("yes"), "the free reached another clone");
let doc = Document::open(file).unwrap();
let mut replaced = doc.clone();
let mut dict = Dict::new();
dict.set(b"Type".to_vec(), Object::name("ObjStm"));
dict.set(b"N".to_vec(), Object::Int(1));
dict.set(b"First".to_vec(), Object::Int(4));
replaced.set(ObjRef::new(4, 0), crate::Stream::from_raw(dict, b"3 0 << /Hidden (no) >>".to_vec()));
assert_eq!(text_of(&replaced, 3, b"Hidden").as_deref(), Some("no"));
assert_eq!(text_of(&doc, 3, b"Hidden").as_deref(), Some("yes"), "the replacement reached another clone");
}
/// A member that failed to load from the file's object stream is kept as a failure, but replacing
/// or freeing that stream forgets it: the member then reads from the edit.
#[test]
fn an_edited_object_stream_forgets_its_members_failures() {
let file = Arc::new(object_stream_file("3 0 ", ") broken", 0));
let mut doc = Document::open(file.clone()).unwrap();
assert!(doc.try_get(3).is_err(), "the file's member does not parse");
let mut dict = Dict::new();
dict.set(b"Type".to_vec(), Object::name("ObjStm"));
dict.set(b"N".to_vec(), Object::Int(1));
dict.set(b"First".to_vec(), Object::Int(4));
doc.set(ObjRef::new(4, 0), crate::Stream::from_raw(dict, b"3 0 << /Hidden (fixed) >>".to_vec()));
assert_eq!(text_of(&doc, 3, b"Hidden").as_deref(), Some("fixed"), "the cached failure outlived the edit");
assert!(doc.try_get(3).is_ok());
let mut doc = Document::open(file).unwrap();
assert!(matches!(doc.try_get(3), Err(CosError::Syntax { .. })), "the file's member does not parse");
doc.free(ObjRef::new(4, 0));
assert_eq!(doc.try_get(3), Err(CosError::MissingObject(4)), "the freed stream's member");
}
/// An object stream framed by one clone's edit of its indirect `/Length` is that clone's alone:
/// the others still read its members within the file's length.
#[test]
fn an_edited_object_stream_length_leaves_other_clones_members_alone() {
let file = Arc::new(object_stream_file_with("3 0 ", "<< /Hidden (cut endstream here) >>", 0, true));
let doc = Document::open(file).unwrap();
let mut edited = doc.clone();
// Ends the stream at the `endstream` inside the member's string.
let cut = "3 0 << /Hidden (cut".len();
edited.set(ObjRef::new(6, 0), Object::Int(cut as i64));
assert_ne!(text_of(&edited, 3, b"Hidden").as_deref(), Some("cut endstream here"), "the edited length was not used");
assert_eq!(text_of(&doc, 3, b"Hidden").as_deref(), Some("cut endstream here"), "the edited length reached another clone");
}
/// A `/Length` read through one clone's edits and reused within the same load still keeps that
/// load to the clone. Here the file's offsets are header-relative behind junk, so object 10's
/// offset first lands on object 12, whose length (object 8, in the edited object stream 9) is
/// read; object 10's own length then loads object stream 4, framed by object 8 too.
#[test]
fn a_reused_edited_length_leaves_other_clones_members_alone() {
let mut out = b"%PDF-1.5\n".to_vec();
let mut offs = [0usize; 13];
let mut obj = |out: &mut Vec<u8>, n: usize, body: &[u8]| {
offs[n] = out.len();
out.extend_from_slice(format!("{n} 0 obj\n").as_bytes());
out.extend_from_slice(body);
out.extend_from_slice(b"\nendobj\n");
};
let stream = |dict: &str, data: &str| format!("<< {dict} >>\nstream\n{data}\nendstream").into_bytes();
obj(&mut out, 1, b"<< /Type /Catalog /Pages 2 0 R >>");
obj(&mut out, 2, b"<< /Type /Pages /Kids [] /Count 0 /Extra 3 0 R >>");
let inner = "11 0 3 2 5 << /Hidden (cut endstream here) >>";
obj(&mut out, 4, &stream("/Type /ObjStm /N 2 /First 9 /Length 8 0 R", inner));
let lengths = format!("8 0 {}", inner.len());
obj(&mut out, 9, &stream(&format!("/Type /ObjStm /N 1 /First 4 /Length {}", lengths.len()), &lengths));
obj(&mut out, 12, &stream("/Length 8 0 R", &"x".repeat(40)));
obj(&mut out, 10, &stream("/Length 11 0 R", "PDATA"));
// A cross-reference stream: a hybrid file's /XRefStm is not looked for behind junk.
let compressed = [(3, 4, 1), (8, 9, 0), (11, 4, 0)];
let mut data = Vec::new();
for (n, off) in offs.iter().enumerate() {
let row = match compressed.iter().find(|c| c.0 == n) {
Some(&(_, stream, index)) => [2, 0, stream, index],
None if *off == 0 => [0, 0, 0, 0],
None => {
let [.., hi, lo] = u32::try_from(*off).unwrap().to_be_bytes();
[1, hi, lo, 0]
}
};
data.extend_from_slice(&row);
}
let table = out.len();
let dict = format!("<< /Type /XRef /Size 14 /W [1 2 1] /Index [0 13] /Root 1 0 R /Length {} >>", data.len());
out.extend_from_slice(format!("13 0 obj\n{dict}\nstream\n").as_bytes());
out.extend_from_slice(&data);
out.extend_from_slice(format!("\nendstream\nendobj\nstartxref\n{table}\n%%EOF\n").as_bytes());
let mut file = vec![b'J'; offs[10] - offs[12] - 1];
file.push(b'\n');
file.extend_from_slice(&out);
let doc = Document::open(Arc::new(file)).unwrap();
let mut edited = doc.clone();
let cut = "11 0 3 2 5 << /Hidden (cut".len();
let mut dict = Dict::new();
dict.set(b"Type".to_vec(), Object::name("ObjStm"));
dict.set(b"N".to_vec(), Object::Int(1));
dict.set(b"First".to_vec(), Object::Int(4));
edited.set(ObjRef::new(9, 0), crate::Stream::from_raw(dict, format!("8 0 {cut}").into_bytes()));
assert!(matches!(&*edited.get(ObjRef::new(10, 0)), Object::Stream(s) if &s.raw[..] == b"PDATA"));
assert_ne!(text_of(&edited, 3, b"Hidden").as_deref(), Some("cut endstream here"), "the edited length was not used");
assert_eq!(text_of(&doc, 3, b"Hidden").as_deref(), Some("cut endstream here"), "the edited length reached another clone");
}
/// A classic table with `/Size` rows and object 0's free entry first, its subsection numbered
/// `1 n` instead of `0 n` (#2676): every row is one number off.
fn misnumbered_table(start: u32) -> Vec<u8> {
@@ -1255,7 +1414,7 @@ fn temporary_reader_keeps_file_views_and_evicts_data_of_its_own() {
// cleared after the next read, including a failed one.
let owned = Object::Stream(crate::Stream::from_raw(Dict::new(), vec![b'y'; ObjectReader::STREAM_BYTES + 1]));
let size = held_bytes(&owned, &reader.document.data, 0);
reader.document.cache.lock().unwrap().insert(3, Arc::new(owned), size);
reader.document.cache.lock().unwrap().insert(3, Ok(Arc::new(owned)), size);
assert_eq!(*reader.get(ObjRef::new(99, 0)), Object::Null);
assert!(reader.document.cache.lock().unwrap().is_empty(), "owned data over budget is evicted");
}
@@ -1468,8 +1627,14 @@ fn hybrid_reference_files_read_the_xref_stream_too() {
/// stream puts it at position `member` of object stream 4, which holds `header` (the number
/// and offset pairs) and then `members`.
fn object_stream_file(header: &str, members: &str, member: u8) -> Vec<u8> {
object_stream_file_with(header, members, member, false)
}
/// [`object_stream_file`], with the stream's `/Length` in object 6 when `indirect_length`.
fn object_stream_file_with(header: &str, members: &str, member: u8, indirect_length: bool) -> Vec<u8> {
let mut out = b"%PDF-1.5\n".to_vec();
let mut offs = [0usize; 6];
let size = if indirect_length { 7 } else { 6 };
let mut offs = vec![0usize; size];
let mut obj = |out: &mut Vec<u8>, n: usize, body: &[u8]| {
offs[n] = out.len();
out.extend_from_slice(format!("{n} 0 obj\n").as_bytes());
@@ -1480,7 +1645,8 @@ fn object_stream_file(header: &str, members: &str, member: u8) -> Vec<u8> {
obj(&mut out, 2, b"<< /Type /Pages /Kids [] /Count 0 /Extra 3 0 R >>");
let count = header.split_whitespace().count() / 2;
let inner = format!("{header}{members}");
let mut stm = format!("<< /Type /ObjStm /N {count} /First {} /Length {} >>\nstream\n", header.len(), inner.len()).into_bytes();
let length = if indirect_length { "6 0 R".to_string() } else { inner.len().to_string() };
let mut stm = format!("<< /Type /ObjStm /N {count} /First {} /Length {length} >>\nstream\n", header.len()).into_bytes();
stm.extend_from_slice(inner.as_bytes());
stm.extend_from_slice(b"\nendstream");
obj(&mut out, 4, &stm);
@@ -1489,13 +1655,16 @@ fn object_stream_file(header: &str, members: &str, member: u8) -> Vec<u8> {
xs.extend_from_slice(&data);
xs.extend_from_slice(b"\nendstream");
obj(&mut out, 5, &xs);
if indirect_length {
obj(&mut out, 6, inner.len().to_string().as_bytes());
}
let table = out.len();
out.extend_from_slice(b"xref\n0 6\n0000000000 65535 f \n");
out.extend_from_slice(format!("xref\n0 {size}\n0000000000 65535 f \n").as_bytes());
for (n, off) in offs.iter().enumerate().skip(1) {
let line = if n == 3 { "0000000000 65535 f \n".to_string() } else { format!("{off:010} 00000 n \n") };
out.extend_from_slice(line.as_bytes());
}
out.extend_from_slice(format!("trailer << /Size 6 /Root 1 0 R /XRefStm {} >>\nstartxref\n{table}\n%%EOF\n", offs[5]).as_bytes());
out.extend_from_slice(format!("trailer << /Size {size} /Root 1 0 R /XRefStm {} >>\nstartxref\n{table}\n%%EOF\n", offs[5]).as_bytes());
out
}
@@ -1849,6 +2018,105 @@ fn a_stream_whose_length_is_itself_and_never_ends_loads_quickly() {
rx.recv_timeout(std::time::Duration::from_secs(20)).expect("loading must not retry exponentially");
}
/// A file whose object 3 (its `/Info`) can't be parsed: the rest reads.
fn with_unreadable_info() -> Vec<u8> {
build(
&["<< /Type /Catalog /Pages 2 0 R >>", "<< /Type /Pages /Kids [] /Count 0 >>", "<< /Title (Report) /Author ) >>"],
"/Root 1 0 R /Info 3 0 R",
)
}
/// An object that can't be parsed reads as null and is listed once, by number and generation,
/// with why. Clones and scans share the list, an edit takes the object off it (in that document
/// only), and a sound file lists nothing.
#[test]
fn unreadable_objects_are_listed_with_the_reason() {
let doc = Document::open(Arc::new(with_unreadable_info())).unwrap();
assert!(doc.unreadable_objects().is_empty(), "nothing has read object 3 yet");
assert_eq!(*doc.get(ObjRef::new(3, 0)), Object::Null);
assert!(doc.try_get(3).is_err());
let lines = doc.unreadable_objects();
assert_eq!(lines.len(), 1, "read twice, listed once: {lines:?}");
assert!(lines[0].starts_with("object 3 0 could not be read: syntax error"), "{lines:?}");
let scanned = doc.clone();
assert_eq!(scanned.scan_objects_checked().filter(|(_, o)| o.is_err()).count(), 1);
assert_eq!(doc.unreadable_objects(), lines, "the scan found the same object");
let unread = Document::open(Arc::new(with_unreadable_info())).unwrap();
let scanned = unread.clone();
assert_eq!(scanned.scan_objects_checked().filter(|(_, o)| o.is_err()).count(), 1);
assert_eq!(unread.unreadable_objects(), lines, "a clone's scan reaches the original");
let mut edited = doc.clone();
edited.set(ObjRef::new(3, 0), Object::Dict(Dict::new()));
assert!(edited.unreadable_objects().is_empty(), "the edit replaced it");
assert_eq!(doc.unreadable_objects(), lines);
let sound =
build(&["<< /Type /Catalog /Pages 2 0 R >>", "<< /Type /Pages /Kids [] /Count 0 >>", "<< /Title (Report) >>"], "/Root 1 0 R /Info 3 0 R");
let sound = Document::open(Arc::new(sound)).unwrap();
assert_eq!(sound.scan_objects_checked().filter(|(_, o)| o.is_err()).count(), 0);
assert!(sound.unreadable_objects().is_empty());
}
/// A file rebuilt from its object headers still lists an object the scan could not read.
#[test]
fn a_rebuilt_file_lists_the_objects_its_scan_could_not_read() {
let text = "%PDF-1.7\n1 0 obj\n<< /Type /Catalog /Pages 2 0 R >>\nendobj\n2 0 obj\n<< /Type /Pages /Kids [] /Count 0 >>\nendobj\n3 0 obj\n<< /A ) >>\nendobj\n%%EOF\n";
let doc = Document::open(Arc::new(text.as_bytes().to_vec())).unwrap();
assert!(doc.repair_log().iter().any(|l| l.starts_with("rebuilt cross-reference index")), "{:?}", doc.repair_log());
let lines = doc.unreadable_objects();
assert_eq!(lines.len(), 1, "{lines:?}");
assert!(lines[0].starts_with("object 3 0 could not be read: syntax error"), "{lines:?}");
}
/// The list keeps at most 100 objects and then says there were more; an object whose
/// cross-reference offset holds no object is listed with that as its reason.
#[test]
fn the_unreadable_object_list_is_bounded() {
let mut bodies = vec!["<< /Type /Catalog /Pages 2 0 R >>", "<< /Type /Pages /Kids [] /Count 0 >>"];
bodies.extend(std::iter::repeat_n("<< /A ) >>", 120));
let doc = Document::open(Arc::new(build(&bodies, "/Root 1 0 R"))).unwrap();
assert_eq!(doc.scan_objects_checked().filter(|(_, o)| o.is_err()).count(), 120);
let lines = doc.unreadable_objects();
assert_eq!(lines.len(), 101, "{:?}", lines.last());
assert!(lines[0].starts_with("object 3 0 could not be read"), "{}", lines[0]);
assert_eq!(lines[100], "more objects could not be read; only the first 100 are listed");
let base = build(&["<< /Type /Catalog /Pages 2 0 R >>", "<< /Type /Pages /Kids [] /Count 0 >>", "(three)"], "/Root 1 0 R");
let text = String::from_utf8_lossy(&base).into_owned();
let gone = text.replacen("3 0 obj\n(three)", "9 0 obj\n(three)", 1);
let doc = Document::open(Arc::new(gone.into_bytes())).unwrap();
assert!(doc.try_get(3).is_err());
assert_eq!(
doc.unreadable_objects(),
["object 3 0 could not be read: no readable copy was found at its cross-reference offset or elsewhere in the file"]
);
}
/// The unreadable list is shared by clones, and each hides what it has edited. A clone that edited
/// every listed object still lists its own failures: hidden entries don't use up its allowance.
#[test]
fn objects_a_clone_edited_do_not_use_up_its_unreadable_allowance() {
let mut bodies = vec!["<< /Type /Catalog /Pages 2 0 R >>", "<< /Type /Pages /Kids [] /Count 0 >>"];
bodies.extend(std::iter::repeat_n("<< /A ) >>", 110));
let doc = Document::open(Arc::new(build(&bodies, "/Root 1 0 R"))).unwrap();
for num in 3..103 {
assert!(doc.try_get(num).is_err());
}
assert_eq!(doc.unreadable_objects().len(), 100);
let mut edited = doc.clone();
for num in 3..103 {
edited.set(ObjRef::new(num, 0), Object::Null);
}
for num in 103..113 {
assert!(edited.try_get(num).is_err());
}
let lines = edited.unreadable_objects();
assert_eq!(lines.len(), 10, "{lines:?}");
assert!(lines[0].starts_with("object 103 0 could not be read"), "{}", lines[0]);
let original = doc.unreadable_objects();
assert_eq!(original.len(), 101, "the original still shows at most 100, then says there were more");
assert_eq!(original[100], "more objects could not be read; only the first 100 are listed");
}
#[test]
fn a_restored_revision_keeps_what_the_saved_file_held_in_object_streams() {
// The earlier revision: an empty document read back from a file whose catalog and pages
@@ -1884,10 +2152,10 @@ fn the_object_cache_keeps_its_total_as_objects_come_and_go() {
let two = Arc::new(Object::Array(vec![Object::Int(1); 10]));
let (a, b) = (held_bytes(&one, &data, 0), held_bytes(&two, &data, 0));
let mut cache = super::state::ObjectCache::default();
cache.insert(1, one.clone(), a);
cache.insert(2, two, b);
cache.insert(1, Ok(one.clone()), a);
cache.insert(2, Ok(two), b);
assert_eq!(cache.owned(), a + b);
cache.insert(2, one, a);
cache.insert(2, Ok(one), a);
assert_eq!(cache.owned(), a + a, "a replaced object no longer counts");
cache.retain(|n| *n == 1);
assert_eq!((cache.len(), cache.owned()), (1, a));
@@ -1921,3 +2189,29 @@ fn the_editor_cache_charges_a_dictionary_for_the_values_it_holds() {
let sum = cache.values().fold(0usize, |n, o| n.saturating_add(held_bytes(o, &doc.data, 0)));
assert_eq!(cache.owned(), sum, "the kept total is the cache's own");
}
#[test]
fn find_agrees_with_a_plain_window_search() {
let naive = |hay: &[u8], needle: &[u8], from: usize, to: usize| {
if from >= to || needle.is_empty() || to - from < needle.len() {
return None;
}
hay[from..to].windows(needle.len()).position(|w| w == needle).map(|p| p + from)
};
let mut seed = 0x9E37_79B9_7F4A_7C15u64;
let mut next = move || {
seed ^= seed << 13;
seed ^= seed >> 7;
seed ^= seed << 17;
seed
};
let alphabet = b"/XRef t\n<>0";
for _ in 0..20_000 {
let hay: Vec<u8> = (0..next() % 200).map(|_| alphabet[(next() % alphabet.len() as u64) as usize]).collect();
let needles: [&[u8]; 5] = [b"/XRef", b"trailer", b"R", b"ff", b"<<"];
let needle = needles[(next() % 5) as usize];
let from = (next() % (hay.len() as u64 + 1)) as usize;
let to = from + (next() % (hay.len() as u64 + 1 - from as u64)) as usize;
assert_eq!(find(&hay, needle, from, to), naive(&hay, needle, from, to), "{hay:?} {needle:?} {from} {to}");
}
}
crates/cos/src/object.rs
+17 -6
View File
@@ -297,13 +297,18 @@ impl Stream {
/// [`Stream::decoded_within`] for a caller charging an allowance: the filter error is kept
/// typed, so a caller can tell a spent limit from corrupt data, `max` bounds all filter
/// stages together, and the result gives the bytes they decoded (the work done, on success
/// and on failure).
pub(crate) fn decoded_within_counted(&self, max: usize) -> Result<(Vec<u8>, usize), (pdfcraft_filters::FilterError, usize)> {
/// stages together, and the result gives whether the data stopped at damage (it is then what
/// decoded before it) and the bytes the stages decoded (the work done, on success and on
/// failure).
pub(crate) fn decoded_within_counted(&self, max: usize) -> Result<(Vec<u8>, bool, usize), (pdfcraft_filters::FilterError, usize)> {
let mut damaged = false;
let decode = |chain: &[(Filter, Params)], data: &[u8], max| {
pdfcraft_filters::decode_tolerant_counted(chain, data, max).map(|(v, _, spent)| (v, spent))
pdfcraft_filters::decode_tolerant_counted(chain, data, max).map(|(v, partial, spent)| {
damaged = partial;
(v, spent)
})
};
self.decode_chain(max, decode)
self.decode_chain(max, decode).map(|(v, spent)| (v, damaged, spent))
}
/// [`Stream::decoded_within`] spending from `work`, what an operation may still do across a
@@ -337,8 +342,14 @@ impl Stream {
/// Decoded data, bounded by [`MAX_DECODED`]; any corruption is an error.
pub fn decoded_strict(&self) -> Result<Vec<u8>, CosError> {
self.decoded_strict_within(MAX_DECODED)
}
/// Like [`Stream::decoded_strict`], bounded by `max` (and [`MAX_DECODED`]): for data that
/// will be copied elsewhere, where a damaged stream must not pass for a shorter one.
pub fn decoded_strict_within(&self, max: usize) -> Result<Vec<u8>, CosError> {
let decode = |chain: &[(Filter, Params)], data: &[u8], max| pdfcraft_filters::decode(chain, data, max).map(|v| (v, 0)).map_err(|e| (e, 0));
self.decode_chain(MAX_DECODED, decode).map(|(v, _)| v).map_err(|(e, _)| CosError::Filter(e.to_string()))
self.decode_chain(max, decode).map(|(v, _)| v).map_err(|(e, _)| CosError::Filter(e.to_string()))
}
/// [`Stream::decoded_strict`] bounded by `max` and counted like
crates/cos/src/writer.rs
+129 -21
View File
@@ -15,7 +15,7 @@ use std::io::Write;
use std::sync::Arc;
use crate::CosError;
use crate::document::{Document, ObjectReader, XrefEntry};
use crate::document::{Document, ObjectReader, XrefEntry, held_bytes};
use crate::object::{Dict, ObjRef, Object, PdfString, Stream};
/// Serialize one object. Deterministic: same object → same bytes.
@@ -23,9 +23,7 @@ pub fn serialize(o: &Object, out: &mut Vec<u8>) {
match o {
Object::Null => out.extend_from_slice(b"null"),
Object::Bool(b) => out.extend_from_slice(if *b { b"true" } else { b"false" }),
Object::Int(i) => {
let _ = write!(out, "{i}");
}
Object::Int(i) => write_int(*i, out),
Object::Real(r) => write_real(*r, out),
Object::String(s) => write_string(s, out),
Object::Name(n) => write_name(n, out),
@@ -42,11 +40,46 @@ pub fn serialize(o: &Object, out: &mut Vec<u8>) {
Object::Dict(d) => write_dict(d, out),
Object::Stream(s) => write_stream(s, out),
Object::Ref(r) => {
let _ = write!(out, "{} {} R", r.num, r.generation);
write_digits(u64::from(r.num), out);
out.push(b' ');
write_digits(u64::from(r.generation), out);
out.extend_from_slice(b" R");
}
}
}
/// Upper-case hexadecimal digits.
const HEX: &[u8; 16] = b"0123456789ABCDEF";
/// The decimal digits of `v`, the same bytes `write!(out, "{v}")` gives, without the formatter.
fn write_digits(mut v: u64, out: &mut Vec<u8>) {
// `u64::MAX` has 20 digits, so `at` never goes below 0.
let mut buf = [0u8; 20];
let mut at = buf.len();
loop {
at -= 1;
buf[at] = b'0' + (v % 10) as u8;
v /= 10;
if v == 0 {
break;
}
}
out.extend_from_slice(&buf[at..]);
}
/// A signed integer in decimal, as `write!(out, "{i}")` writes it.
fn write_int(i: i64, out: &mut Vec<u8>) {
if i < 0 {
out.push(b'-');
}
write_digits(i.unsigned_abs(), out);
}
/// `b` as two upper-case hexadecimal digits.
fn hex_pair(b: u8) -> [u8; 2] {
[HEX[usize::from(b >> 4)], HEX[usize::from(b & 15)]]
}
fn write_real(r: f64, out: &mut Vec<u8>) {
if !r.is_finite() {
out.push(b'0');
@@ -55,7 +88,7 @@ fn write_real(r: f64, out: &mut Vec<u8>) {
// A whole number is written as an integer only within the integer range Annex C gives
// (±2³¹−1), so `4e9` or `1e30` stays a real token for readers with 32-bit integers.
if r.fract() == 0.0 && r.abs() <= f64::from(i32::MAX) {
let _ = write!(out, "{}", r as i32);
write_int(i64::from(r as i32), out);
return;
}
// Plain decimal notation (§7.3.3 has no exponents). Rust's `Display` for f64 prints the
@@ -76,7 +109,8 @@ fn write_name(n: &[u8], out: &mut Vec<u8>) {
if regular {
out.push(b);
} else {
let _ = write!(out, "#{b:02X}");
out.push(b'#');
out.extend_from_slice(&hex_pair(b));
}
}
}
@@ -84,9 +118,10 @@ fn write_name(n: &[u8], out: &mut Vec<u8>) {
fn write_string(s: &PdfString, out: &mut Vec<u8>) {
let binary = s.bytes.iter().filter(|b| !(b' '..=b'~').contains(*b) && !matches!(b, b'\n' | b'\r' | b'\t')).count();
if s.hex || binary * 4 > s.bytes.len().max(1) {
out.reserve(s.bytes.len().saturating_mul(2).saturating_add(2));
out.push(b'<');
for b in &s.bytes {
let _ = write!(out, "{b:02X}");
for &b in &s.bytes {
out.extend_from_slice(&hex_pair(b));
}
out.push(b'>');
return;
@@ -105,7 +140,7 @@ fn write_string(s: &PdfString, out: &mut Vec<u8>) {
0x0C => out.extend_from_slice(b"\\f"),
b' '..=b'~' => out.push(b),
_ => {
let _ = write!(out, "\\{b:03o}");
out.extend_from_slice(&[b'\\', b'0' + (b >> 6), b'0' + ((b >> 3) & 7), b'0' + (b & 7)]);
}
}
}
@@ -275,14 +310,22 @@ pub fn write_incremental(doc: &Document, opts: &SaveOptions) -> Result<Vec<u8>,
/// The objects [`write_full`] writes: everything reachable from the trailer's /Root, /Info and
/// /Encrypt and from its unknown entries, breadth-first, with the new number each gets
/// (aliases of one slot included) and, in visit order, whether it is a stream.
type SaveOrder = (HashMap<ObjRef, u32>, Vec<(ObjRef, bool)>);
/// (aliases of one slot included) and, in visit order, whether it is a stream and the parsed
/// object where the budget allowed keeping it.
type SaveOrder = (HashMap<ObjRef, u32>, Vec<(ObjRef, bool)>, Vec<Option<Arc<Object>>>);
fn full_save_order(reader: &mut ObjectReader, trailer_in: &Dict) -> Result<SaveOrder, CosError> {
/// Number the objects reachable from the trailer. `retain` keeps parsed objects for the write
/// pass, up to the budget. A caller that needs only the numbers passes `false`, so the traversal
/// holds no parsed objects beyond the current one.
fn full_save_order(reader: &mut ObjectReader, trailer_in: &Dict, retain: bool) -> Result<SaveOrder, CosError> {
// Breadth-first walk from the trailer, assigning new numbers in visit order.
let mut map: HashMap<ObjRef, u32> = HashMap::new();
// Record the type while traversing so classifying packed objects needs no second parse.
let mut order: Vec<(ObjRef, bool)> = Vec::new();
// The parsed object of each entry in `order`, kept for the write pass while the budget
// allows it. Past the budget the write pass reads the object again.
let mut kept: Vec<Option<Arc<Object>>> = Vec::new();
let mut kept_budget = KeptObjects::default();
let mut queue: VecDeque<ObjRef> = VecDeque::new();
let visit_refs = |o: &Object, queue: &mut VecDeque<ObjRef>| collect_refs(o, &mut |r| queue.push_back(r));
for key in [&b"Root"[..], b"Info", b"Encrypt"] {
@@ -321,8 +364,9 @@ fn full_save_order(reader: &mut ObjectReader, trailer_in: &Dict) -> Result<SaveO
map.insert(canonical, num);
order.push((canonical, matches!(*o, Object::Stream(_))));
visit_refs(&o, &mut queue);
kept.push((retain && kept_budget.admit(&o, reader.document.bytes())).then_some(o));
}
Ok((map, order))
Ok((map, order, kept))
}
/// The object numbers a full save of `doc` would write: exactly the objects [`write_full`]
@@ -331,7 +375,7 @@ fn full_save_order(reader: &mut ObjectReader, trailer_in: &Dict) -> Result<SaveO
pub fn reachable_objects(doc: &Document) -> Result<HashSet<u32>, CosError> {
let mut reader = doc.object_reader();
let trailer_in = reader.document.trailer().clone();
let (_, order) = full_save_order(&mut reader, &trailer_in)?;
let (_, order, _) = full_save_order(&mut reader, &trailer_in, false)?;
Ok(order.into_iter().map(|(r, _)| r.num).collect())
}
@@ -345,7 +389,7 @@ pub fn write_full(doc: &Document, opts: &SaveOptions) -> Result<Vec<u8>, CosErro
let mut reader = doc.object_reader();
stamp_mod_date(&mut reader.document, opts);
let trailer_in = reader.document.trailer().clone();
let (map, order) = full_save_order(&mut reader, &trailer_in)?;
let (map, order, mut kept) = full_save_order(&mut reader, &trailer_in, true)?;
let version = reader.document.version().chars().take(3).collect::<String>();
let version = if opts.object_streams && version.as_str() < "1.5" { "1.5".to_string() } else { version };
let mut out = Vec::new();
@@ -375,14 +419,14 @@ pub fn write_full(doc: &Document, opts: &SaveOptions) -> Result<Vec<u8>, CosErro
let mut rows: BTreeMap<u32, Row> = BTreeMap::new();
// Keep only references, not renamed copies of the entire object graph. Standalone
// objects still precede object streams, preserving the established output order.
let mut packed: Vec<(u32, ObjRef)> = Vec::new();
let mut packed: Vec<(u32, ObjRef, Option<Arc<Object>>)> = Vec::new();
for (i, (r, is_stream)) in order.iter().enumerate() {
let num = output_number(i)?;
let in_stream = opts.object_streams && !is_stream && !encryption_objects.contains(&num) && root_outside != Some(num);
if in_stream {
packed.push((num, *r));
packed.push((num, *r, kept[i].take()));
} else {
let o = reread(&mut reader, *r, *is_stream)?;
let o = written_object(&mut reader, *r, *is_stream, kept[i].take())?;
let o = renumber(&prepared(&reader.document, r.num, num, 0, &o), &map);
rows.insert(num, Row::InFile(out.len() as u64, 0));
write_indirect(num, 0, &o, &mut out);
@@ -392,9 +436,9 @@ pub fn write_full(doc: &Document, opts: &SaveOptions) -> Result<Vec<u8>, CosErro
// individually: the whole stream is (§7.5.7, §7.6.2).
let mut next = output_number(order.len())?;
let mut batch = ObjectStreamBatch::default();
for (num, reference) in packed {
for (num, reference, parsed) in packed {
let mut encoded = Vec::new();
serialize(&renumber(&*reread(&mut reader, reference, false)?, &map), &mut encoded);
serialize(&renumber(&*written_object(&mut reader, reference, false, parsed)?, &map), &mut encoded);
if batch.count > 0 && batch.body.len().saturating_add(encoded.len()).saturating_add(1) > OBJSTM_BYTES {
batch.flush(&reader.document, &mut next, &mut rows, &mut out)?;
}
@@ -428,6 +472,38 @@ pub fn write_full(doc: &Document, opts: &SaveOptions) -> Result<Vec<u8>, CosErro
Ok(out)
}
/// What `write_full` keeps of the objects it parsed while numbering them, measured with
/// [`held_bytes`] (the parsed tree and any data that is not a view into the file). Beyond the
/// budget objects are read again when written, which bounds the memory of a save of a huge
/// document.
#[derive(Default)]
struct KeptObjects {
bytes: usize,
}
impl KeptObjects {
const BYTES: usize = 64 * 1024 * 1024;
/// Whether `object` still fits the budget; if so it is counted against it.
fn admit(&mut self, object: &Object, data: &Arc<Vec<u8>>) -> bool {
let bytes = self.bytes.saturating_add(held_bytes(object, data, 0));
if bytes > Self::BYTES {
return false;
}
self.bytes = bytes;
true
}
}
/// The object to write for `r`: the one parsed while numbering when it was kept, else `r` read
/// again.
fn written_object(reader: &mut ObjectReader, r: ObjRef, is_stream: bool, kept: Option<Arc<Object>>) -> Result<Arc<Object>, CosError> {
match kept {
Some(o) => Ok(o),
None => reread(reader, r, is_stream),
}
}
/// Assemble the full rewrite's trailer and apply its identifier policy.
fn full_save_trailer(doc: &Document, trailer_in: &Dict, map: &HashMap<ObjRef, u32>, opts: &SaveOptions, out: &[u8]) -> Dict {
let mut trailer = Dict::new();
@@ -935,6 +1011,38 @@ mod tests {
Lexer::new(&out, 0).object().expect("parses")
}
#[test]
fn kept_objects_stop_at_their_byte_budget() {
let data = Arc::new(Vec::new());
let string = Object::String(PdfString::literal(vec![b'x'; 1000]));
let array = Object::Array(vec![Object::Int(1), Object::Null]);
let mut kept = KeptObjects::default();
assert!(kept.admit(&string, &data) && kept.admit(&array, &data));
assert_eq!(kept.bytes, held_bytes(&string, &data, 0) + held_bytes(&array, &data, 0));
let mut full = KeptObjects { bytes: KeptObjects::BYTES - held_bytes(&string, &data, 0) + 1 };
let before = full.bytes;
assert!(!full.admit(&string, &data), "one byte over the budget");
assert_eq!(full.bytes, before, "a refused object is not counted");
let mut deep = Object::Null;
for _ in 0..300 {
deep = Object::Array(vec![deep]);
}
assert!(!KeptObjects::default().admit(&deep, &data), "nesting past the limit is never kept");
}
#[test]
fn kept_objects_count_names_and_dictionary_keys() {
let data = Arc::new(Vec::new());
let name = Object::Name(vec![b'n'; KeptObjects::BYTES + 1]);
let mut kept = KeptObjects::default();
assert!(!kept.admit(&name, &data), "a name over the byte budget is not kept");
assert_eq!(kept.bytes, 0);
let mut dict = Dict::new();
dict.set(vec![b'k'; KeptObjects::BYTES + 1], Object::Null);
assert!(!kept.admit(&Object::Dict(dict), &data), "nor a dictionary key");
assert_eq!(kept.bytes, 0);
}
#[test]
fn a_second_read_that_differs_fails_the_save_instead_of_writing_null() {
let mut doc = Document::new_empty();
crates/cos/tests/it/main.rs
+1
View File
@@ -5,5 +5,6 @@ mod corpus;
mod encryption;
mod generations;
mod proptests;
mod serialize_format;
mod shared_streams;
mod writer;
crates/cos/tests/it/serialize_format.rs
+116
View File
@@ -0,0 +1,116 @@
//! `serialize` writes integers, references, hexadecimal strings, octal escapes and `#xx` name
//! escapes with its own digit loops. They must give the bytes the standard formatter gives.
use pdfcraft_cos::{ObjRef, Object, PdfString, serialize};
use proptest::prelude::*;
fn bytes_of(o: &Object) -> Vec<u8> {
let mut v = Vec::new();
serialize(o, &mut v);
v
}
#[test]
fn integer_extremes_and_digit_boundaries() {
let mut cases = vec![0i64, 1, -1, 9, 10, 99, 100, i64::MAX, i64::MIN, i64::MIN + 1, i64::from(i32::MAX), i64::from(i32::MIN)];
let mut p = 1i64;
while let Some(next) = p.checked_mul(10) {
cases.extend([next - 1, next, next + 1, -next]);
p = next;
}
for i in cases {
assert_eq!(bytes_of(&Object::Int(i)), i.to_string().into_bytes(), "{i}");
}
}
#[test]
fn reference_extremes() {
for (n, g) in [(0, 0), (1, 0), (u32::MAX, u16::MAX), (10, 65535), (4_000_000_000, 7)] {
assert_eq!(bytes_of(&Object::Ref(ObjRef::new(n, g))), format!("{n} {g} R").into_bytes());
}
}
#[test]
fn every_byte_value_in_hex_strings_octal_escapes_and_names() {
let all: Vec<u8> = (0..=255).collect();
let hex: String = all.iter().map(|b| format!("{b:02X}")).collect();
assert_eq!(bytes_of(&Object::String(PdfString { bytes: all.clone(), hex: true })), format!("<{hex}>").into_bytes());
// A literal string with a few control bytes escapes them in octal.
let lit = bytes_of(&Object::String(PdfString { bytes: b"abcdefghijkl\x01\x7f\xffz".to_vec(), hex: false }));
assert_eq!(lit, b"(abcdefghijkl\\001\\177\\377z)");
let name = bytes_of(&Object::Name(all.clone()));
let mut want = b"/".to_vec();
for &b in &all {
let regular = (b'!'..=b'~').contains(&b) && !matches!(b, b'(' | b')' | b'<' | b'>' | b'[' | b']' | b'{' | b'}' | b'/' | b'%' | b'#');
if regular {
want.push(b);
} else {
want.extend_from_slice(format!("#{b:02X}").as_bytes());
}
}
assert_eq!(name, want);
}
#[test]
fn whole_real_at_i32_min_stays_a_real_token() {
// -2^31 is outside the Annex C integer range (±2^31-1), so it keeps its trailing point.
assert_eq!(bytes_of(&Object::Real(f64::from(i32::MIN))), b"-2147483648.");
}
proptest! {
#[test]
fn integers_match_the_formatter(i in any::<i64>()) {
prop_assert_eq!(bytes_of(&Object::Int(i)), format!("{i}").into_bytes());
}
#[test]
fn whole_reals_match_the_formatter(i in -i32::MAX..=i32::MAX) {
prop_assert_eq!(bytes_of(&Object::Real(f64::from(i))), format!("{i}").into_bytes());
}
#[test]
fn references_match_the_formatter(n in any::<u32>(), g in any::<u16>()) {
prop_assert_eq!(bytes_of(&Object::Ref(ObjRef::new(n, g))), format!("{n} {g} R").into_bytes());
}
#[test]
fn hex_strings_match_the_formatter(bytes in proptest::collection::vec(any::<u8>(), 0..300)) {
let mut want = String::from("<");
for b in &bytes {
want.push_str(&format!("{b:02X}"));
}
want.push('>');
prop_assert_eq!(bytes_of(&Object::String(PdfString { bytes, hex: true })), want.into_bytes());
}
#[test]
fn binary_literal_strings_match_the_formatter(bytes in proptest::collection::vec(any::<u8>(), 0..300)) {
// Mostly-binary input is written as hex; mostly-text input with escapes is a literal.
let got = bytes_of(&Object::String(PdfString { bytes: bytes.clone(), hex: false }));
let binary = bytes.iter().filter(|b| !(b' '..=b'~').contains(*b) && !matches!(b, b'\n' | b'\r' | b'\t')).count();
let mut want = Vec::new();
if binary * 4 > bytes.len().max(1) {
want.push(b'<');
for b in &bytes {
want.extend_from_slice(format!("{b:02X}").as_bytes());
}
want.push(b'>');
} else {
want.push(b'(');
for &b in &bytes {
match b {
b'(' | b')' | b'\\' => want.extend_from_slice(&[b'\\', b]),
b'\n' => want.extend_from_slice(b"\\n"),
b'\r' => want.extend_from_slice(b"\\r"),
b'\t' => want.extend_from_slice(b"\\t"),
0x08 => want.extend_from_slice(b"\\b"),
0x0C => want.extend_from_slice(b"\\f"),
b' '..=b'~' => want.push(b),
_ => want.extend_from_slice(format!("\\{b:03o}").as_bytes()),
}
}
want.push(b')');
}
prop_assert_eq!(got, want);
}
}
crates/create/src/extract.rs
+21 -9
View File
@@ -183,12 +183,17 @@ fn image(doc: &Document, s: &Stream, budget: &mut ImageBudget) -> Result<(&'stat
.unwrap_or(false);
// In a PDF a CMYK JPEG's samples are ink amounts unless /Decode inverts them, while image
// viewers read a CMYK JPEG file as inverted (Adobe's convention): as is, it would show as
// a negative.
let as_is_negative = jpeg && !inverted && matches!(space(doc, s.dict.get(b"ColorSpace")), Ok(Space::Cmyk));
// a negative. The space is read once: an indexed one's lookup table is paid for each time.
let read = (jpeg && !inverted).then(|| space(doc, s.dict.get(b"ColorSpace"), budget));
let as_is_negative = matches!(read, Some(Ok(Space::Cmyk)));
if jpeg && alpha.is_none() && key.is_none() && !as_is_negative {
return as_is();
}
let mut space = if mask { Space::Gray } else { space(doc, s.dict.get(b"ColorSpace"))? };
let mut space = match read {
_ if mask => Space::Gray,
Some(read) => read?,
None => space(doc, s.dict.get(b"ColorSpace"), budget)?,
};
let (samples, bpc) = if jpeg {
if !budget.pay_jpeg((w as u64).saturating_mul(h as u64).saturating_mul(space.components() as u64), &s.raw) {
return Err(FilterError::Spent.to_string());
@@ -436,7 +441,10 @@ impl Space {
/// A lookup table holds at most 256 entries of 4 bytes; decoding stops well past that.
const MAX_LOOKUP: usize = 64 << 10;
fn space(doc: &Document, cs: Option<&Object>) -> Result<Space, String> {
/// The colour space `cs` names; an indexed one's lookup table is read within `budget`, as the
/// image's samples are, since each image reads it again and its filters can do work that makes
/// nothing.
fn space(doc: &Document, cs: Option<&Object>, budget: &mut ImageBudget) -> Result<Space, String> {
let Some(cs) = cs else { return Err("the image has no colour space".into()) };
let cs = doc.resolve(cs);
let unsupported = |n: &[u8]| Err(format!("{} images can't be exported yet", String::from_utf8_lossy(n)));
@@ -470,7 +478,7 @@ fn space(doc: &Document, cs: Option<&Object>) -> Result<Space, String> {
if nested {
return Err("nested indexed colour spaces aren't valid".into());
}
let base = space(doc, a.get(1))?;
let base = space(doc, a.get(1), budget)?;
let hival = match a.get(2).map(|value| doc.resolve(value)).as_deref() {
Some(Object::Int(value)) if (0..=255).contains(value) => *value as u8,
_ => return Err("an indexed colour space with an invalid hival".into()),
@@ -478,10 +486,14 @@ fn space(doc: &Document, cs: Option<&Object>) -> Result<Space, String> {
let table = match a.get(3).map(|t| doc.resolve(t)).as_deref() {
Some(Object::String(s)) => s.bytes.clone(),
// Through its parameters given by reference too, as the image's samples.
Some(Object::Stream(t)) => doc
.with_direct_parms(t)
.decoded_within(MAX_LOOKUP)
.map_err(|_| "an indexed colour space with an oversized or damaged lookup table".to_string())?,
Some(Object::Stream(t)) => {
let refused = budget.refused();
match budget.decode_filters(doc, t, MAX_LOOKUP) {
Ok(table) => table,
Err(_) if budget.refused() > refused => return Err(FilterError::Spent.to_string()),
Err(_) => return Err("an indexed colour space with an oversized or damaged lookup table".into()),
}
}
_ => return Err("an indexed colour space without a lookup table".into()),
};
Ok(Space::Indexed(Box::new(base), hival, table))
crates/create/src/tests.rs
+92
View File
@@ -2409,6 +2409,98 @@ fn flate_images_past_the_exports_allowance_are_left_out_with_the_reason() {
assert_eq!(budget.refused(), 1);
}
/// An indexed image's lookup table, given as a stream, is read within the export's allowance as
/// the image's samples are: each image reads it again, and its filters can do work that makes
/// nothing (Flate over empty blocks). Past the allowance the image is left out with the reason.
#[test]
fn a_lookup_table_is_read_within_the_exports_allowance() {
use pdfcraft_cos::{Dict, Document, ImageBudget, Object, PdfString, Stream};
let table = [255, 0, 0, 0, 0, 255];
let lookup = Stream::flate(Dict::new(), &table);
let exported = |as_stream: bool, work: u64| {
let mut doc = Document::new_empty();
let table = if as_stream { Object::Ref(doc.add(Object::Stream(lookup.clone()))) } else { Object::String(PdfString::literal(table.to_vec())) };
let mut dict = Dict::new();
dict.set(b"Subtype".to_vec(), Object::name("Image"));
dict.set(b"Width".to_vec(), Object::Int(2));
dict.set(b"Height".to_vec(), Object::Int(1));
dict.set(b"BitsPerComponent".to_vec(), Object::Int(8));
dict.set(b"ColorSpace".to_vec(), Object::Array(vec![Object::name("Indexed"), Object::name("DeviceRGB"), Object::Int(1), table]));
let image = doc.add(Object::Stream(Stream::flate(dict, &[0, 1])));
let mut budget = ImageBudget::with_work(work);
let file = crate::extract::image_file_within(&doc, image, &mut budget);
(file, work - budget.work_left(), budget.refused())
};
let (direct, image_alone, _) = exported(false, u64::MAX);
let (streamed, both, _) = exported(true, u64::MAX);
assert_eq!(streamed.unwrap(), direct.unwrap(), "the same picture");
let read = (lookup.raw.len() + table.len()) as u64 + pdfcraft_filters::FLATE_READ_WORK * lookup.raw.len() as u64;
assert_eq!(both, image_alone + read, "the table is paid for as it is read");
let (refused, _, count) = exported(true, image_alone);
assert!(refused.unwrap_err().contains("more image decoding than"));
assert_eq!(count, 1);
}
/// A JPEG's colour space is read once for an export, its lookup table paid for once: whether it is
/// CMYK (a negative as is) and what its samples are both come from that reading. An indexed JPEG
/// isn't valid, and is written as it is, but its soft mask makes it go on past the first. With the
/// mask, a table the allowance can't pay for leaves the image out, with the reason.
#[test]
fn a_jpegs_lookup_table_is_paid_for_once() {
use pdfcraft_cos::{Dict, Document, ImageBudget, Object, PdfString, Stream};
let table = [255, 0, 0, 0, 0, 255];
let lookup = Stream::flate(Dict::new(), &table);
let mut jpeg = Vec::new();
image::GrayImage::from_pixel(2, 1, image::Luma([1])).write_to(&mut std::io::Cursor::new(&mut jpeg), image::ImageFormat::Jpeg).unwrap();
// `given`: None leaves the colour space out, Some(true) gives the table as a stream.
let exported = |given: Option<bool>, work: u64| {
let mut doc = Document::new_empty();
let table = given.map(|as_stream| {
if as_stream { Object::Ref(doc.add(Object::Stream(lookup.clone()))) } else { Object::String(PdfString::literal(table.to_vec())) }
});
let mut mask = Dict::new();
for (k, v) in
[("Subtype", Object::name("Image")), ("Width", Object::Int(2)), ("Height", Object::Int(1)), ("BitsPerComponent", Object::Int(8))]
{
mask.set(k.as_bytes().to_vec(), v);
}
mask.set(b"ColorSpace".to_vec(), Object::name("DeviceGray"));
let mask = doc.add(Object::Stream(Stream::flate(mask, &[255, 128])));
let mut dict = Dict::new();
for (k, v) in
[("Subtype", Object::name("Image")), ("Width", Object::Int(2)), ("Height", Object::Int(1)), ("BitsPerComponent", Object::Int(8))]
{
dict.set(k.as_bytes().to_vec(), v);
}
if let Some(table) = table {
dict.set(b"ColorSpace".to_vec(), Object::Array(vec![Object::name("Indexed"), Object::name("DeviceRGB"), Object::Int(1), table]));
}
dict.set(b"Filter".to_vec(), Object::name("DCTDecode"));
dict.set(b"SMask".to_vec(), Object::Ref(mask));
let image = doc.add(Object::Stream(Stream::from_raw(dict, jpeg.clone())));
let mut budget = ImageBudget::with_work(work);
let file = crate::extract::image_file_within(&doc, image, &mut budget);
(file, work - budget.work_left(), budget.refused())
};
let (direct, direct_cost, _) = exported(Some(false), u64::MAX);
let (streamed, streamed_cost, _) = exported(Some(true), u64::MAX);
assert_eq!(direct, Ok(("jpg", jpeg.clone())), "written as it is");
assert_eq!(streamed, direct);
let read = (lookup.raw.len() + table.len()) as u64 + pdfcraft_filters::FLATE_READ_WORK * lookup.raw.len() as u64;
assert_eq!(streamed_cost, direct_cost + read, "one reading of the table");
// Without a colour space nothing is read after the soft mask, so this is what the mask costs.
let (none, mask, _) = exported(None, u64::MAX);
assert_eq!(none.unwrap_err(), "the image has no colour space");
// Read first, the table would be refused a step short of the mask too, and the mask after it.
assert!(read >= mask, "the table costs at least what the mask does");
assert_eq!(exported(Some(true), mask - 1).2, 1, "the mask is read first, and refused alone");
let (refused, _, count) = exported(Some(true), mask + read - 1);
let refused = refused.unwrap_err();
assert!(refused.contains("more image decoding than"), "{refused}");
assert_eq!(count, 1, "refused at the table, and nothing after it asks to be paid for");
}
/// A JPEG is written as it is, at no cost, unless something means decoding it, here a mask: that
/// pays for its samples, and past the allowance the image goes, with the reason, as it would come out without
/// its transparency otherwise.
crates/edit/src/added.rs
+4 -1
View File
@@ -23,6 +23,7 @@ mod layout;
mod store;
mod types;
pub(crate) use draw::thai_glyph_resources;
#[cfg(test)]
use draw::{GLYPHS_PER_FONT, LaidLine, draw_laid_shaped};
#[cfg(test)]
@@ -31,7 +32,7 @@ use layout::Piece;
pub(crate) use layout::WRAP_CALLS;
pub use layout::{PASTE_STEP, first_undrawable, lines, paste_place, paste_rect, text_rect, text_width};
pub use store::{add_content, delete_content, isolate_content, list_added, scan_markers, update_content};
pub use types::{Added, AddedImage, AddedText, Align, CHAR_SPACING_LIMIT, Family, MarkerScan};
pub use types::{Added, AddedImage, AddedText, Align, CHAR_SPACING_LIMIT, Family, Ink, MarkerScan};
#[derive(Clone, Debug, PartialEq)]
pub enum Content {
@@ -87,3 +88,5 @@ fn align_from(code: Option<i64>) -> Align {
#[cfg(test)]
mod arabic_tests;
#[cfg(test)]
mod thai_tests;
crates/edit/src/added/draw.rs
+61 -5
View File
@@ -1,7 +1,7 @@
use super::layout::{
Piece, arabic_error, hebrew_error, is_arabic, is_hebrew, is_thai, latin_width, norm, shaped_layout, text_rect, thai_error, wrapped,
};
use super::types::{AddedText, Align, Family};
use super::types::{AddedText, Align, Family, Ink};
use super::{
Content, Dict, Document, EditError, GlyphError, GlyphOutline, HashMap, HashSet, ObjRef, Object, PdfString, ShapedCluster, ShapedScript, Stream,
align_code, literal, matrix_number, n, shaped_glyph, win_ansi,
@@ -105,8 +105,7 @@ pub(super) fn draw_laid_shaped(
fonts.set(name.clone().into_bytes(), Object::Ref(font));
names.push(name);
}
let [cr, cg, cb] = t.color.map(|v| v.clamp(0.0, 1.0));
out.extend(format!("BT {} {} {} rg\n", n(cr), n(cg), n(cb)).bytes());
out.extend(format!("BT {}\n", t.fill_op()).bytes());
if t.char_spacing != 0.0 {
out.extend(format!("{} Tc\n", n(t.char_spacing)).bytes());
}
@@ -293,8 +292,7 @@ pub(super) fn draw(doc: &mut Document, c: &Content, view: [f64; 6], taken: &Dict
}
res.set(b"Font".to_vec(), Object::Dict(fonts));
let r = text_rect(t);
let [cr, cg, cb] = t.color.map(|v| v.clamp(0.0, 1.0));
out.extend(format!("BT /{name} {} Tf {} {} {} rg\n", n(t.size), n(cr), n(cg), n(cb)).bytes());
out.extend(format!("BT /{name} {} Tf {}\n", n(t.size), t.fill_op()).bytes());
if t.char_spacing != 0.0 {
out.extend(format!("{} Tc\n", n(t.char_spacing)).bytes());
}
@@ -355,6 +353,12 @@ pub(super) fn params(c: &Content) -> Dict {
d.set(b"Font".to_vec(), Object::name(t.family.base_font(t.bold, t.italic)));
d.set(b"Size".to_vec(), Object::Real(t.size));
d.set(b"Color".to_vec(), arr(&t.color));
// The ink beside its RGB look, which a reader that knows no ink keeps showing.
match t.painted_ink() {
Some(Ink::Gray(g)) => d.set(b"Gray".to_vec(), Object::Real(g)),
Some(Ink::Cmyk(c)) => d.set(b"CMYK".to_vec(), arr(&c)),
None => {}
}
d.set(b"Align".to_vec(), Object::Int(align_code(t.align)));
// Each paragraph's own alignment, or null where it follows /Align.
let aligns = crate::paragraph_align::tidy(t.aligns.clone(), t.align);
@@ -386,3 +390,55 @@ pub(super) fn params(c: &Content) -> Dict {
}
d
}
/// Existing Thai text uses absolute shaped glyph positions. Reuse Add Text's bounded outline
/// writer instead of trying to embed the matching craft-fonts variable program as a static face.
pub(crate) fn thai_glyph_resources(
doc: &mut Document,
fonts: &mut Dict,
shaped: &pdfcraft_fonts::source_shaping::ShapedRun,
) -> Result<Vec<(String, Vec<u8>)>, EditError> {
if shaped.glyphs.len() > 65_534 {
return Err(EditError::Invalid("shaped row exceeds its geometry budget".into()));
}
let mut outlines = HashMap::new();
let mut clusters = Vec::new();
let mut index = HashMap::new();
let mut indices = Vec::new();
for glyph in &shaped.glyphs {
let id = glyph.id.to_u32();
let key = (ShapedScript::Thai, id);
if let std::collections::hash_map::Entry::Vacant(entry) = outlines.entry(key) {
entry.insert(shaped_glyph(ShapedScript::Thai, id).map_err(|e| thai_error(e, &glyph.text))?);
}
let at = *index.entry((id, glyph.text.as_str())).or_insert_with(|| {
let at = clusters.len();
let width = outlines.get(&key).map_or(0.0, |g| g.width);
clusters.push(ShapedCluster { script: ShapedScript::Thai, glyphs: vec![(id, [0.0, 0.0])], advance: width, text: glyph.text.clone() });
at
});
indices.push(at);
}
// Validate all outlines before adding resources. Type 3 character codes are single bytes;
// shard long rows and return the appropriate resource/code for every positioned glyph.
let mut names = Vec::new();
for chunk in clusters.chunks(GLYPHS_PER_FONT) {
let refs: Vec<_> = chunk.iter().collect();
let font = type3_shaped(doc, &refs, &outlines);
doc.update_dict(font, |f| f.set(b"Name".to_vec(), Object::name("PCThai")))?;
let mut name = format!("PCThai{}", font.num);
while fonts.contains(name.as_bytes()) {
name.push('_');
}
fonts.set(name.as_bytes().to_vec(), Object::Ref(font));
names.push(name);
}
indices
.into_iter()
.map(|at| {
let name = names.get(at / GLYPHS_PER_FONT).ok_or_else(|| EditError::Invalid("Thai font resource is missing".into()))?;
let code = u8::try_from(at % GLYPHS_PER_FONT + 1).map_err(|_| EditError::Invalid("Thai glyph limit".into()))?;
Ok((name.clone(), vec![code]))
})
.collect()
}
crates/edit/src/added/store.rs
+39 -3
View File
@@ -1,8 +1,33 @@
use super::draw::{draw, params};
use super::layout::{first_undrawable, is_thai, nums, shaped_arabic, thai_error};
use super::types::{Added, AddedImage, AddedText, CHAR_SPACING_LIMIT, Family, MarkerScan, TAG};
use super::types::{Added, AddedImage, AddedText, CHAR_SPACING_LIMIT, Family, Ink, MarkerScan, TAG};
use super::{Content, Dict, Document, EditError, GlyphError, MarkKind, ObjRef, Object, Stream, align_from, check, contents, page_list, place_tagged};
/// A text item's gray or CMYK ink. Items written before inks, and an ink out of range or not
/// made only of numbers (a `/Gray` name, a `/CMYK` entry that isn't a number), have none: the
/// text paints in its RGB `/Color`.
fn ink(doc: &Document, d: &Dict) -> Option<Ink> {
let ink = if let Some(g) = d.get(b"Gray") {
Ink::Gray(read_value(doc, g, Object::as_f64)?)
} else {
// Four entries or none: a longer array is never copied or read through.
read_value(doc, d.get(b"CMYK")?, |cmyk| {
let [c, m, y, k] = cmyk.as_array()?.as_slice() else { return None };
Some(Ink::Cmyk([c.as_f64()?, m.as_f64()?, y.as_f64()?, k.as_f64()?]))
})?
};
Some(ink).filter(|i| i.valid())
}
/// `f` of `o`, a reference followed. A direct value is read where it is, never copied, so a
/// large one a file puts there costs nothing to look at.
fn read_value<T>(doc: &Document, o: &Object, f: impl FnOnce(&Object) -> T) -> T {
match o {
Object::Ref(_) => f(&doc.resolve(o)),
direct => f(direct),
}
}
fn parse(doc: &Document, d: &Dict) -> Option<Content> {
let r = nums(doc, d.get(b"Rect"));
let rect: [f64; 4] = r.try_into().ok()?;
@@ -21,7 +46,7 @@ fn parse(doc: &Document, d: &Dict) -> Option<Content> {
.map(|a| a.iter().take(crate::paragraph_align::count(&text)).map(|v| v.as_int().map(|c| align_from(Some(c)))).collect())
})
.unwrap_or_default();
Some(Content::Text(AddedText {
let mut t = AddedText {
rect,
text,
family,
@@ -29,11 +54,19 @@ fn parse(doc: &Document, d: &Dict) -> Option<Content> {
italic,
size: d.get(b"Size").and_then(Object::as_f64).unwrap_or(12.0),
color: if c.len() == 3 { [c[0], c[1], c[2]] } else { [0.0; 3] },
ink: ink(doc, d),
align,
aligns: crate::paragraph_align::tidy(aligns, align),
// Items written before character spacing have none.
char_spacing: d.get(b"CharSpacing").and_then(Object::as_f64).filter(|v| v.is_finite()).unwrap_or(0.0),
}))
};
// An ink whose look isn't the stored colour (a file edited by hand) is none.
t.drop_stale_ink();
// One that stands is the ink as painted, finer digits dropped, and its look.
if let Some(i) = t.ink {
t.set_ink(Some(i));
}
Some(Content::Text(t))
}
b"Image" => {
let crop = nums(doc, d.get(b"Crop"));
@@ -138,6 +171,9 @@ fn validate(c: &Content) -> Result<(), EditError> {
"the character spacing must be between -{CHAR_SPACING_LIMIT} and {CHAR_SPACING_LIMIT} points"
)));
}
if t.ink.is_some_and(|i| !i.valid()) {
return Err(EditError::Invalid("a gray level and each CMYK value must be from 0 to 1".into()));
}
if (r[2] - r[0]).abs() < 1.0 {
return Err(EditError::Invalid("the text box is too narrow".into()));
}
crates/edit/src/added/thai_tests.rs
+59
View File
@@ -0,0 +1,59 @@
//! Existing Thai text drawn from the bundled Thai outlines (`thai_glyph_resources`): one Type 3
//! code per distinct glyph and cluster text, in fonts of `GLYPHS_PER_FONT` codes each. The
//! outline checks need the craft-fonts Thai face; the budget check runs without it.
use pdfcraft_cos::{Dict, Document};
use pdfcraft_fonts::source_shaping::{ShapedGlyph, ShapedRun};
use super::*;
/// `count` copies of `glyph`, each with cluster text of its own (so each needs a code of its own).
fn distinct(glyph: &ShapedGlyph, count: usize) -> ShapedRun {
let glyphs = (0..count).map(|i| ShapedGlyph { position: [i as f64, 0.0], text: format!("{i}"), ..glyph.clone() }).collect();
ShapedRun { glyphs, advance: count as f64 }
}
#[test]
fn thai_glyph_resources_refuse_a_row_past_the_glyph_budget() {
let glyph = ShapedGlyph { id: Default::default(), position: [0.0, 0.0], text: String::new() };
let over = ShapedRun { glyphs: vec![glyph; 65_535], advance: 0.0 };
let (mut doc, mut fonts) = (Document::new_empty(), Dict::new());
assert!(thai_glyph_resources(&mut doc, &mut fonts, &over).is_err());
assert!(fonts.is_empty(), "a refused row adds no font");
}
/// Codes fill a font up to `GLYPHS_PER_FONT`, the next distinct cluster starts another font at
/// code 1, and the same glyph with the same text again reuses its code. One glyph drawn for two
/// cluster texts takes two codes, since each code's ToUnicode gives its own text.
#[test]
fn thai_glyph_resources_shard_distinct_clusters_at_the_font_boundary() {
let Some(face) = pdfcraft_fonts::document_thai_font() else {
eprintln!(
"skipping thai_glyph_resources_shard_distinct_clusters_at_the_font_boundary: built without craft-fonts (set CRAFT_FONTS_DIR to run it)"
);
return;
};
let ko = pdfcraft_fonts::source_shaping::shape(face.bytes, "ก").unwrap().glyphs.remove(0);
for (clusters, fonts_used) in [(GLYPHS_PER_FONT - 1, 1), (GLYPHS_PER_FONT, 1), (GLYPHS_PER_FONT + 1, 2)] {
let mut shaped = distinct(&ko, clusters);
shaped.glyphs.push(shaped.glyphs[0].clone());
let (mut doc, mut fonts) = (Document::new_empty(), Dict::new());
let placed = thai_glyph_resources(&mut doc, &mut fonts, &shaped).unwrap();
assert_eq!((fonts.len(), placed.len()), (fonts_used, clusters + 1), "{clusters} clusters");
assert_eq!(placed.last(), placed.first(), "a repeated glyph and text reuses its code");
let codes: std::collections::HashSet<_> = placed.iter().collect();
assert_eq!(codes.len(), clusters, "every distinct cluster has a code of its own");
assert_eq!(placed[GLYPHS_PER_FONT - 2].1, [(GLYPHS_PER_FONT - 1) as u8]);
if clusters > GLYPHS_PER_FONT {
assert_eq!(placed[GLYPHS_PER_FONT - 1].1, [GLYPHS_PER_FONT as u8]);
assert_eq!(placed[GLYPHS_PER_FONT].1, [1], "the next font starts at code 1");
assert_ne!(placed[GLYPHS_PER_FONT].0, placed[0].0);
}
}
// At the budget: one cluster repeated 65 534 times needs one code.
let mut row = distinct(&ko, 1);
row.glyphs = vec![row.glyphs[0].clone(); 65_534];
let (mut doc, mut fonts) = (Document::new_empty(), Dict::new());
let placed = thai_glyph_resources(&mut doc, &mut fonts, &row).unwrap();
assert_eq!((fonts.len(), placed.len()), (1, 65_534));
}
crates/edit/src/added/types.rs
+101 -1
View File
@@ -1,5 +1,5 @@
use super::layout::norm;
use super::{Content, MarkKind, ObjRef, Std14};
use super::{Content, MarkKind, ObjRef, Std14, n};
pub(super) const TAG: &str = "Added";
@@ -90,6 +90,60 @@ pub enum Align {
Justify,
}
/// A device colour added text is painted in instead of RGB, for print files: a text's black in
/// CMYK stays 100 K rather than a DeviceRGB black the printer separates into rich black.
#[derive(Clone, Copy, Debug, PartialEq)]
pub enum Ink {
/// A DeviceGray level, from 0 (black) to 1 (white): painted with `g`.
Gray(f64),
/// DeviceCMYK `[c, m, y, k]`, each from 0 to 1 (`[0, 0, 0, 1]` is 100 K): painted with `k`.
Cmyk([f64; 4]),
}
impl Ink {
/// Whether every component is a number from 0 to 1.
pub fn valid(self) -> bool {
self.components().iter().all(|v| v.is_finite() && (0.0..=1.0).contains(v))
}
/// The ink as its operator paints it: each component to the thousandth that `g` and `k` are
/// written with (as RGB's `rg`), so the item, what it reads back as and the page agree.
pub fn painted(self) -> Ink {
let round = |v: f64| (v * 1000.0).round() / 1000.0;
match self {
Ink::Gray(g) => Ink::Gray(round(g)),
Ink::Cmyk(c) => Ink::Cmyk(c.map(round)),
}
}
fn components(&self) -> &[f64] {
match self {
Ink::Gray(g) => std::slice::from_ref(g),
Ink::Cmyk(c) => c,
}
}
/// How the ink looks on screen: gray is the same level in each of red, green and blue
/// (ISO 32000-2 §10.4.2.2), CMYK takes each colour plus black from 1 (§10.4.2.5).
pub fn rgb(self) -> [f64; 3] {
match self {
Ink::Gray(g) => [g.clamp(0.0, 1.0); 3],
Ink::Cmyk([c, m, y, k]) => [c, m, y].map(|v| 1.0 - (v + k).clamp(0.0, 1.0)),
}
}
/// The operator that makes this ink the fill colour, without a line end.
pub(super) fn fill_op(self) -> String {
match self {
Ink::Gray(g) => format!("{} g", n(g.clamp(0.0, 1.0))),
Ink::Cmyk(c) => {
let [c, m, y, k] = c.map(|v| v.clamp(0.0, 1.0));
format!("{} {} {} {} k", n(c), n(m), n(y), n(k))
}
}
}
}
#[derive(Clone, Debug, PartialEq)]
pub struct AddedText {
/// The text box in display space; its width wraps the text and its top is the first line's
@@ -100,7 +154,14 @@ pub struct AddedText {
pub bold: bool,
pub italic: bool,
pub size: f64,
/// The RGB colour: what the text is painted in, unless it has an [`AddedText::ink`] this is
/// the look of.
pub color: [f64; 3],
/// A gray or CMYK ink the text is painted in, with `color` its look ([`Ink::rgb`]). Set it
/// with [`AddedText::set_ink`] and the colour with [`AddedText::set_color`], which drops an
/// ink the colour no longer shows; one left behind by setting `color` directly paints nothing
/// ([`AddedText::painted_ink`]) until [`AddedText::drop_stale_ink`] forgets it.
pub ink: Option<Ink>,
pub align: Align,
/// Extra space after every character, in points: the PDF's character spacing (`Tc`).
pub char_spacing: f64,
@@ -114,6 +175,44 @@ impl AddedText {
pub fn paragraph_align(&self, i: usize) -> Align {
crate::paragraph_align::align_of(&self.aligns, i, self.align)
}
/// Paint the text in `ink` (its look becomes [`AddedText::color`]), or in RGB with `None`. An
/// ink in range is kept as painted ([`Ink::painted`]); one out of range stays as given, for
/// saving to refuse.
pub fn set_ink(&mut self, ink: Option<Ink>) {
let ink = ink.map(|i| if i.valid() { i.painted() } else { i });
if let Some(i) = ink {
self.color = i.rgb();
}
self.ink = ink;
}
/// Paint the text in RGB `rgb`. Its ink stays only when `rgb` is the ink's own look; once
/// gone it stays gone, so picking that look again later paints in RGB.
pub fn set_color(&mut self, rgb: [f64; 3]) {
self.color = rgb;
self.drop_stale_ink();
}
/// Forget an ink `color` no longer shows, so two styles that paint the same compare equal.
pub fn drop_stale_ink(&mut self) {
self.ink = self.painted_ink();
}
/// The ink the text is painted in, or `None` for RGB: the one it has while `color` is still
/// its look.
pub fn painted_ink(&self) -> Option<Ink> {
// Within rounding: a look read back from a file, or from a colour picker, is the same.
self.ink.filter(|i| i.rgb().iter().zip(self.color).all(|(a, b)| (a - b).abs() < 1e-6))
}
/// The operator that sets the text's fill colour, without a line end.
pub(super) fn fill_op(&self) -> String {
match self.painted_ink() {
Some(ink) => ink.fill_op(),
None => crate::marks::rgb(self.color),
}
}
}
impl Default for AddedText {
@@ -126,6 +225,7 @@ impl Default for AddedText {
italic: false,
size: 12.0,
color: [0.0; 3],
ink: None,
align: Align::Left,
aligns: Vec::new(),
char_spacing: 0.0,
crates/edit/src/backdrop.rs
+4 -4
View File
@@ -208,7 +208,7 @@ pub(crate) fn restore(doc: &mut Document, page: &pdfcraft_model::Page) -> Result
let same = |w: &ObjRef| match &*doc.get(*w) {
Object::Stream(s) => {
[TAG, &b"PCMark"[..], b"Subtype"].iter().all(|k| s.dict.get(k).is_none())
&& DecodeBudget::new(SCAN_BYTES, 0).decode_strict(s).ok().as_deref() == Some(bytes.as_slice())
&& DecodeBudget::new(SCAN_BYTES, 0).decode_strict(doc, s).ok().as_deref() == Some(bytes.as_slice())
}
_ => false,
};
@@ -248,7 +248,7 @@ fn restored_copy(doc: &Document, page: &pdfcraft_model::Page, copy: ObjRef) -> O
return None;
}
let hash = note.get(b"Before")?.as_string()?;
let bytes = DecodeBudget::new(SCAN_BYTES, 0).decode_strict(s).ok()?;
let bytes = DecodeBudget::new(SCAN_BYTES, 0).decode_strict(doc, s).ok()?;
let parsed = pdfcraft_content::parse(head(&bytes));
// The `n` standing in for the fill, after exactly the operators that came before it.
let fill = parsed.ops.get(index)?;
@@ -384,11 +384,11 @@ fn scan(doc: &Document, p: &pdfcraft_model::Page, swap: Option<(ObjRef, &[u8])>)
if b.len() > SCAN_BYTES.saturating_sub(total) {
return None;
}
budget.decode_strict(&Stream::from_raw(Dict::new(), b.to_vec())).ok()?
budget.decode_strict(doc, &Stream::from_raw(Dict::new(), b.to_vec())).ok()?
}
// Tolerant decoding can return an empty or partial stream after corruption.
// That cannot prove which paint preceded the candidate backdrop.
_ => budget.decode_strict(s).ok()?,
_ => budget.decode_strict(doc, s).ok()?,
};
if bytes.len() > SCAN_BYTES.saturating_sub(total).saturating_sub(1) {
return None;
crates/edit/src/background.rs
+2 -2
View File
@@ -1,7 +1,7 @@
//! A page's background colour: the colour the page is painted with before anything else, as
//! Edit a PDF ▸ Background and many producers do it, for exports that keep a page colour (#2992).
use pdfcraft_content::{Matrix, Op, Pieces};
use pdfcraft_content::{Matrix, Op};
use pdfcraft_cos::{Dict, Document};
use crate::{EditError, Rgb};
@@ -51,7 +51,7 @@ pub fn page_background(doc: &Document, page: usize) -> Result<Option<Rgb>, EditE
let resources = p.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default();
let gstates = resources.get(b"ExtGState").map(|g| doc.resolve(g)).and_then(|g| g.as_dict().cloned()).unwrap_or_default();
let streams = crate::text::content_streams(doc, &p.dict)?;
let pieces = Pieces::join(&streams.iter().map(|(_, d)| d.as_slice()).collect::<Vec<_>>());
let pieces = crate::text::joined(&streams);
let ops = pieces.parse().ops;
let mut st = State { ctm: Matrix::IDENTITY, space: Space::Gray, fill: Some([0.0; 3]), alpha: 1.0, plain: true, clip: Some(Vec::new()) };
crates/edit/src/images.rs
+84 -48
View File
@@ -1,15 +1,15 @@
//! Edit a PDF ▸ edit existing images and grouped artwork (Image/Form XObjects painted by `Do`
//! in its own content streams), and changing one: move/resize/rotate/flip (the `Do` is wrapped in
//! `q … cm … Q` with the extra transform), replace (it draws another XObject in the same place),
//! or delete. Only the stream that draws it is rewritten, as a new object.
//! delete, or ungroup grouped artwork (its content drawn in place, see `ungroup`). Only the stream that draws it is rewritten, as a new object.
use kurbo::{BezPath, Point, Shape};
use pdfcraft_content::{Matrix, Op, Pieces, parse, serialize_ops};
use pdfcraft_content::{Matrix, Op, parse, serialize_ops};
use pdfcraft_cos::{Dict, Document, ObjRef, Object};
use std::sync::Arc;
use crate::EditError;
use crate::text::content_streams;
use crate::text::{ContentStream, content_streams, joined};
/// The identity of a soft mask (`/SMask`) in the PDF graphics state.
#[derive(Clone, Debug)]
@@ -47,9 +47,9 @@ pub struct PageImage {
pub width: u32,
pub height: u32,
/// The content stream its `Do` (its first token) is in, and the `Do`'s index among the
/// operators of the page's streams joined ([`Pieces`]).
/// operators of the page's streams joined ([`pdfcraft_content::Pieces`]).
pub stream: usize,
op: usize,
pub(crate) op: usize,
/// CTM before `Do`, excluding a Form's own Matrix.
draw_matrix: [f64; 6],
clips: Vec<ClipPath>,
@@ -103,11 +103,6 @@ fn too_many_images() -> EditError {
EditError::Invalid(format!("the page draws more than {MAX_PAGE_IMAGES} images"))
}
/// The page's streams joined into the one stream they are: an operator may span two of them.
fn joined(streams: &[(Object, Vec<u8>)]) -> Pieces {
Pieces::join(&streams.iter().map(|(_, data)| data.as_slice()).collect::<Vec<_>>())
}
fn page_of(doc: &Document, page: usize) -> Result<pdfcraft_model::Page, EditError> {
pdfcraft_model::pages(doc).into_iter().nth(page).ok_or(EditError::NoSuchPage(page))
}
@@ -470,6 +465,9 @@ pub enum ImageChange {
Arrange(ArrangeOrder),
/// Remove any private crop on the image.
RemoveCrop,
/// Grouped Form artwork only: draw the form's content in place of its `Do`, so its text and
/// pictures become the page's own and can be edited one by one (#2964).
Ungroup,
}
/// The user-space transform that maps box `from` onto box `to` (move and resize).
@@ -532,12 +530,17 @@ fn exclusive_clip(ops: &[Op], op: usize) -> Option<(usize, Matrix)> {
/// The `q cm … Q` whose only content after its `cm` is `start..=end` (a `Do`, or a balanced
/// `q … Q` scope), with the index and value of that `cm`. Another transform of the same
/// placement is folded into it, so repeated moves and resizes don't nest one more `q` each.
fn enclosing_transform(ops: &[Op], start: usize, end: usize) -> Option<(usize, Matrix)> {
/// None when its `Q` is at or past `hidden` ([`crate::text::first_op_after_damage`]): a stream
/// between that wasn't read may draw in that `cm` too.
fn enclosing_transform(ops: &[Op], start: usize, end: usize, hidden: usize) -> Option<(usize, Matrix)> {
let at = start.checked_sub(1)?;
let cm = ops.get(at)?;
if !ops.get(at.checked_sub(1)?)?.is("q") || !ops.get(end.checked_add(1)?)?.is("Q") || !cm.is("cm") || cm.operands.len() != 6 {
return None;
}
if end.checked_add(1)? >= hidden {
return None;
}
let m = Matrix(cm.nums::<6>()?);
m.0.iter().all(|v| v.is_finite()).then_some((at, m))
}
@@ -1057,7 +1060,9 @@ fn determine_arrange_spans(
has_other_paint: bool,
marked_unit: bool,
) -> Result<(std::ops::Range<usize>, usize, usize, usize), EditError> {
let scope = image_scopes(ops, target, "arranged")
// Nothing is hidden for Arrange: it takes out the scope it finds, so a scope that closes past a
// stream not read whole changes a stream past it, which `rewritten_contents` refuses.
let scope = image_scopes(ops, target, "arranged", ops.len())
.ok()
.and_then(|s| s.isolated)
.filter(|&(start, end, _)| start >= target_unit.start && end < target_unit.end);
@@ -1191,13 +1196,7 @@ fn build_relocated_unit_ops(
}
/// Change the stacking order of `img`, drawn on page `p` by its content streams `all`.
fn arrange_image(
doc: &mut Document,
p: &pdfcraft_model::Page,
all: &[(Object, Vec<u8>)],
img: &PageImage,
order: ArrangeOrder,
) -> Result<(), EditError> {
fn arrange_image(doc: &mut Document, p: &pdfcraft_model::Page, all: &[ContentStream], img: &PageImage, order: ArrangeOrder) -> Result<(), EditError> {
let pieces = joined(all);
let ops = pieces.parse().ops;
let target = img.op;
@@ -1264,7 +1263,7 @@ fn arrange_image(
ops.get(dest_unit.start).map(|op| pieces.piece_of(op.span.start))
};
let owned_destination = destination_piece
.filter(|piece| all.get(*piece).is_some_and(|(obj, _)| doc.resolve(obj).as_dict().is_some_and(|d| d.get(b"PCAdded").is_some())));
.filter(|piece| all.get(*piece).is_some_and(|s| doc.resolve(&s.entry).as_dict().is_some_and(|d| d.get(b"PCAdded").is_some())));
if let Some(piece) = owned_destination {
let whole_unit = ops.iter().enumerate().all(|(index, op)| {
let (first, last) = pieces.pieces_of(op);
@@ -1439,13 +1438,14 @@ fn arrange_image(
};
if let Some(piece) = owned_destination {
let insertion = piece.saturating_add(usize::from(insert_after));
crate::text::drawn_before_damage(all, insertion)?;
let changed = pieces.splice([(target_span, removal_replacement)]);
let contents = crate::text::rewritten_contents(doc, all, changed);
let contents = crate::text::rewritten_contents(doc, all, changed)?;
let mut list = match contents {
Object::Array(list) => list,
one => vec![one],
};
let insertion = piece.saturating_add(usize::from(insert_after));
if insertion > list.len() {
return Err(EditError::Invalid("the destination content stream is missing".into()));
}
@@ -1467,7 +1467,7 @@ fn arrange_image(
}
let changed = pieces.splice(edits);
let contents = crate::text::rewritten_contents(doc, all, changed);
let contents = crate::text::rewritten_contents(doc, all, changed)?;
doc.update_dict(p.obj, |d| d.set(b"Contents".to_vec(), contents))?;
Ok(())
}
@@ -1501,8 +1501,10 @@ impl ImageScopes {
}
}
/// [`ImageScopes`] of the `Do` at `target`. An image inside a text object cannot be `done`.
fn image_scopes(ops: &[Op], target: usize, done: &str) -> Result<ImageScopes, EditError> {
/// [`ImageScopes`] of the `Do` at `target`. An image inside a text object cannot be `done`. A scope
/// still open at op `hidden` ([`crate::text::first_op_after_damage`]) holds a stream that wasn't
/// read, which may paint there: it counts as painting something else.
fn image_scopes(ops: &[Op], target: usize, done: &str, hidden: usize) -> Result<ImageScopes, EditError> {
struct Scope {
start: usize,
matrix: Matrix,
@@ -1521,6 +1523,11 @@ fn image_scopes(ops: &[Op], target: usize, done: &str) -> Result<ImageScopes, Ed
let mut isolated = None;
let mut outermost = None;
for (index, op) in ops.iter().enumerate() {
if index == hidden {
for scope in &mut stack {
scope.other_paint = true;
}
}
match op.op.as_slice() {
b"q" => {
if stack.len() >= 256 {
@@ -1606,20 +1613,26 @@ fn app_crop_ranges(ops: &[Op], start: usize, target: usize) -> Vec<std::ops::Ran
}
/// Remove only crops written by PdfCraft, preserving the document author's clipping paths.
fn remove_image_crop(doc: &mut Document, p: &pdfcraft_model::Page, all: &[(Object, Vec<u8>)], img: &PageImage) -> Result<(), EditError> {
fn remove_image_crop(doc: &mut Document, p: &pdfcraft_model::Page, all: &[ContentStream], img: &PageImage) -> Result<(), EditError> {
let pieces = joined(all);
let ops = pieces.parse().ops;
let target = img.op;
let scope = image_scopes(&ops, target, "cropped")?;
let Some((start, _, _)) = scope.transform_scope(target) else {
return Err(EditError::Invalid("the image has no PdfCraft crop to remove".into()));
let crops = |hidden| -> Result<Vec<std::ops::Range<usize>>, EditError> {
let scope = image_scopes(&ops, target, "cropped", hidden)?;
Ok(scope.transform_scope(target).map(|(start, _, _)| app_crop_ranges(&ops, start, target)).unwrap_or_default())
};
let crops = app_crop_ranges(&ops, start, target);
if crops.is_empty() {
return Err(EditError::Invalid("the image has no PdfCraft crop to remove".into()));
let crops_there = crops(crate::text::first_op_after_damage(all, &pieces, &ops))?;
if crops_there.is_empty() {
// A crop in a scope that closes past a stream not read whole may clip what that stream draws.
return Err(EditError::Invalid(if crops(ops.len())?.is_empty() {
"the image has no PdfCraft crop to remove".into()
} else {
"the image's crop may also clip what a content stream that is damaged or can't be decoded draws, so it isn't removed".into()
}));
}
let crops = crops_there;
let changed = crate::text::splice(&pieces, &ops, |index| (Vec::new(), !crops.iter().any(|r| r.contains(&index))));
let contents = crate::text::rewritten_contents(doc, all, changed);
let contents = crate::text::rewritten_contents(doc, all, changed)?;
doc.update_dict(p.obj, |d| d.set(b"Contents".to_vec(), contents))?;
Ok(())
}
@@ -1650,6 +1663,26 @@ pub fn change_image(doc: &mut Document, page: usize, index: usize, change: &Imag
ImageChange::Crop(keep) => {
return crop_image_impl(doc, &p, &all, &img, *keep);
}
ImageChange::Ungroup => {
if !img.is_form {
return Err(EditError::Invalid("only grouped Form artwork can be ungrouped".into()));
}
let res = p.dict.get(b"Resources").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()).unwrap_or_default();
let space = crate::ungroup::page_blend_space(doc, &p.dict);
// One allowance for the form and everything it draws.
let mut budget = pdfcraft_cos::DecodeBudget::new(crate::ungroup::MAX_FORM_BYTES, 0);
let crate::ungroup::Ungrouped { ops: inline, resources: mut merged, patterns, .. } =
crate::ungroup::ungroup_ops(doc, &res, &ops, img.op, space.as_ref(), Matrix(img.matrix), &mut budget).map_err(|r| r.error)?;
if !patterns.is_empty() {
let mut table = merged.get(b"Pattern").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()).unwrap_or_default();
for (name, copy) in patterns {
table.set(name, Object::Ref(doc.add(copy)));
}
merged.set(b"Pattern".to_vec(), Object::Dict(table));
}
resources = Some(merged);
inline
}
ImageChange::Replace(r) => {
if img.is_form {
return Err(EditError::Invalid(
@@ -1666,7 +1699,7 @@ pub fn change_image(doc: &mut Document, page: usize, index: usize, change: &Imag
}
};
let changed = crate::text::splice(&pieces, &ops, |index| if index == img.op { (replacement.clone(), false) } else { (Vec::new(), true) });
let contents = crate::text::rewritten_contents(doc, &all, changed);
let contents = crate::text::rewritten_contents(doc, &all, changed)?;
doc.update_dict(p.obj, |d| {
d.set(b"Contents".to_vec(), contents);
if let Some(res) = resources {
@@ -1687,7 +1720,7 @@ pub(crate) fn selectable_images(
Ok(page_images(doc, page)?
.into_iter()
.enumerate()
.filter(|(_, image)| !all.get(image.stream).and_then(|(o, _)| o.as_ref()).is_some_and(|r| excluded.contains(&r)))
.filter(|(_, image)| !all.get(image.stream).and_then(|s| s.entry.as_ref()).is_some_and(|r| excluded.contains(&r)))
.collect())
}
@@ -1700,7 +1733,7 @@ pub(crate) fn painted_over(doc: &Document, page: usize, rect: [f64; 4], from: us
fn transform_image(
doc: &mut Document,
page: &pdfcraft_model::Page,
streams: &[(Object, Vec<u8>)],
streams: &[ContentStream],
image: &PageImage,
transform: [f64; 6],
) -> Result<(), EditError> {
@@ -1715,8 +1748,9 @@ fn transform_image(
}
// A private path crop moves with its image regardless of intervening state operators.
// Shared crops retain the existing fixed-clip behavior, preserving neighbouring paint.
let hidden = crate::text::first_op_after_damage(streams, &pieces, &ops);
let (start, end, placement) =
image_scopes(&ops, target, "transformed")?.transform_scope(target).unwrap_or((target, target, Matrix(image.draw_matrix)));
image_scopes(&ops, target, "transformed", hidden)?.transform_scope(target).unwrap_or((target, target, Matrix(image.draw_matrix)));
let inverse = placement.invert().ok_or_else(|| EditError::Invalid("the image has no area".into()))?;
let local = placement.then(&Matrix(transform)).then(&inverse);
if !local.0.iter().all(|v| v.is_finite() && v.abs() <= 1e12) {
@@ -1725,7 +1759,7 @@ fn transform_image(
// Already alone in a `q cm … Q` (its own placement, or the wrapper an earlier edit added):
// change that `cm` in place. The CTM inside is `local · cm · outer` either way.
if let Some((at, cm)) = enclosing_transform(&ops, start, end) {
if let Some((at, cm)) = enclosing_transform(&ops, start, end, hidden) {
let folded = local.then(&cm);
if !folded.0.iter().all(|v| v.is_finite() && v.abs() <= 1e12) {
return Err(EditError::Invalid("the image transform is too large".into()));
@@ -1733,7 +1767,7 @@ fn transform_image(
let changed = crate::text::splice(&pieces, &ops, |index| {
if index == at { (vec![Op::new("cm", folded.0.map(Object::Real).to_vec())], false) } else { (Vec::new(), true) }
});
let contents = crate::text::rewritten_contents(doc, streams, changed);
let contents = crate::text::rewritten_contents(doc, streams, changed)?;
doc.update_dict(page.obj, |d| d.set(b"Contents".to_vec(), contents))?;
return Ok(());
}
@@ -1754,7 +1788,7 @@ fn transform_image(
(Vec::new(), true)
}
});
let contents = crate::text::rewritten_contents(doc, streams, changed);
let contents = crate::text::rewritten_contents(doc, streams, changed)?;
doc.update_dict(page.obj, |d| d.set(b"Contents".to_vec(), contents))?;
Ok(())
}
@@ -1766,7 +1800,7 @@ fn transform_image(
fn crop_image_impl(
doc: &mut Document,
page: &pdfcraft_model::Page,
streams: &[(Object, Vec<u8>)],
streams: &[ContentStream],
image: &PageImage,
keep: [f64; 4],
) -> Result<(), EditError> {
@@ -1790,8 +1824,9 @@ fn crop_image_impl(
if !ops.get(target).is_some_and(|op| op.is("Do")) {
return Err(EditError::Invalid("the image operation could not be located".into()));
}
let (start, end, placement) =
image_scopes(&ops, target, "cropped")?.transform_scope(target).unwrap_or((target, target, Matrix(image.draw_matrix)));
let (start, end, placement) = image_scopes(&ops, target, "cropped", crate::text::first_op_after_damage(streams, &pieces, &ops))?
.transform_scope(target)
.unwrap_or((target, target, Matrix(image.draw_matrix)));
let inverse = placement.invert().ok_or_else(|| EditError::Invalid("the image has no area".into()))?;
let corners = [(x0, y0), (x1, y0), (x1, y1), (x0, y1)].map(|(x, y)| inverse.apply(x, y));
if !corners.iter().all(|(x, y)| x.is_finite() && y.is_finite() && x.abs() <= 1e12 && y.abs() <= 1e12) {
@@ -1844,7 +1879,7 @@ fn crop_image_impl(
(Vec::new(), true)
}
});
let contents = crate::text::rewritten_contents(doc, streams, changed);
let contents = crate::text::rewritten_contents(doc, streams, changed)?;
doc.update_dict(page.obj, |d| d.set(b"Contents".to_vec(), contents))?;
Ok(())
}
@@ -1872,6 +1907,7 @@ pub(crate) fn translate_images(doc: &mut Document, page: usize, indexes: &[usize
}
let pieces = joined(&all);
let ops = pieces.parse().ops;
let hidden = crate::text::first_op_after_damage(&all, &pieces, &ops);
let mut inside_text = false;
for (index, op) in ops.iter().enumerate() {
if op.is("BT") {
@@ -1889,7 +1925,7 @@ pub(crate) fn translate_images(doc: &mut Document, page: usize, indexes: &[usize
let mut folds = std::collections::BTreeMap::new();
for image in images.iter().filter(|image| image.has_editor_crop()) {
let Some(&(_, _, offset)) = plans.get(&image.op) else { continue };
let Some((start, end, placement)) = image_scopes(&ops, image.op, "moved")?.transform_scope(image.op) else {
let Some((start, end, placement)) = image_scopes(&ops, image.op, "moved", hidden)?.transform_scope(image.op) else {
return Err(EditError::Invalid("the image crop cannot be moved independently".into()));
};
let inverse = placement.invert().ok_or_else(|| EditError::Invalid("a selected image has no area".into()))?;
@@ -1906,7 +1942,7 @@ pub(crate) fn translate_images(doc: &mut Document, page: usize, indexes: &[usize
};
// An enclosing cm is before the captured placement; an inside-scope cm is after it.
// Their multiplication orders differ when that cm scales or shears a translation.
let folded = enclosing_transform(&ops, start, end)
let folded = enclosing_transform(&ops, start, end, hidden)
.map(|(at, cm)| (at, local.then(&cm)))
.or_else(|| leading_transform().map(|(at, cm)| (at, cm.then(&local))));
if let Some((at, folded)) = folded {
@@ -1942,7 +1978,7 @@ pub(crate) fn translate_images(doc: &mut Document, page: usize, indexes: &[usize
// An image already alone in `q cm Do Q` moves by changing that `cm`, not by nesting another.
for index in 0..ops.len() {
let Some(&(movement, _, _)) = plans.get(&index) else { continue };
let Some((at, cm)) = enclosing_transform(&ops, index, index) else { continue };
let Some((at, cm)) = enclosing_transform(&ops, index, index, hidden) else { continue };
let folded = movement.then(&cm);
if !folded.0.iter().all(|v| v.is_finite() && v.abs() <= 1e12) {
return Err(EditError::Invalid("the image move is too large".into()));
@@ -1977,7 +2013,7 @@ pub(crate) fn translate_images(doc: &mut Document, page: usize, indexes: &[usize
out.push(Op::new("Q", vec![]));
(out, false)
});
let contents = crate::text::rewritten_contents(doc, &all, data);
let contents = crate::text::rewritten_contents(doc, &all, data)?;
doc.update_dict(p.obj, |d| d.set(b"Contents".to_vec(), contents))?;
Ok(())
}
crates/edit/src/lib.rs
+5 -2
View File
@@ -43,12 +43,15 @@ pub use backdrop::{Backdrop, backdrops_left_off, page_backdrop, page_backdrops};
pub use text::strip_subset_tag;
mod background;
pub use background::page_background;
mod rules;
pub use rules::page_rules;
pub mod objects;
pub use objects::{EditableObject, MAX_MOVE_OBJECTS, ObjectKind, ObjectTarget, editable_objects, move_objects};
mod flatten;
pub use flatten::{flatten, flatten_fill_sign};
pub mod images;
mod ungroup;
pub use images::{ArrangeOrder, ImageChange, PageImage, change_image, page_images, reading_images, rect_to_rect, turn_about_centre, turn_by_degrees};
pub mod bidi;
pub mod indic;
@@ -61,8 +64,8 @@ pub use text::{
};
pub mod added;
pub use added::{
Added, AddedImage, AddedText, Align, Content, Family, MarkerScan, add_content, delete_content, first_undrawable, isolate_content, list_added,
scan_markers, update_content,
Added, AddedImage, AddedText, Align, Content, Family, Ink, MarkerScan, add_content, delete_content, first_undrawable, isolate_content,
list_added, scan_markers, update_content,
};
mod apply;
crates/edit/src/objects.rs
+14 -9
View File
@@ -4,8 +4,9 @@
use std::collections::{BTreeMap, BTreeSet};
use pdfcraft_content::{Matrix, Op, Pieces};
use pdfcraft_cos::{Dict, Document, Object};
use pdfcraft_cos::{DecodeBudget, Dict, Document, Object};
use crate::text::ContentStream;
use crate::{Content, EditError};
pub const MAX_MOVE_OBJECTS: usize = 1_000;
@@ -152,7 +153,7 @@ fn added_geometry(doc: &Document, page: usize) -> Result<(Pieces, Vec<AddedGeome
return Ok((Pieces::default(), Vec::new()));
}
let streams = checked_streams(doc, page)?;
let pieces = Pieces::join(&streams.iter().map(|(_, bytes)| bytes.as_slice()).collect::<Vec<_>>());
let pieces = crate::text::joined(&streams);
let ops = pieces.parse().ops;
let owned: BTreeSet<_> = added.iter().map(|a| a.obj).collect();
let mut geometry = BTreeMap::new();
@@ -161,7 +162,7 @@ fn added_geometry(doc: &Document, page: usize) -> Result<(Pieces, Vec<AddedGeome
let mut stack = Vec::new();
for (index, op) in ops.iter().enumerate() {
let (stream, end) = pieces.pieces_of(op);
let object = streams.get(stream).and_then(|(r, _)| r.as_ref());
let object = streams.get(stream).and_then(|s| s.entry.as_ref());
if object.is_some_and(|r| owned.contains(&r)) {
if let std::collections::btree_map::Entry::Vacant(entry) = first.entry(stream) {
if stream != end || !op.is("q") {
@@ -275,7 +276,7 @@ fn translate_added(doc: &mut Document, page: usize, indexes: &[usize], offset: [
})
.collect::<Result<Vec<_>, EditError>>()?;
let data = pieces.splice(edits);
let contents = crate::text::rewritten_contents(doc, &streams, data);
let contents = crate::text::rewritten_contents(doc, &streams, data)?;
let refs = match &contents {
Object::Array(a) => a.clone(),
one => vec![one.clone()],
@@ -288,9 +289,9 @@ fn translate_added(doc: &mut Document, page: usize, indexes: &[usize], offset: [
Ok(())
}
/// A missing or undecodable preceding stream can hide graphics state that affects every
/// A missing, undecodable or damaged preceding stream can hide graphics state that affects every
/// later item. Refuse group editing rather than inventing an inventory from partial data.
fn checked_streams(doc: &Document, page: usize) -> Result<Vec<(Object, Vec<u8>)>, EditError> {
fn checked_streams(doc: &Document, page: usize) -> Result<Vec<ContentStream>, EditError> {
let p = pdfcraft_model::pages(doc).into_iter().nth(page).ok_or(EditError::NoSuchPage(page))?;
let contents = match p.dict.get(b"Contents") {
None => Vec::new(),
@@ -305,6 +306,7 @@ fn checked_streams(doc: &Document, page: usize) -> Result<Vec<(Object, Vec<u8>)>
},
};
let mut remaining = MAX_CONTENT_BYTES;
let mut decode = DecodeBudget::new(usize::MAX, 0);
let mut depth = 0usize;
let mut operations = 0usize;
let mut streams = Vec::with_capacity(contents.len());
@@ -313,9 +315,12 @@ fn checked_streams(doc: &Document, page: usize) -> Result<Vec<(Object, Vec<u8>)>
let Object::Stream(stream) = &*resolved else {
return Err(EditError::Invalid("a page content stream is missing".into()));
};
let bytes = stream
.decoded_within(remaining.min(64 * 1024 * 1024))
let (bytes, damaged) = decode
.decode_with_damage(doc, stream, remaining.min(64 * 1024 * 1024))
.map_err(|_| EditError::Invalid("the page content cannot be decoded within its size limit".into()))?;
if damaged {
return Err(EditError::Invalid("a page content stream is damaged partway".into()));
}
remaining =
remaining.checked_sub(bytes.len()).ok_or_else(|| EditError::Invalid("the page content is too large for group selection".into()))?;
let parsed = pdfcraft_content::parse(&bytes);
@@ -333,7 +338,7 @@ fn checked_streams(doc: &Document, page: usize) -> Result<Vec<(Object, Vec<u8>)>
return Err(EditError::Invalid("the page graphics state is nested too deeply".into()));
}
}
streams.push((object, bytes));
streams.push(ContentStream { entry: object, data: bytes, read: crate::text::Read::Whole });
}
Ok(streams)
}
crates/edit/src/rules.rs
+142
View File
@@ -0,0 +1,142 @@
//! A page's ruled lines: the horizontal and vertical strokes (and thin filled bars) its content
//! draws, so exports can tell a ruled table from text that merely lines up (#5149).
use pdfcraft_content::Matrix;
use pdfcraft_cos::Document;
use crate::EditError;
/// How deep `q` may nest (deeper saves are not tracked).
const MAX_SAVES: usize = 64;
/// At most this many rules are returned, and a path keeps at most this many segments, so a
/// hostile page stays bounded.
pub const MAX_RULES: usize = 4096;
/// A segment whose ends differ by at most this much across its run (points) is horizontal or
/// vertical; a filled rectangle at most this thick is a bar drawn as a rule.
const THIN: f64 = 2.0;
/// The page's ruled lines, each as its box `[x0, y0, x1, y1]` in default user space (`x0 <= x1`,
/// `y0 <= y1`; a stroke has zero thickness): horizontal and vertical straight segments of stroked
/// paths (rectangle edges included) and filled rectangles at most two points thick. Curves,
/// slanted lines and paths that are only clipped are left out, and so are the contents of forms.
/// At most [`MAX_RULES`] rules, in content order.
pub fn page_rules(doc: &Document, page: usize) -> Result<Vec<[f64; 4]>, EditError> {
let pages = pdfcraft_model::pages(doc);
let p = pages.get(page).ok_or(EditError::NoSuchPage(page))?;
let streams = crate::text::content_streams(doc, &p.dict)?;
let pieces = crate::text::joined(&streams);
let ops = pieces.parse().ops;
let mut out: Vec<[f64; 4]> = Vec::new();
let mut ctm = Matrix::IDENTITY;
let mut saved: Vec<Matrix> = Vec::new();
let mut lost_saves = 0usize;
// The current path in page space: its straight segments, its rectangles, the current point
// and where the current subpath started.
let mut segments: Vec<[(f64, f64); 2]> = Vec::new();
let mut rects: Vec<[(f64, f64); 4]> = Vec::new();
let mut at: Option<(f64, f64)> = None;
let mut start: Option<(f64, f64)> = None;
for op in &ops {
if out.len() >= MAX_RULES {
break;
}
match op.op.as_slice() {
b"q" => {
if saved.len() < MAX_SAVES {
saved.push(ctm);
} else {
lost_saves = lost_saves.saturating_add(1);
}
}
b"Q" => {
if lost_saves > 0 {
lost_saves -= 1;
} else if let Some(m) = saved.pop() {
ctm = m;
}
}
b"cm" => {
if let Some(m) = Matrix::from_operands(&op.operands) {
ctm = m.then(&ctm);
}
}
b"m" => {
let p = op.nums::<2>().map(|[x, y]| ctm.apply(x, y));
(at, start) = (p, p);
}
b"l" => {
let p = op.nums::<2>().map(|[x, y]| ctm.apply(x, y));
if let (Some(a), Some(b)) = (at, p)
&& segments.len() < MAX_RULES
{
segments.push([a, b]);
}
at = p;
}
b"c" => at = op.nums::<6>().map(|[.., x, y]| ctm.apply(x, y)),
b"v" | b"y" => at = op.nums::<4>().map(|[.., x, y]| ctm.apply(x, y)),
b"h" => {
if let (Some(a), Some(b)) = (at, start)
&& segments.len() < MAX_RULES
{
segments.push([a, b]);
}
at = start;
}
b"re" => {
if let Some([x, y, w, h]) = op.nums::<4>() {
let q = [ctm.apply(x, y), ctm.apply(x + w, y), ctm.apply(x + w, y + h), ctm.apply(x, y + h)];
if rects.len() < MAX_RULES {
rects.push(q);
}
(at, start) = (Some(q[0]), Some(q[0]));
}
}
b"S" | b"s" | b"B" | b"B*" | b"b" | b"b*" => {
let closes = matches!(op.op.as_slice(), b"s" | b"b" | b"b*");
if closes && let (Some(a), Some(b)) = (at, start) {
segments.push([a, b]);
}
let edges = rects.iter().flat_map(|q| [[q[0], q[1]], [q[1], q[2]], [q[2], q[3]], [q[3], q[0]]]);
out.extend(segments.iter().copied().chain(edges).filter_map(axis_aligned));
(segments, rects, at, start) = (Vec::new(), Vec::new(), None, None);
}
b"f" | b"F" | b"f*" => {
out.extend(rects.iter().filter_map(thin_bar));
(segments, rects, at, start) = (Vec::new(), Vec::new(), None, None);
}
b"n" => (segments, rects, at, start) = (Vec::new(), Vec::new(), None, None),
_ => {}
}
}
out.truncate(MAX_RULES);
Ok(out)
}
/// The box of a horizontal or vertical segment, or `None` for a slanted, tiny or non-finite one.
fn axis_aligned([a, b]: [(f64, f64); 2]) -> Option<[f64; 4]> {
let r = [a.0.min(b.0), a.1.min(b.1), a.0.max(b.0), a.1.max(b.1)];
let (w, h) = (r[2] - r[0], r[3] - r[1]);
(r.iter().all(|v| v.is_finite()) && w.min(h) <= THIN && w.max(h) > THIN).then_some(r)
}
/// The box of a filled rectangle drawn as a rule: upright, at most [`THIN`] thick and longer than that.
fn thin_bar(q: &[(f64, f64); 4]) -> Option<[f64; 4]> {
let upright = |a: (f64, f64), b: (f64, f64)| (a.0 - b.0).abs() <= 0.01 || (a.1 - b.1).abs() <= 0.01;
if !(upright(q[0], q[1]) && upright(q[1], q[2])) {
return None;
}
let xs = q.map(|p| p.0);
let ys = q.map(|p| p.1);
let r = [
xs.iter().copied().fold(f64::INFINITY, f64::min),
ys.iter().copied().fold(f64::INFINITY, f64::min),
xs.iter().copied().fold(f64::NEG_INFINITY, f64::max),
ys.iter().copied().fold(f64::NEG_INFINITY, f64::max),
];
let (w, h) = (r[2] - r[0], r[3] - r[1]);
(r.iter().all(|v| v.is_finite()) && w.min(h) <= THIN && w.max(h) > THIN).then_some(r)
}
crates/edit/src/tests.rs
+2961
View File
File diff suppressed because it is too large. Load diff
crates/edit/src/text.rs
+6 -2
View File
@@ -31,6 +31,7 @@ pub use session::TextSession;
mod blocks;
mod editor;
mod fallback;
pub mod forms;
mod interp;
mod lines;
mod rewrite;
@@ -42,8 +43,8 @@ pub use blocks::{
BlockKey, BlockStyle, RangeStyle, ReadingBlocks, TextBlock, block_overlapping, clear_block, reading_blocks, reading_blocks_checked,
replace_block, text_blocks,
};
use editor::layout_draft;
pub use editor::{TextEditLayout, TextEditorFont, register_text_font, text_edit_layout, text_editor_fonts};
use editor::{builtin_thai_choice, layout_draft};
#[cfg(test)]
pub(crate) use fallback::SOURCE_SPACE_CALLS;
use fallback::{
@@ -55,7 +56,10 @@ use fallback::{add_type3_font, is_type3_fallback};
pub(crate) use fallback::{glyph_path, type3_font_dict, unique_font_name};
use interp::{Origin, Paint, union};
use lines::page_dict;
pub(crate) use lines::{FormBudget, content_streams, content_streams_within, dict_in_place, form_call, resource_dict, rewritten_contents, splice};
pub(crate) use lines::{
ContentStream, FormBudget, Read, content_streams, content_streams_within, dict_in_place, drawn_before_damage, first_op_after_damage, form_call,
joined, resource_dict, rewritten_contents, splice,
};
pub use lines::{MAX_FORM_DEPTH, MAX_FORM_VISITS, glyph_text_lines, text_lines};
pub use rewrite::{LineEdit, replace_line, rewrite_block};
use rewrite::{coincident_ops, rewrite_block_from};
crates/edit/src/text/blocks.rs
+12 -5
View File
@@ -1,7 +1,7 @@
use super::interp::{Carry, interpret};
use super::lines::{content_streams, page_dict, reading_lines, rewritten_contents, splice, text_lines};
use super::lines::{content_streams, joined, page_dict, reading_lines, rewritten_contents, splice, text_lines};
use super::rewrite::{LineEdit, coincident_ops, rewrite_block};
use super::{Document, EditError, HashMap, Object, Op, Pieces, TextLine, group_blocks, regions};
use super::{Document, EditError, HashMap, Object, Op, TextLine, group_blocks, regions};
/// An editable text region: adjacent runs on visual rows, followed by aligned rows.
/// A region may contain several source fonts, sizes, colours and weights.
@@ -37,6 +37,13 @@ pub struct TextBlock {
}
impl TextBlock {
/// The one size a rewrite and the editor set an OCR phrase in, in place of each word's own
/// (#3313): the block's size, which for invisible text is its words' median. `None` for
/// visible text, whose runs keep their sizes.
pub fn phrase_size(&self) -> Option<f64> {
(self.invisible && self.size.is_finite() && self.size > 0.0).then_some(self.size)
}
/// Preserve styles outside the changed ranges; inserted characters inherit the surrounding
/// source run. The texts are aligned word by word, so words left untouched between two edits
/// keep their own runs (#2198); past [`MAX_CHANGED_WORDS`] the draft counts as one change.
@@ -270,7 +277,7 @@ pub fn clear_block(doc: &mut Document, page: usize, block: usize) -> Result<(),
let p = page_dict(doc, page)?;
let res = p.dict.get(b"Resources").and_then(|r| doc.resolve(r).as_dict().cloned()).unwrap_or_default();
let streams = content_streams(doc, &p.dict)?;
let pieces = Pieces::join(&streams.iter().map(|(_, d)| d.as_slice()).collect::<Vec<_>>());
let pieces = joined(&streams);
let ops = pieces.parse().ops;
let shown = interpret(doc, &ops, &res, &mut HashMap::new(), &mut Carry::new(), None, None)?;
let mut patches = HashMap::new();
@@ -297,7 +304,7 @@ pub fn clear_block(doc: &mut Document, page: usize, block: usize) -> Result<(),
patches.insert(s.op, replacement);
}
let data = splice(&pieces, &ops, |i| patches.remove(&i).map_or((Vec::new(), true), |v| (v, false)));
let contents = rewritten_contents(doc, &streams, data);
let contents = rewritten_contents(doc, &streams, data)?;
doc.update_dict(p.obj, |d| d.set(b"Contents".to_vec(), contents))?;
Ok(())
}
@@ -400,7 +407,7 @@ pub fn replace_block(doc: &mut Document, page: usize, block: usize, text: &str)
pub struct BlockStyle {
/// Disjoint UTF-8 ranges in the replacement, each preserving unspecified source style.
pub ranges: Vec<RangeStyle>,
/// Explicit page font resource (from register_text_font or the page font inventory).
/// Explicit page font resource or bundled document-font choice from the page font inventory.
pub resource: Option<String>,
/// A standard font family, bold, italic.
pub family: Option<(crate::added::Family, bool, bool)>,
crates/edit/src/text/editor.rs
+37 -4
View File
@@ -2,15 +2,30 @@ use super::blocks::{BlockStyle, TextBlock};
use super::fallback::MAX_TYPE3_GLYPHS;
use super::lines::{page_dict, text_lines};
use super::rewrite::rewrite_block_from;
use super::{DecodeBudget, Document, EditError, Metrics, Object, TextLine, TextSession};
use super::{DecodeBudget, Dict, Document, EditError, Metrics, Object, TextLine, TextSession};
/// An embedded sfnt source face, read only for the inline editor. Bytes stay in memory and
/// are never installed, copied into assets, or used as a font for another document.
/// An embedded source font or an approved bundled document-font choice.
/// Source bytes stay in memory and are never installed or copied into another document.
#[derive(Clone, Debug)]
pub struct TextEditorFont {
pub resource: String,
pub face: String,
pub bytes: Vec<u8>,
/// A selectable replacement, rather than a source/matching preview face. Merely listing
/// this choice must not reload an open editor's font atlas.
pub choice_only: bool,
}
/// Reserved document-font choice for explicitly replacing text with the bundled Thai face.
/// This is a selector, never a font resource written into a PDF.
const BUILTIN_THAI_FONT: &str = "builtin:thai";
pub(super) fn builtin_thai_choice(fonts: &Dict) -> String {
let mut choice = BUILTIN_THAI_FONT.to_owned();
while fonts.contains(choice.as_bytes()) {
choice.push('_');
}
choice
}
pub fn text_editor_fonts(doc: &Document, page: usize) -> Result<Vec<TextEditorFont>, EditError> {
@@ -32,6 +47,7 @@ pub fn text_editor_fonts(doc: &Document, page: usize) -> Result<Vec<TextEditorFo
resource: String::from_utf8_lossy(name).into_owned(),
face: Metrics::from_dict(doc, &font).base_font.rsplit('+').next().unwrap_or_default().to_owned(),
bytes,
choice_only: false,
});
}
// The approved full face follows all source subsets in the editor's font stack. It can
@@ -43,9 +59,26 @@ pub fn text_editor_fonts(doc: &Document, page: usize) -> Result<Vec<TextEditorFo
continue;
}
budget -= full.bytes.len();
result.push(TextEditorFont { resource: format!("PCApproved{}", full.name()), face: face.clone(), bytes: full.bytes.to_vec() });
result.push(TextEditorFont {
resource: format!("PCApproved{}", full.name()),
face: face.clone(),
bytes: full.bytes.to_vec(),
choice_only: false,
});
}
}
// A missing source family must not make the bundled Thai replacement undiscoverable.
// Offering it does not change the source; callers explicitly select this choice to replace it.
if let Some(thai) = pdfcraft_fonts::document_thai_font()
&& thai.bytes.len() <= budget
{
result.push(TextEditorFont {
resource: builtin_thai_choice(&fonts),
face: thai.family.into(),
bytes: thai.bytes.to_vec(),
choice_only: true,
});
}
Ok(result)
}
crates/edit/src/text/fallback.rs
+4 -4
View File
@@ -24,13 +24,13 @@ const HEBREW_FONT_NAME: &str = "PCHebrew";
/// The `/Name` of the Type 3 fonts generated from the bundled Latin (Inter) face.
const LATIN_FONT_NAME: &str = "PCLatin";
/// Whether `font` is one of those generated fonts (Latin, Japanese or Hebrew): a replacement longer
/// Whether `font` is one of those generated fonts (Latin, Japanese, Hebrew or Thai): a replacement longer
/// than one of them is split across several, which read back as one line (see [`same_font`]).
pub(super) fn is_type3_fallback(font: &Dict) -> bool {
font.name(b"Subtype") == Some(b"Type3")
&& font
.name(b"Name")
.is_some_and(|n| n == JAPANESE_FONT_NAME.as_bytes() || n == HEBREW_FONT_NAME.as_bytes() || n == LATIN_FONT_NAME.as_bytes())
&& font.name(b"Name").is_some_and(|n| {
n == JAPANESE_FONT_NAME.as_bytes() || n == HEBREW_FONT_NAME.as_bytes() || n == LATIN_FONT_NAME.as_bytes() || n == b"PCThai"
})
}
/// The generated font's `/Name` and its resource prefix: the bundled Latin face, Hebrew, or the
crates/edit/src/text/forms.rs
+283
View File
@@ -0,0 +1,283 @@
//! Make text in ordinary Form XObjects available to the existing paragraph editor.
//!
//! This replaces each supported placement's `Do`, never the shared form object. The same
//! source glyph operators become page content; no cover rectangle or replacement text box is
//! added. Unsupported compositing and malformed streams stay intact.
use pdfcraft_content::Matrix;
use pdfcraft_cos::{Dict, Document, Object};
use super::{ContentStream, Read, content_streams, joined, page_dict, rewritten_contents, splice};
use crate::{EditError, ungroup};
/// Result of preparing one page. Refused groups remain in the page, with an actionable reason.
#[derive(Clone, Debug, Default)]
pub struct Preparation {
pub expanded: usize,
pub refused: Vec<String>,
pub repaired_fonts: usize,
}
/// How many levels of nested groups preparation opens on one page. Text nested deeper stays
/// grouped and is reported in [`Preparation::refused`].
const MAX_LEVELS: usize = 8;
/// What preparing one page may decode, and add to its content, in all.
const MAX_PREPARED_BYTES: usize = 16 << 20;
/// What one pass of [`prepare`] inlined, as positions in the next pass's parse of the page.
struct Inlined {
/// How many operators that parse has; with any other count the positions are wrong.
ops: usize,
/// Where each inlined form's operators are: the placements in them are new.
ranges: Vec<std::ops::Range<usize>>,
/// Earlier placements refused only because their resources needed renaming
/// ([`crate::ungroup::Refusal::renaming`]), each with the reason it was given.
waiting: Vec<(usize, String)>,
}
/// Prepare grouped source text atomically, within eight levels and 64 placements per page, and
/// repair the page fonts' damaged legacy Thai maps (`repair_thai_fonts`), grouped or not.
/// Shared forms and other pages are untouched. Preparation is a distinct undoable engine edit.
pub fn prepare(doc: &mut Document, page: usize) -> Result<Preparation, EditError> {
let mut next = doc.clone();
let mut result = Preparation::default();
let mut bytes_left = MAX_PREPARED_BYTES;
let mut visits = 0usize;
let mut decode = pdfcraft_cos::DecodeBudget::new(MAX_PREPARED_BYTES, 0);
// Forms found to be artwork, or refused for a reason in the form itself (`Refusal::intrinsic`):
// a placement of one that an expanded group adds is left as it is without being read or
// counted, so logos drawn inside many text groups don't use up the limit. A form refused only
// where it is drawn is read again where an expanded group draws it.
let mut settled = std::collections::HashSet::new();
// What the previous pass inlined. Only the placements inside it are new. Every other placement
// was read and counted on an earlier pass, so a group refused where it is drawn counts once.
// Reading one again gives the same answer (an inlined form restores the graphics state it
// changes, and the decode budget only shrinks), except for a group refused because its
// resources needed renaming: names merged in for other groups since can make that
// unnecessary. Such a group is read again, uncounted, once its resource names say so.
let mut inlined: Option<Inlined> = None;
// One pass opens one level of every placement; the last pass only reports what is deeper.
for pass in 0..=MAX_LEVELS {
let p = page_dict(&next, page)?;
let streams = strict_streams(&next, &p.dict)?;
let pieces = joined(&streams);
let parsed = pieces.parse();
if parsed.skipped > 0 {
return Err(EditError::Invalid("the page has unreadable content; grouped text was left unchanged".into()));
}
let ops = parsed.ops;
// `None` reads every placement: on the first pass, and if the page didn't parse back to
// the operators written, where the positions can't be trusted. Reading one again can only
// count it twice; skipping one could leave its text grouped without a reason.
let (new, mut waiting) = match inlined.take() {
Some(i) if i.ops == ops.len() => (Some(i.ranges), i.waiting),
_ => (None, Vec::new()),
};
let mut still_waiting = Vec::new();
let resources = p.dict.get(b"Resources").map(|r| next.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default();
let space = ungroup::page_blend_space(&next, &p.dict);
let images: std::collections::BTreeMap<_, _> =
crate::page_images(&next, page)?.into_iter().filter(|i| i.is_form).map(|i| (i.op, i)).collect();
let mut replacements = std::collections::BTreeMap::new();
let mut merged = resources;
let (mut refused_here, mut expanded_here) = (std::collections::HashSet::new(), std::collections::HashSet::new());
for (at, op) in ops.iter().enumerate() {
let Some(image) = images.get(&at) else { continue };
if image.object.is_some_and(|o| settled.contains(&o)) {
continue;
}
// An earlier placement is read again only when it was refused for a rename that the
// names merged since have made unnecessary. The last pass opens nothing, so there it
// keeps its reason.
let earlier = new.as_ref().is_some_and(|ranges| !ranges.iter().any(|r| r.contains(&at)));
if earlier {
let Some(w) = waiting.iter().position(|(i, _)| *i == at) else { continue };
let (_, reason) = waiting.swap_remove(w);
// The name exactly as the `Do` writes it: a name need not be UTF-8, and
// `image.name` is only its display form.
if pass == MAX_LEVELS || !op.last_name().is_some_and(|name| ungroup::merges_without_renaming(&next, &merged, name)) {
still_waiting.push((at, reason));
continue;
}
// What reading it again finds replaces the reason it was given.
if let Some(i) = result.refused.iter().position(|r| *r == reason) {
result.refused.remove(i);
}
}
if !earlier {
visits = visits.saturating_add(1);
if visits > 64 {
return Err(EditError::Invalid("too many grouped text placements on this page (limit 64); the page was left unchanged".into()));
}
}
match ungroup::ungroup_ops(&next, &merged, &ops, at, space.as_ref(), Matrix(image.matrix), &mut decode) {
Ok(ungroup::Ungrouped { ops: inline, resources: mut res, patterns, shows_text }) => {
// Vector and image artwork needs no preparation. Nested text groups are
// visited on the next pass, with their inherited placement and clip in force.
match shows_text {
Some(true) => {}
Some(false) => {
settled.extend(image.object);
continue;
}
// Whether it has text can't be known: it is left as it is, and said.
None => {
result.refused.push(format!("{}: it draws recursive forms, or nested content that can't be checked", image.name));
continue;
}
}
if pass == MAX_LEVELS {
result.refused.push(format!("{}: its text is grouped more than {MAX_LEVELS} levels deep", image.name));
continue;
}
let data_len = pdfcraft_content::serialize_ops(&inline).len();
bytes_left = bytes_left.checked_sub(data_len).ok_or_else(|| {
EditError::Invalid("grouped page content exceeds the 16 MiB editing budget; the page was left unchanged".into())
})?;
if !patterns.is_empty() {
let mut table = res.get(b"Pattern").map(|r| next.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default();
for (name, value) in patterns {
table.set(name, Object::Ref(next.add(value)));
}
res.set(b"Pattern".to_vec(), Object::Dict(table));
}
merged = res;
replacements.insert(at, inline);
result.expanded += 1;
expanded_here.extend(image.object);
}
Err(refusal) => {
let reason = format!("{}: {}", image.name, refusal.error);
if refusal.renaming {
still_waiting.push((at, reason.clone()));
}
if refusal.intrinsic {
refused_here.extend(image.object);
}
result.refused.push(reason);
}
}
}
settled.extend(refused_here.difference(&expanded_here).copied());
if replacements.is_empty() {
break;
}
// Where an operator lands in the next pass: each replaced `Do` before it becomes its
// inlined operators.
let moved = |i: usize| replacements.range(..i).fold(i, |n, (_, inline)| n.saturating_add(inline.len()).saturating_sub(1));
inlined = Some(Inlined {
ops: moved(ops.len()),
ranges: replacements.iter().map(|(at, inline)| moved(*at)..moved(*at).saturating_add(inline.len())).collect(),
waiting: still_waiting.into_iter().map(|(at, reason)| (moved(at), reason)).collect(),
});
let changed = splice(&pieces, &ops, |at| match replacements.remove(&at) {
Some(ops) => (ops, false),
None => (Vec::new(), true),
});
let contents = rewritten_contents(&mut next, &streams, changed)?;
next.update_dict(p.obj, |d| {
d.set(b"Contents".to_vec(), contents);
d.set(b"Resources".to_vec(), Object::Dict(merged));
})?;
}
result.refused.sort();
result.refused.dedup();
// The page's own damaged maps are repaired too, those of text that was never grouped
// included: its text reads wrong all the same.
result.repaired_fonts = repair_thai_fonts(&mut next, page)?;
if result.expanded > 0 || result.repaired_fonts > 0 {
for reason in &result.refused {
next.note_repair(format!("Grouped text left intact: {reason}"));
}
if result.expanded > 0 {
next.note_repair(format!(
"Prepared {} grouped text placement(s) on page {} for direct source-text editing",
result.expanded,
page.saturating_add(1)
));
}
*doc = next;
}
Ok(result)
}
/// Whether grouped artwork `image` on `page` (its index in `page_images`) shows text that
/// [`prepare`] would make editable, read as its first pass reads it, without changing anything:
/// a logo or a refused group is `false`. A group refused only because its resources need renaming
/// is `true`, as another group's names can still let [`prepare`] open it.
pub fn grouped_text_at(doc: &Document, page: usize, image: usize) -> Result<bool, EditError> {
let Some(img) = crate::page_images(doc, page)?.into_iter().nth(image).filter(|i| i.is_form) else { return Ok(false) };
let p = page_dict(doc, page)?;
let parsed = joined(&strict_streams(doc, &p.dict)?).parse();
if parsed.skipped > 0 {
return Ok(false);
}
let resources = p.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default();
let space = ungroup::page_blend_space(doc, &p.dict);
let mut budget = pdfcraft_cos::DecodeBudget::new(MAX_PREPARED_BYTES, 0);
Ok(match ungroup::ungroup_ops(doc, &resources, &parsed.ops, img.op, space.as_ref(), Matrix(img.matrix), &mut budget) {
Ok(u) => u.shows_text == Some(true),
Err(r) => r.renaming,
})
}
/// A font is copied into this page: repairing the map must never alter a shared form/font.
fn repair_thai_fonts(doc: &mut Document, page: usize) -> Result<usize, EditError> {
let p = page_dict(doc, page)?;
let mut res = p.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default();
let mut fonts = res.get(b"Font").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default();
let mut count = 0;
let mut candidates = 0usize;
for (name, value) in fonts.clone().iter() {
let Some(mut font) = doc.resolve(value).as_dict().cloned() else { continue };
if font.name(b"BaseFont").is_some_and(|n| String::from_utf8_lossy(n).contains("THSarabunPSK")) {
candidates += 1;
if candidates > 64 {
return Err(EditError::Invalid("too many legacy Thai font maps to repair safely; the page was left unchanged".into()));
}
} else {
continue;
}
let Some(map) = pdfcraft_fonts::pdf::recovered_thai_map(doc, &font) else { continue };
// Append corrective bfchar entries after the original entries so valid Latin mappings
// remain. The original map's accidental oversized range is still overridden for Thai.
let Some(Object::Stream(tu)) = font.get(b"ToUnicode").map(|r| doc.resolve(r)).as_deref().cloned() else { continue };
let old = pdfcraft_cos::DecodeBudget::new(1 << 20, 0).decode_strict(doc, &tu)?;
// After the last `endcmap`: a stream with more than one must not have an earlier block
// (or its oversized range) after the corrective entries.
let marker = b"endcmap";
let Some(at) = old.windows(marker.len()).rposition(|w| w == marker) else { continue };
let mut added = format!("\n{} beginbfchar\n", map.len());
for (code, ch) in map {
added.push_str(&format!("<{code:04x}> <{:04x}>\n", u32::from(ch)));
}
added.push_str("endbfchar\n");
let mut bytes = old.get(..at).unwrap_or_default().to_vec();
bytes.extend_from_slice(added.as_bytes());
bytes.extend_from_slice(old.get(at..).unwrap_or_default());
let tu = doc.add(Object::Stream(pdfcraft_cos::Stream::flate(Dict::new(), &bytes)));
font.set(b"ToUnicode".to_vec(), Object::Ref(tu));
let object = doc.add(Object::Dict(font));
fonts.set(name.clone(), Object::Ref(object));
count += 1;
}
if count > 0 {
res.set(b"Font".to_vec(), Object::Dict(fonts));
doc.update_dict(p.obj, |d| d.set(b"Resources".to_vec(), Object::Dict(res)))?;
doc.note_repair(format!(
"Recovered the damaged legacy THSarabunPSK Thai character map in {count} page font(s); original glyph programs were preserved"
));
}
Ok(count)
}
/// Editing cannot infer the state after a damaged page stream. Every piece must decode whole
/// before a placement is expanded, rather than treating an undecodable piece as empty.
fn strict_streams(doc: &Document, page: &Dict) -> Result<Vec<ContentStream>, EditError> {
let streams = content_streams(doc, page)?;
if streams.iter().any(|s| s.read != Read::Whole) {
return Err(EditError::Invalid("the page has a missing or unreadable content stream".into()));
}
Ok(streams)
}
crates/edit/src/text/lines.rs
+132 -23
View File
@@ -61,16 +61,37 @@ impl TextLine {
}
}
/// The page's content streams as listed, each with its decoded data (empty when it doesn't
/// decode), within one page's [`DecodeBudget`]. An error when the content runs past it: reading
/// or rewriting a page from part of its content would lose the rest.
pub(crate) fn content_streams(doc: &Document, page: &Dict) -> Result<Vec<(Object, Vec<u8>)>, EditError> {
/// One entry of a page's `/Contents`, and what it decoded to.
pub(crate) struct ContentStream {
/// The entry as listed.
pub(crate) entry: Object,
/// Its decoded data, empty when it doesn't decode.
pub(crate) data: Vec<u8>,
/// How much of it decoded.
pub(crate) read: Read,
}
/// How much of a content stream decoded ([`rewritten_contents`] says what each allows).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(crate) enum Read {
Whole,
/// Damaged partway: the data is what decoded before the damage. What follows it is unknown,
/// the graphics state and syntax the later streams start in too.
Damaged,
/// Not at all (a filter this reader lacks, say): the data is empty.
Unread,
}
/// The page's content streams as listed, each with its decoded data, within one page's
/// [`DecodeBudget`]. An error when the content runs past it: reading or rewriting a page from
/// part of its content would lose the rest.
pub(crate) fn content_streams(doc: &Document, page: &Dict) -> Result<Vec<ContentStream>, EditError> {
content_streams_within(doc, page, &mut DecodeBudget::page())
}
/// [`content_streams`], decoded against `budget`, which a walk of the page's forms then goes on
/// spending.
pub(crate) fn content_streams_within(doc: &Document, page: &Dict, budget: &mut DecodeBudget) -> Result<Vec<(Object, Vec<u8>)>, EditError> {
pub(crate) fn content_streams_within(doc: &Document, page: &Dict, budget: &mut DecodeBudget) -> Result<Vec<ContentStream>, EditError> {
let list: Vec<Object> = match page.get(b"Contents") {
None => Vec::new(),
Some(c) => match &*doc.resolve(c) {
@@ -79,20 +100,26 @@ pub(crate) fn content_streams_within(doc: &Document, page: &Dict, budget: &mut D
},
};
let mut out = Vec::with_capacity(list.len());
for o in list {
let data = match &*doc.resolve(&o) {
Object::Stream(s) => match budget.decode(s) {
Ok(data) => data,
for entry in list {
let (data, read) = match &*doc.resolve(&entry) {
Object::Stream(s) => match budget.decode_with_damage(doc, s, usize::MAX) {
Ok((data, damaged)) => (data, if damaged { Read::Damaged } else { Read::Whole }),
Err(e) if budget.remaining() == 0 => return Err(EditError::Invalid(format!("the page content is too large to read: {e}"))),
Err(_) => Vec::new(),
Err(_) => (Vec::new(), Read::Unread),
},
_ => Vec::new(),
// Nothing to lose: a null or missing entry draws nothing.
_ => (Vec::new(), Read::Whole),
};
out.push((o, data));
out.push(ContentStream { entry, data, read });
}
Ok(out)
}
/// The page's streams joined into the one stream they are: an operator may span two of them.
pub(crate) fn joined(streams: &[ContentStream]) -> Pieces {
Pieces::join(&streams.iter().map(|s| s.data.as_slice()).collect::<Vec<_>>())
}
/// Rewrite a page's content (its streams joined, see [`Pieces`]): `edit(i)` gives the operators
/// to insert before operator `i` of `ops` (parsed from the joined pieces) and whether to keep it.
/// Everything else is copied byte for byte. A page's streams are one stream in pieces, split
@@ -120,26 +147,108 @@ pub(crate) fn splice(pieces: &Pieces, ops: &[Op], mut edit: impl FnMut(usize) ->
pieces.splice(edits)
}
/// An error when content at `/Contents` index `at` (a stream there, or one inserted there) is drawn
/// after a stream not read whole, damaged partway or not decoding at all: what wasn't read can
/// change how it reads, the graphics state it draws in among it.
pub(crate) fn drawn_before_damage(streams: &[ContentStream], at: usize) -> Result<(), EditError> {
if at > first_damaged(streams) {
return Err(EditError::Invalid(
"this change comes after a content stream that is damaged or can't be decoded, and what wasn't read can change how the rest of the page reads"
.into(),
));
}
Ok(())
}
/// The index of the first stream not read whole, or the number of streams.
fn first_damaged(streams: &[ContentStream]) -> usize {
streams.iter().position(|s| s.read != Read::Whole).unwrap_or(streams.len())
}
/// The index of the first of `ops`, parsed from `pieces` (the [`joined`] `streams`), drawn after a
/// stream not read whole, or `ops.len()`. A `q … Q` still open there may hold what wasn't read,
/// painting in the state it sets. An operator counts by its keyword: one whose operands start
/// before such a stream takes them from around what wasn't read.
pub(crate) fn first_op_after_damage(streams: &[ContentStream], pieces: &Pieces, ops: &[Op]) -> usize {
let damaged = first_damaged(streams);
ops.iter().position(|op| pieces.pieces_of(op).1 > damaged).unwrap_or(ops.len())
}
/// Whether `new` changes a text object still open at the first stream not read whole. The text
/// object goes on in it, and the text it shows there starts where this text ends: a change of this
/// text moves it. A change before the text object doesn't.
fn changes_text_open_into_damage(streams: &[ContentStream], new: &[Vec<u8>]) -> bool {
let upto = first_damaged(streams).saturating_add(1);
if upto > streams.len() {
return false;
}
let before = Pieces::join(&streams.iter().take(upto).map(|s| s.data.as_slice()).collect::<Vec<_>>());
let mut open = None;
for op in before.parse().ops {
match op.op.as_slice() {
b"BT" => open = Some(op.span.start),
b"ET" => open = None,
_ => {}
}
}
let Some(at) = open else { return false };
let after = Pieces::join(&new.iter().take(upto).map(Vec::as_slice).collect::<Vec<_>>());
// Unchanged from the text object's start to the stream not read whole: it ends the same.
!before.data().get(at..).is_some_and(|text| after.data().ends_with(text))
}
/// The page's `/Contents` after a [`splice`]: pieces whose bytes changed become new streams
/// (keeping their dictionaries), the others stay as they were.
pub(crate) fn rewritten_contents(doc: &mut Document, streams: &[(Object, Vec<u8>)], new: Vec<Vec<u8>>) -> Object {
let mut contents: Vec<Object> = Vec::with_capacity(streams.len());
for ((obj, old), data) in streams.iter().zip(new) {
if *old == data {
contents.push(obj.clone());
/// (keeping their dictionaries), the others stay as they were. An error, before anything is
/// added, when a piece that changed is a stream that didn't decode whole, or comes after one.
/// Written from what decoded, a damaged stream would lose everything after the damage, and one
/// that doesn't decode all of it. What follows the damage is unknown, so a later stream may not
/// read as it seems: a string or comment the damage left open runs on into it, and operators the
/// damage hides can change the graphics state it draws in. A stream that doesn't decode hides its
/// operators all the same.
pub(crate) fn rewritten_contents(doc: &mut Document, streams: &[ContentStream], new: Vec<Vec<u8>>) -> Result<Object, EditError> {
// The last stream that changed: when it is drawn before the damage, so are the others.
if let Some(i) = streams.iter().zip(&new).rposition(|(s, data)| s.data != *data) {
drawn_before_damage(streams, i)?;
}
if changes_text_open_into_damage(streams, &new) {
return Err(EditError::Invalid(
"this change is in a text object that goes on in a content stream that is damaged or can't be decoded, whose text starts where this text ends"
.into(),
));
}
for (s, data) in streams.iter().zip(&new) {
if s.data == *data {
continue;
}
let mut dict = match &*doc.resolve(obj) {
match s.read {
Read::Whole => {}
Read::Damaged => {
return Err(EditError::Invalid(
"this change is in a content stream that is damaged partway, and rewriting it would lose what follows the damage".into(),
));
}
Read::Unread => {
return Err(EditError::Invalid("this change is in a content stream that can't be decoded, and rewriting it would lose it".into()));
}
}
}
let mut contents: Vec<Object> = Vec::with_capacity(streams.len());
for (s, data) in streams.iter().zip(new) {
if s.data == data {
contents.push(s.entry.clone());
continue;
}
let mut dict = match &*doc.resolve(&s.entry) {
Object::Stream(s) => s.dict.clone(),
_ => Dict::new(),
};
dict.remove(b"Length");
contents.push(Object::Ref(doc.add(Object::Stream(Stream::flate(dict, &data)))));
}
match <[Object; 1]>::try_from(contents) {
Ok(match <[Object; 1]>::try_from(contents) {
Ok([one]) => one,
Err(many) => Object::Array(many),
}
})
}
pub(crate) fn page_dict(doc: &Document, page: usize) -> Result<pdfcraft_model::Page, EditError> {
@@ -201,7 +310,7 @@ fn text_lines_in(doc: &Document, p: &pdfcraft_model::Page, actual_text: bool) ->
let mut carry = Carry::new();
// The streams are one stream in pieces (§7.8.2), split between any two tokens: parse them
// joined, so an operator whose operands end one piece still has them.
let pieces = Pieces::join(&content_streams(doc, &p.dict)?.into_iter().map(|(_, d)| d).collect::<Vec<_>>());
let pieces = joined(&content_streams(doc, &p.dict)?);
let ops = pieces.parse().ops;
let properties = res.get(b"Properties").and_then(|o| doc.dict(o)).unwrap_or_default();
let shown = interpret(doc, &ops, &res, &mut cache, &mut carry, None, actual_text.then_some(&properties))?;
@@ -608,7 +717,7 @@ pub(super) fn reading_lines(doc: &Document, page: usize) -> Result<(Vec<TextLine
// do: a comment, an operator's operands or a marked-content sequence (its /ActualText too)
// can run on from one stream into the next. Text still counts as the stream its operator
// starts in, which keeps lines from different streams apart.
let pieces = Pieces::join(&streams.into_iter().map(|(_, data)| data).collect::<Vec<_>>());
let pieces = joined(&streams);
let ops = pieces.parse().ops;
let stream_of = |i: usize| ops.get(i).map_or(0, |op| pieces.pieces_of(op).0);
walk(&mut w, &ops, &res, &mut carry, &stream_of)?;
crates/edit/src/text/rewrite.rs
+20 -14
View File
@@ -4,10 +4,10 @@ use super::fallback::{
source_program_completes, source_show, source_space, source_width, type3_encode, type3_font,
};
use super::interp::{Carry, Origin, TextState, interpret};
use super::lines::{content_streams, page_dict, rewritten_contents, splice, text_lines};
use super::lines::{content_streams, joined, page_dict, rewritten_contents, splice, text_lines};
use super::{
Dict, Document, EditError, HashMap, Matrix, Metrics, Object, Op, PdfString, Pieces, TextLine, TextSession, alignment_frame, group_blocks,
is_centred, parse, rewrite_styled_block,
Dict, Document, EditError, HashMap, Matrix, Metrics, Object, Op, PdfString, TextLine, TextSession, alignment_frame, group_blocks, is_centred,
parse, rewrite_styled_block,
};
/// What replacing a line did.
@@ -56,7 +56,7 @@ pub(crate) fn replace_line_with_limit(doc: &mut Document, page: usize, line: usi
let mut res = p.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default();
let mut fonts_res = res.get(b"Font").map(|f| doc.resolve(f)).and_then(|f| f.as_dict().cloned()).unwrap_or_default();
let streams = content_streams(doc, &p.dict)?;
let pieces = Pieces::join(&streams.iter().map(|(_, d)| d.as_slice()).collect::<Vec<_>>());
let pieces = joined(&streams);
let ops = pieces.parse().ops;
let first = target.ops.first().copied().filter(|i| *i < ops.len()).ok_or_else(|| EditError::Invalid("the page's content changed".into()))?;
// The line's own font, when it can show every character.
@@ -127,7 +127,7 @@ pub(crate) fn replace_line_with_limit(doc: &mut Document, page: usize, line: usi
}
// Rebuild: the line's first operator becomes the replacement, its others go.
let new_data = splice(&pieces, &ops, |i| if i == first { (std::mem::take(&mut replacement), false) } else { (Vec::new(), !drops.contains(&i)) });
let contents = rewritten_contents(doc, &streams, new_data);
let contents = rewritten_contents(doc, &streams, new_data)?;
let page_ref = p.obj;
if substituted.is_some() {
res.set(b"Font".to_vec(), Object::Dict(fonts_res));
@@ -749,12 +749,13 @@ struct ParaStyle {
k: f64,
}
/// Spacing and scaling (text state), with the chosen values.
fn paragraph_style(o: &Origin, style: &BlockStyle) -> Result<(String, ParaStyle), EditError> {
/// Spacing and scaling (text state), with the chosen values. `phrase` is the size an OCR phrase
/// is drawn at when none is chosen, in place of its first word's own (#3313).
fn paragraph_style(o: &Origin, style: &BlockStyle, phrase: Option<f64>) -> Result<(String, ParaStyle), EditError> {
let (font_name, old_size) = o.state.font.clone().ok_or_else(|| EditError::Invalid("the paragraph has no font".into()))?;
let k = o.k.max(1e-6);
// The size in text space: points ÷ the text-to-user scale.
let size = style.size.map_or(old_size, |pt| (pt / k).max(0.1));
let size = style.size.or(phrase).map_or(old_size, |pt| (pt / k).max(0.1));
let mut state = o.state.clone();
if let Some(c) = style.char_spacing {
state.char_spacing = c / k;
@@ -762,6 +763,11 @@ fn paragraph_style(o: &Origin, style: &BlockStyle) -> Result<(String, ParaStyle)
if let Some(sc) = style.scale {
state.scale = (sc / 100.0).clamp(0.1, 10.0);
}
// At the phrase's size, the first word's text matrix still carries its own width: scale it
// back, so the line is as wide as the scan whatever size OCR guessed for that word.
if phrase.is_some() && style.size.is_none() && style.scale.is_none() {
state.scale = (state.scale * old_size / size).clamp(0.1, 10.0);
}
Ok((font_name, ParaStyle { state, size, old_size, k }))
}
@@ -834,7 +840,7 @@ pub(crate) fn rewrite_block_from(
}
let upright = dispatch.upright;
let streams = content_streams(doc, &p.dict)?;
let pieces = Pieces::join(&streams.iter().map(|(_, d)| d.as_slice()).collect::<Vec<_>>());
let pieces = joined(&streams);
let ops = pieces.parse().ops;
let text = paragraph_text(text.unwrap_or(&b.text));
// The characters to show, for choosing fonts: the breaks are moves, not glyphs.
@@ -842,7 +848,7 @@ pub(crate) fn rewrite_block_from(
let o = &first.origin;
let last = members.last().copied().unwrap_or(first);
let emit = &last.origin;
let (font_name, ParaStyle { state: o_state, size, old_size, k }) = paragraph_style(o, style)?;
let (font_name, ParaStyle { state: o_state, size, old_size, k }) = paragraph_style(o, style, b.phrase_size())?;
let source = source_font(doc, &fonts_res, &font_name, first, &flat, style);
let reuse = source.reuse;
// The chosen style also determines the real Japanese fallback outlines and advances.
@@ -970,7 +976,7 @@ pub(crate) fn rewrite_block_from(
let insert = if i == at { std::mem::take(&mut block_ops) } else { Vec::new() };
(insert, !dropped.contains(&i))
});
let contents = rewritten_contents(doc, &streams, new_data);
let contents = rewritten_contents(doc, &streams, new_data)?;
let new_font = !reuse;
if new_font {
res.set(b"Font".to_vec(), Object::Dict(fonts_res));
@@ -1040,7 +1046,7 @@ pub(crate) fn selectable_blocks(
!b.lines
.iter()
.filter_map(|i| lines.get(*i))
.any(|l| streams.get(l.stream).and_then(|(o, _)| o.as_ref()).is_some_and(|r| excluded.contains(&r)))
.any(|l| streams.get(l.stream).and_then(|s| s.entry.as_ref()).is_some_and(|r| excluded.contains(&r)))
})
.collect())
}
@@ -1134,7 +1140,7 @@ pub(crate) fn translate_blocks(doc: &mut Document, page: usize, indexes: &[usize
}
let res = p.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default();
let fonts = res.get(b"Font").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default();
let pieces = Pieces::join(&streams.iter().map(|(_, d)| d.as_slice()).collect::<Vec<_>>());
let pieces = joined(&streams);
let ops = pieces.parse().ops;
let shown = interpret(doc, &ops, &res, &mut HashMap::new(), &mut Carry::new(), None, None)?;
let mut patches = HashMap::new();
@@ -1205,7 +1211,7 @@ pub(crate) fn translate_blocks(doc: &mut Document, page: usize, indexes: &[usize
return Err(EditError::Invalid("a selected text operation cannot be moved".into()));
}
let data = splice(&pieces, &ops, |i| patches.remove(&i).map_or((Vec::new(), true), |v| (v, false)));
let contents = rewritten_contents(doc, &streams, data);
let contents = rewritten_contents(doc, &streams, data)?;
doc.update_dict(p.obj, |d| d.set(b"Contents".to_vec(), contents))?;
Ok(())
}
crates/edit/src/text_regions.rs
+4 -3
View File
@@ -2,9 +2,10 @@
use super::{
BlockStyle, CraftFont, Dict, Document, EditError, HashMap, LineEdit, MAX_TYPE3_GLYPHS, Matrix, Metrics, Object, Op, PdfString, Pieces,
RangeStyle, TextBlock, TextLine, Type3Glyph, coincident_ops, content_streams, fallback_face, japanese_glyph_from, needs_type3, no_fallback_font,
no_glyph_for, page_dict, paragraph_separator, reveal_ops, rewritten_contents, same_base_font, same_font, source_encodes, source_family,
source_program, source_show, source_space, source_width, splice, strip_subset_tag, type3_faces, type3_glyph, union, write_type3_glyphs,
RangeStyle, TextBlock, TextLine, Type3Glyph, builtin_thai_choice, coincident_ops, content_streams, fallback_face, japanese_glyph_from,
needs_type3, no_fallback_font, no_glyph_for, page_dict, paragraph_separator, reveal_ops, rewritten_contents, same_base_font, same_font,
source_encodes, source_family, source_program, source_show, source_space, source_width, splice, strip_subset_tag, type3_faces, type3_glyph,
union, write_type3_glyphs,
};
#[path = "text_regions/grouping.rs"]
crates/edit/src/text_regions/grouping.rs
+22
View File
@@ -398,6 +398,17 @@ fn group_layout(lines: &[TextLine], mixed: bool) -> Vec<TextBlock> {
for block in &mut blocks {
block.reflow_text = reflow_text(block, lines);
}
for block in blocks.iter_mut().filter(|block| block.invisible && block.lines.len() > 1) {
// Each word counts once: neighbouring words OCR sized alike share a run.
let words = block
.lines
.iter()
.filter_map(|i| lines.get(*i))
.flat_map(|line| std::iter::repeat_n(line.size, line.text.split_whitespace().count().max(1)));
if let Some(size) = median_size(words) {
block.size = size;
}
}
blocks
}
@@ -452,3 +463,14 @@ fn break_gap(block: &TextBlock, lines: &[TextLine], row_index: usize) -> Option<
// measure leaves almost no room for the next word.
(slack <= size * 0.75 && width >= size * 8.0 && (next[2] - next[0]) + size * 0.25 > slack).then_some(1)
}
/// The size an OCR phrase is set in. Recognize text sizes each word from its own box and a guess
/// at how much of the em its letters fill, so one phrase's words come out several points apart
/// (#3313); a rewrite draws the phrase at this one size. `None` when no size is usable.
fn median_size(sizes: impl Iterator<Item = f64>) -> Option<f64> {
let mut sizes: Vec<f64> = sizes.filter(|size| size.is_finite() && *size > 0.0).collect();
sizes.sort_by(f64::total_cmp);
let middle = sizes.len() / 2;
let upper = sizes.get(middle).copied()?;
if sizes.len().is_multiple_of(2) { sizes.get(middle.saturating_sub(1)).map(|lower| (lower + upper) / 2.0) } else { Some(upper) }
}
crates/edit/src/text_regions/styled.rs
+676 -99
View File
@@ -2,9 +2,9 @@ use super::super::interp::TextState;
use super::grouping::{across, is_centred, row_rect};
use super::{
BlockStyle, CraftFont, Dict, Document, EditError, HashMap, LineEdit, MAX_TYPE3_GLYPHS, Matrix, Metrics, Object, Op, PdfString, Pieces,
RangeStyle, TextBlock, TextLine, Type3Glyph, coincident_ops, content_streams, fallback_face, japanese_glyph_from, needs_type3, no_fallback_font,
no_glyph_for, page_dict, reveal_ops, rewritten_contents, source_encodes, source_family, source_program, source_show, source_space, source_width,
splice, type3_faces, type3_glyph, write_type3_glyphs,
RangeStyle, TextBlock, TextLine, Type3Glyph, builtin_thai_choice, coincident_ops, content_streams, fallback_face, japanese_glyph_from,
needs_type3, no_fallback_font, no_glyph_for, page_dict, reveal_ops, rewritten_contents, source_encodes, source_family, source_program,
source_show, source_space, source_width, splice, type3_faces, type3_glyph, write_type3_glyphs,
};
// The draft is capped at 4 MiB; also bound the outline data prepared across all fallback faces.
@@ -104,9 +104,46 @@ impl FontResolver<'_> {
}
}
struct LegacyFace {
/// Shared by every resource name of one font object: its program is decoded and kept once.
glyphs: Option<std::rc::Rc<pdfcraft_fonts::pdf::LegacyThaiFont>>,
metrics: Metrics,
/// Weight and slant ([`FontStyle::of`]).
style: FontStyle,
/// The font object the resource names, when it is given by reference.
object: Option<pdfcraft_cos::ObjRef>,
/// Its text-space fallback ([`source_space`]), in ems.
space: f64,
}
/// What one edit may decode, and so keep, of the font programs of its THSarabunPSK subsets
/// (each at most 16 MiB): a page may name many of them.
const MAX_THAI_PROGRAMS: usize = 64 << 20;
struct PreparedCharacters {
values: Vec<StyledChar>,
substituted: Option<String>,
thai: ThaiShaping,
}
/// How the Thai spans of an edit are shaped, by the font name their characters carry.
#[derive(Default)]
struct ThaiShaping {
/// Draft-only names drawn with the bundled Thai outlines.
outlined: std::collections::HashSet<String>,
/// Source faces shaped in their existing codes, by resource name.
legacy: HashMap<String, LegacyFace>,
/// Draft-only names whose rows, Latin ones included, are shaped and embedded with the
/// complete program of this same-face resource: a span its subsets can't show in place.
program: HashMap<String, String>,
}
/// How a Thai span keeps its source face ([`legacy_thai_font`]).
enum ThaiSpan {
/// In this resource's existing codes.
InPlace(String),
/// Shaped by this same-face resource's complete program with its own layout.
Program(String),
}
struct CharacterPreparation<'a> {
@@ -125,12 +162,93 @@ struct CharacterPreparation<'a> {
impl CharacterPreparation<'_> {
fn prepare(self) -> Result<PreparedCharacters, EditError> {
let Self { doc, first, lines, draft, spans, style, overrides, fonts, metrics, max_glyphs } = self;
let thai_choice = builtin_thai_choice(fonts);
let mut thai = ThaiShaping::default();
let mut thai_programs = pdfcraft_cos::DecodeBudget::new(MAX_THAI_PROGRAMS, 0);
let mut chars = Vec::new();
let mut substituted = None;
let mut fallback = FallbackBuilder::new(max_glyphs);
let mut resolver = FontResolver { doc, fonts, metrics, full_faces: HashMap::new(), programs: HashMap::new() };
for (range, index) in spans {
let source = lines.get(index).unwrap_or(first);
let span = draft.get(range.clone()).unwrap_or_default();
// Keep the exact source face. A different Thai face requires an explicit choice.
// Interface fallback fonts never enter a PDF.
let selected_resource = overrides.get(&index).and_then(|r| r.resource.as_deref()).or(style.resource.as_deref());
let use_thai_outlines = if selected_resource == Some(thai_choice.as_str()) {
let face = pdfcraft_fonts::document_thai_font().ok_or_else(|| {
EditError::Invalid(format!(
"the bundled Thai font ({thai_choice}) is not in this build, which has no craft-fonts Thai face; load a complete Thai font or choose one the page lists"
))
})?;
if !pdfcraft_fonts::embedding::covers(face.bytes, span) {
let missing = span.chars().find(|c| !pdfcraft_fonts::embedding::covers(face.bytes, c.encode_utf8(&mut [0; 4])));
return Err(EditError::Invalid(match missing {
Some(c) => format!("{} has no glyph for {c} (U+{:04X}); choose a font that covers the text", face.family, u32::from(c)),
None => format!("{} cannot show every character in this selection; choose a font that covers the text", face.family),
}));
}
substituted = Some(face.family.to_owned());
true
} else {
selected_resource.is_none() && matching_thai_outline(resolver.doc, resolver.fonts, source, span)
};
if use_thai_outlines {
substituted = pdfcraft_fonts::document_thai_font().map(|f| f.family.to_owned());
}
let existing_thai = if !use_thai_outlines {
let chosen = match selected_resource {
None => Some(source.clone()),
Some(resource) => resolver.fonts.get(resource.as_bytes()).and_then(|f| resolver.doc.dict(f)).map(|font| {
let mut chosen = source.clone();
chosen.font = resource.to_owned();
chosen.base_font = Metrics::from_dict(resolver.doc, &font).base_font;
chosen
}),
};
match chosen {
Some(chosen) => legacy_thai_font(resolver.doc, resolver.fonts, &chosen, span, &mut thai.legacy, &mut thai_programs)?,
None => None,
}
} else {
None
};
let outline_name = if use_thai_outlines {
// A draft-only name prevents this choice from affecting another style span that
// explicitly selects the original resource. It is never saved into the PDF.
let mut name = format!("PCOutlineThai{index}");
while resolver.metrics.contains_key(&name) {
name.push('_');
}
if let Some(m) = resolver
.metrics
.get(&source.font)
.or_else(|| source.origin.state.font.as_ref().and_then(|(name, _)| resolver.metrics.get(name)))
.cloned()
{
resolver.metrics.insert(name.clone(), m);
}
thai.outlined.insert(name.clone());
Some(name)
} else {
None
};
let (existing_thai, program_name) = match existing_thai {
Some(ThaiSpan::InPlace(name)) => (Some(name), None),
Some(ThaiSpan::Program(resource)) => {
// Draft-only too: another span may still show this resource's existing codes.
let mut name = format!("PCProgramThai{index}");
while resolver.metrics.contains_key(&name) {
name.push('_');
}
if let Some(m) = resolver.metrics.get(&source.font).cloned() {
resolver.metrics.insert(name.clone(), m);
}
thai.program.insert(name.clone(), resource);
(None, Some(name))
}
None => (None, None),
};
let start = range.start;
for (offset, ch) in draft.get(range).unwrap_or_default().char_indices() {
if ch == '\n' {
@@ -144,7 +262,11 @@ impl CharacterPreparation<'_> {
}
let text = ch.to_string();
let requested = overrides.get(&index).and_then(|range| range.resource.as_deref()).or(style.resource.as_deref());
let name = if let Some(name) = resolver.encodable_font(source, &text, requested)? {
let name = if let Some(name) = outline_name.as_ref().or(program_name.as_ref()) {
name.clone()
} else if let Some(name) = &existing_thai {
name.clone()
} else if let Some(name) = resolver.encodable_font(source, &text, requested)? {
name
} else if needs_type3(&text) {
fallback.prepare(source, ch, &text, chars.len())?
@@ -173,7 +295,7 @@ impl CharacterPreparation<'_> {
}
substituted = Some(written.family.to_owned());
}
Ok(PreparedCharacters { values: chars, substituted })
Ok(PreparedCharacters { values: chars, substituted, thai })
}
}
@@ -233,7 +355,16 @@ struct StyledLines<'a> {
type StyledChar = (char, usize, String);
type StyledRows = Vec<Vec<StyledChar>>;
type RowRanges = Vec<std::ops::Range<usize>>;
type ShapedRunCache = HashMap<(usize, usize), (Vec<u8>, pdfcraft_fonts::source_shaping::ShapedRun, String)>;
enum ShapedFont {
Existing(Vec<(String, Vec<u8>)>),
Program { bytes: Vec<u8>, family: String },
ThaiOutlines,
}
struct ShapedText {
run: pdfcraft_fonts::source_shaping::ShapedRun,
font: ShapedFont,
}
type ShapedRunCache = HashMap<(usize, usize), ShapedText>;
struct ShapedRows {
runs: ShapedRunCache,
@@ -243,6 +374,7 @@ struct ShapedRows {
fn prepare_shaped_rows(
doc: &Document,
fonts: &Dict,
thai: &ThaiShaping,
rows: &[Vec<StyledChar>],
kept: &[bool],
advance: impl Fn(&StyledChar) -> f64,
@@ -259,8 +391,8 @@ fn prepare_shaped_rows(
let segment = row.get(at..end).unwrap_or_default();
advances.insert((row_index, at), segment.iter().map(&advance).sum::<f64>());
let text: String = segment.iter().map(|character| character.0).collect();
if let Some((bytes, shaped, family)) = shape_source(doc, fonts, &item.2, &text)? {
runs.insert((row_index, at), (bytes, shaped, family));
if let Some(shaped) = shape_source(doc, fonts, thai, &item.2, &text)? {
runs.insert((row_index, at), shaped);
}
at = end;
}
@@ -268,16 +400,138 @@ fn prepare_shaped_rows(
Ok(ShapedRows { runs, advances })
}
fn embed_shaped_resources(doc: &mut Document, fonts: &mut Dict, runs: &ShapedRunCache) -> Result<HashMap<(usize, usize), String>, EditError> {
type ShapedResources = HashMap<(usize, usize), Vec<(String, Vec<u8>)>>;
fn embed_shaped_resources(doc: &mut Document, fonts: &mut Dict, runs: &ShapedRunCache) -> Result<ShapedResources, EditError> {
let mut resources = HashMap::new();
for (key, (bytes, shaped, family)) in runs {
let mappings: Vec<_> = shaped.glyphs.iter().map(|glyph| (glyph.text.clone(), glyph.id)).collect();
let resource = pdfcraft_fonts::embedding::embed_mapping(doc, fonts, bytes, family, &mappings).map_err(EditError::Invalid)?;
resources.insert(*key, resource);
for (key, shaped) in runs {
let glyphs = match &shaped.font {
ShapedFont::Existing(codes) => codes.clone(),
ShapedFont::ThaiOutlines => crate::added::thai_glyph_resources(doc, fonts, &shaped.run)?,
ShapedFont::Program { bytes, family } => {
let mappings: Vec<_> = shaped.run.glyphs.iter().map(|glyph| (glyph.text.clone(), glyph.id)).collect();
let resource = pdfcraft_fonts::embedding::embed_mapping(doc, fonts, bytes, family, &mappings).map_err(EditError::Invalid)?;
(0..shaped.run.glyphs.len())
.map(|i| {
let code = u16::try_from(i.saturating_add(1)).map_err(|_| EditError::Invalid("shaped glyph limit".into()))?;
Ok((resource.clone(), code.to_be_bytes().to_vec()))
})
.collect::<Result<Vec<_>, EditError>>()?
}
};
resources.insert(*key, glyphs);
}
Ok(resources)
}
/// A positioned glyph run, kept in one text object across Type 3 font shards.
struct ShapedRunDrawing<'a> {
doc: &'a Document,
fonts: &'a Dict,
shaped: &'a pdfcraft_fonts::source_shaping::ShapedRun,
resources: &'a [(String, Vec<u8>)],
text: &'a str,
matrix: [f64; 6],
size: f64,
thai: bool,
}
impl ShapedRunDrawing<'_> {
/// Draw and return the run's advance in text space, including cluster-safe spacing.
fn draw(&self, block_ops: &mut Vec<Op>, state: &mut TextState) -> Result<f64, EditError> {
let Self { doc, fonts, shaped, resources, text, matrix: tm, size, thai } = *self;
let n = pdfcraft_content::num;
let (resource, _) = self.resources.first().ok_or_else(|| EditError::Invalid("shaped run has no glyphs".into()))?;
let mut properties = Dict::new();
properties.set(b"ActualText".to_vec(), Object::String(PdfString::text(text)));
block_ops.push(Op::new("BDC", vec![Object::name("Span"), Object::Dict(properties)]));
state.font = Some((resource.clone(), size));
// One text object for the run: each glyph is placed where shaping put it by a TJ
// adjustment from where the glyph before it ends (its advance in the embedded
// font's widths), and raised by the text rise for a mark set above or below
// (ISO 32000-1 §9.4.3, §9.3.7). Font-program codes are separate from Unicode;
// ToUnicode and ActualText keep the logical text.
let per_unit = 1000.0 / size;
if !per_unit.is_finite() {
return Err(EditError::Invalid("the shaped text has no size".into()));
}
let mut shown_fonts = HashMap::new();
for (name, _) in resources {
if !shown_fonts.contains_key(name) {
let font = fonts.get(name.as_bytes()).and_then(|f| doc.dict(f)).ok_or_else(|| EditError::Invalid("shaped font is missing".into()))?;
shown_fonts.insert(name.clone(), Metrics::from_dict(doc, &font));
}
}
// Keep the original text settings. TJ compensates for the automatic PDF advance
// after each glyph, while shaping keeps marks attached to their cluster.
let (char_spacing, word_spacing, rise) = (state.char_spacing, state.word_spacing, state.rise);
block_ops.push(Op::new("BT", vec![]));
block_ops.extend(state.ops());
block_ops.push(Op::new("Tm", tm.iter().map(|v| n(*v)).collect()));
// `pen`: where the next glyph would start, in thousandths of the font size.
let (mut pen, mut extra, mut raised) = (0.0, 0.0, rise);
let mut shown_glyphs: Vec<Object> = Vec::new();
let mut cluster_spacing = 0.0;
let mut current_resource = resource.as_str();
for (i, glyph) in shaped.glyphs.iter().enumerate() {
let (name, code) = resources.get(i).ok_or_else(|| EditError::Invalid("shaped glyph resource is missing".into()))?;
let shown = shown_fonts.get(name).ok_or_else(|| EditError::Invalid("shaped font is missing".into()))?;
if name != current_resource {
if !shown_glyphs.is_empty() {
block_ops.push(Op::new("TJ", vec![Object::Array(std::mem::take(&mut shown_glyphs))]));
}
block_ops.push(Op::new("Tf", vec![Object::name(name), n(size)]));
current_resource = name;
}
if thai && !glyph.text.is_empty() {
// Apply spacing only after a whole cluster, keeping marks on their base.
extra += cluster_spacing;
cluster_spacing = char_spacing * glyph.text.chars().count() as f64 + word_spacing * glyph.text.matches(' ').count() as f64;
}
let target = (glyph.position[0] * size + extra) * per_unit;
let glyph_rise = rise + glyph.position[1] * size;
if (glyph_rise - raised).abs() > 1e-9 || shown_glyphs.len() >= 4096 {
if !shown_glyphs.is_empty() {
block_ops.push(Op::new("TJ", vec![Object::Array(std::mem::take(&mut shown_glyphs))]));
}
if (glyph_rise - raised).abs() > 1e-9 {
block_ops.push(Op::new("Ts", vec![n(glyph_rise)]));
raised = glyph_rise;
}
}
if (target - pen).abs() > 1e-6 {
shown_glyphs.push(n(pen - target));
}
shown_glyphs.push(Object::String(PdfString::literal(code.clone())));
let cid = code.iter().fold(0_u32, |value, byte| value.saturating_mul(256).saturating_add(u32::from(*byte)));
pen = target
+ if code.is_empty() {
0.0
} else {
shown.width(cid) * 1000.0 + (char_spacing + if shown.is_space(cid, code.len()) { word_spacing } else { 0.0 }) * per_unit
};
if !thai {
extra += char_spacing * glyph.text.chars().count() as f64 + word_spacing * glyph.text.matches(' ').count() as f64;
}
}
let end = (shaped.advance * size + extra + cluster_spacing) * per_unit;
if (end - pen).abs() > 1e-6 {
shown_glyphs.push(n(pen - end));
}
if !shown_glyphs.is_empty() {
block_ops.push(Op::new("TJ", vec![Object::Array(shown_glyphs)]));
}
if (raised - rise).abs() > 1e-9 {
block_ops.push(Op::new("Ts", vec![n(rise)]));
}
block_ops.push(Op::new("ET", vec![]));
(state.char_spacing, state.word_spacing) = (char_spacing, word_spacing);
extra += cluster_spacing;
block_ops.push(Op::new("EMC", vec![]));
Ok(shaped.advance * size + extra)
}
}
fn validate_contextual_rows(rows: &[Vec<StyledChar>], kept: &[bool]) -> Result<(), EditError> {
for (row, _) in rows.iter().zip(kept).filter(|(_, retained)| !**retained) {
let text: String = row.iter().map(|character| character.0).collect();
@@ -321,6 +575,23 @@ fn calibration_factors(
calibration
}
/// The calibration factors fit each word to its box at its own size (its `Tf` in its text space,
/// which a range's size doesn't change). An OCR phrase drawn at one size needs the inverse ratio
/// too, so every word still spans the box the scan shows it in. The ratio is not clamped again:
/// a word OCR sized at three times the phrase's needs three times the calibration.
fn fit_to_phrase(calibration: &mut HashMap<usize, f64>, lines: &[TextLine], phrase: Option<f64>) {
let Some(phrase) = phrase else { return };
for (index, factor) in calibration.iter_mut() {
let own = lines.get(*index).and_then(|source| Some(source.origin.state.font.as_ref()?.1 * source.origin.k));
if let Some(own) = own.filter(|own| own.is_finite() && *own > 0.0) {
let fitted = *factor * own / phrase;
if fitted.is_finite() {
*factor = fitted;
}
}
}
}
fn validated_ranges<'a>(draft: &str, style: &'a BlockStyle) -> Result<Vec<&'a RangeStyle>, EditError> {
let mut ranges: Vec<_> = style.ranges.iter().collect();
ranges.sort_by_key(|range| range.range.start);
@@ -578,35 +849,385 @@ fn apply_range_styles<'a>(
StyledLines { lines: styled_lines, spans: styled_spans, underlines, overrides }
}
/// Shape in the source or an explicitly registered matching full face. Never use interface fonts.
fn shape_source(
/// Select one existing subset of the exact source face for a Thai span. Never replace its
/// family/weight to make an edit succeed; missing glyphs need an explicit font choice. The
/// subsets' programs are read within `budget`, shared by the edit's spans.
fn legacy_thai_font(
doc: &Document,
fonts: &Dict,
name: &str,
text: &str,
) -> Result<Option<(Vec<u8>, pdfcraft_fonts::source_shaping::ShapedRun, String)>, EditError> {
source: &TextLine,
span: &str,
cache: &mut HashMap<String, LegacyFace>,
budget: &mut pdfcraft_cos::DecodeBudget,
) -> Result<Option<ThaiSpan>, EditError> {
if !source.base_font.rsplit('+').next().unwrap_or(&source.base_font).starts_with("THSarabunPSK")
|| !span.chars().any(|c| matches!(u32::from(c), 0x0e00..=0x0e7f))
{
return Ok(None);
}
let family = pdfcraft_fonts::embedding::face_key(&source.base_font);
let mut candidates = vec![source.font.clone()];
candidates.extend(fonts.iter().filter_map(|(name, font)| {
let font = doc.dict(font)?;
(pdfcraft_fonts::embedding::face_key(&Metrics::from_dict(doc, &font).base_font) == family).then(|| String::from_utf8_lossy(name).into_owned())
}));
for name in candidates.into_iter().take(128) {
if cache.contains_key(&name) {
continue;
}
let Some(resource) = fonts.get(name.as_bytes()) else { continue };
let Some(font) = doc.dict(resource) else { continue };
let object = resource.as_ref();
let face = match object.and_then(|o| cache.values().find(|face| face.object == Some(o))) {
Some(alias) => LegacyFace { glyphs: alias.glyphs.clone(), metrics: alias.metrics.clone(), object, ..*alias },
None => {
let metrics = Metrics::from_dict(doc, &font);
let space = source_space(doc, fonts, &metrics);
let glyphs = pdfcraft_fonts::pdf::LegacyThaiFont::from_pdf(doc, &font, space, budget).ok().flatten().map(std::rc::Rc::new);
let style = FontStyle::of(doc, &font, &metrics);
LegacyFace { glyphs, metrics, style, object, space }
}
};
cache.insert(name, face);
}
// A subset left unread would count as unusable, silently changing which subset draws the text.
if budget.remaining() == 0 {
return Err(EditError::Invalid(format!(
"the page's {} subsets need more than {} MiB of font programs for one edit; choose a replacement font for this text",
source.base_font.rsplit('+').next().unwrap_or(&source.base_font),
MAX_THAI_PROGRAMS >> 20
)));
}
// A face none of whose subsets can be shaped in place (no Thai, or neither positioned nor in
// the legacy glyph order) is shaped like any other source font (`shape_source`).
if !has_thai_source_face(cache, &family) {
return Ok(None);
}
let rows = || span.split(['\n', '\r']).filter(|row| !row.is_empty());
for row in rows() {
if let Err(error) = shape_existing_thai(cache, &source.font, row) {
// Not in the face's existing codes. A complete program of the face that shapes every
// row with its own layout (one Load font added as CFF, or whose contextual glyphs it
// gave no code yet) takes the whole span, so no cluster is split between fonts.
let Some(source_font) = fonts.get(source.font.as_bytes()).and_then(|f| doc.dict(f)) else { return Err(error) };
let complete = program_candidates(doc, fonts, &source.font, source_font, &source.base_font)
.into_iter()
.find(|(_, font)| rows().all(|row| shape_with_program(doc, font, row).is_some()));
return match complete {
Some((name, _)) => Ok(Some(ThaiSpan::Program(name))),
None => Err(error),
};
}
}
Ok(Some(ThaiSpan::InPlace(source.font.clone())))
}
/// Whether a subset of the face `family` (a [`face_key`](pdfcraft_fonts::embedding::face_key))
/// in `cache` can be shaped in place: a THSarabunPSK program with its own Thai mark positioning,
/// or a verified legacy positional one.
fn has_thai_source_face(cache: &HashMap<String, LegacyFace>, family: &str) -> bool {
cache.values().any(|face| face.glyphs.is_some() && pdfcraft_fonts::embedding::face_key(&face.metrics.base_font) == family)
}
/// A visual row may use several subsets of the same exact face (often a separate subset for
/// punctuation). Shape Thai clusters intact, and retain each glyph's original PDF font/code.
fn shape_existing_thai(cache: &HashMap<String, LegacyFace>, primary: &str, text: &str) -> Result<ShapedText, EditError> {
if text.len() > 262_144 {
return Err(EditError::Invalid("the Thai source row exceeds its shaping budget".into()));
}
let first = cache.get(primary).ok_or_else(|| EditError::Invalid("the Thai source face is missing".into()))?;
let family = pdfcraft_fonts::embedding::face_key(&first.metrics.base_font);
let mut candidates = vec![(primary, first)];
candidates.extend(
cache
.iter()
.filter(|(name, face)| {
name.as_str() != primary && face.style == first.style && pdfcraft_fonts::embedding::face_key(&face.metrics.base_font) == family
})
.map(|(name, face)| (name.as_str(), face)),
);
// Stable source-first choice, then resource name; never depend on hash iteration order.
if let Some(tail) = candidates.get_mut(1..) {
tail.sort_by_key(|(name, _)| *name);
}
let thai = |c: char| matches!(u32::from(c), 0x0e01..=0x0e5b | 0xf700..=0xf71a);
let mut run = pdfcraft_fonts::source_shaping::ShapedRun { glyphs: Vec::new(), advance: 0.0 };
let mut resources = Vec::new();
let mut chars = text.char_indices().peekable();
while let Some((at, ch)) = chars.next() {
if thai(ch) {
let mut end = at + ch.len_utf8();
while let Some(&(next, c)) = chars.peek() {
if !thai(c) {
break;
}
chars.next();
end = next + c.len_utf8();
}
let segment = text.get(at..end).ok_or_else(|| EditError::Invalid("invalid Thai character boundary".into()))?;
let unshaped = |why: String| {
EditError::Invalid(format!(
"{} cannot keep this edit in the original face: {}; load the complete original font or explicitly choose a replacement",
first.metrics.base_font,
if why.is_empty() { "its source glyph program is incomplete" } else { &why }
))
};
// The whole run in one subset keeps any kerning between its clusters. A subsetter may
// instead have split the run's glyphs across subsets of the face: then each cluster
// (a base and the marks on it, which shape together) comes from a subset that has it.
let pieces = match shape_in_one_subset(&candidates, segment) {
Ok(whole) => vec![whole],
Err(_) => thai_clusters(segment)
.into_iter()
.map(|cluster| shape_in_one_subset(&candidates, cluster).map_err(unshaped))
.collect::<Result<Vec<_>, _>>()?,
};
for (name, shaped) in pieces {
for (mut glyph, code) in shaped.shaped.glyphs.into_iter().zip(shaped.codes) {
glyph.position[0] += run.advance;
resources.push((name.to_owned(), code));
run.glyphs.push(glyph);
}
run.advance += shaped.shaped.advance;
}
} else {
let logical = ch.to_string();
let found = candidates.iter().find_map(|(name, face)| {
let code = face.metrics.encode(&logical)?;
let first = face.metrics.codes(&code).first()?.0;
// `encode` accepts a CID the subset emptied (#1443): drawing it would show nothing.
// Try the next subset; a space then falls back to its measured gap below.
if face.metrics.has_outline(first) == Some(false) {
return None;
}
Some((*name, code, face.metrics.width(first)))
});
let (name, code, width) = if let Some(found) = found {
found
} else if ch == ' ' {
// The source subset may omit spaces. Use a gap measured from a matching
// document subset, then the face's existing text-space fallback.
let width = candidates
.iter()
.find_map(|(_, f)| {
let code = f.metrics.encode(" ")?;
Some(f.metrics.width(f.metrics.codes(&code).first()?.0))
})
.unwrap_or(first.space);
(primary, Vec::new(), width)
} else {
return Err(EditError::Invalid(format!(
"{} has no source glyph for {ch} (U+{:04X}); load the complete original face or explicitly select a replacement",
first.metrics.base_font,
u32::from(ch)
)));
};
run.glyphs.push(pdfcraft_fonts::source_shaping::ShapedGlyph { id: Default::default(), position: [run.advance, 0.0], text: logical });
resources.push((name.to_owned(), code));
run.advance += width;
}
if run.glyphs.len() > 65_534 || !run.advance.is_finite() {
return Err(EditError::Invalid("the Thai source row exceeds its shaping budget".into()));
}
}
Ok(ShapedText { run, font: ShapedFont::Existing(resources) })
}
/// Shape `text` in the first candidate subset that has every glyph it needs, and name it. The
/// error is the first subset's reason (empty when no candidate has a usable glyph program).
fn shape_in_one_subset<'a>(candidates: &[(&'a str, &LegacyFace)], text: &str) -> Result<(&'a str, pdfcraft_fonts::pdf::ExistingThaiRun), String> {
let mut why = String::new();
for (name, face) in candidates {
let Some(font) = &face.glyphs else { continue };
match font.shape(text) {
Ok(shaped) => return Ok((name, shaped)),
Err(e) if why.is_empty() => why = e,
Err(_) => {}
}
}
Err(why)
}
/// `text` (a run of Thai characters) cut before each character that doesn't attach to the one
/// before it: a cluster is a base or leading vowel with the vowels and tone marks above and below
/// it, and Sara Am, whose Nikhahit sits on the base. Legacy presentation forms (U+F700–U+F71A)
/// are kept with what precedes them.
fn thai_clusters(text: &str) -> Vec<&str> {
let attaches = |c: char| matches!(u32::from(c), 0x0e31 | 0x0e33..=0x0e3a | 0x0e47..=0x0e4e | 0xf700..=0xf71a);
let mut starts: Vec<usize> = text.char_indices().filter(|(at, c)| *at == 0 || !attaches(*c)).map(|(at, _)| at).collect();
starts.push(text.len());
// Each boundary is a `char_indices` position or the end, strictly increasing: every `get`
// succeeds and no cluster is empty.
starts.windows(2).filter_map(|w| text.get(*w.first()?..*w.get(1)?)).collect()
}
/// Whether the approved Thai face can replace this source, whose embedded subset cannot
/// shape the replacement. A usable complete source program always takes precedence.
fn matching_thai_outline(doc: &Document, fonts: &Dict, source: &TextLine, span: &str) -> bool {
let Some(face) = pdfcraft_fonts::document_thai_font() else { return false };
// The bundled face is Regular and upright. A source its descriptor or name marks bold or
// italic keeps that style, so completing it with these outlines would be a silent change:
// it needs an explicit choice instead. (Variable-font instances often keep the family name.)
let style = fonts
.get(source.font.as_bytes())
.and_then(|f| doc.dict(f))
.map_or(FontStyle { bold: source.bold, italic: source.italic }, |f| FontStyle::of(doc, &f, &Metrics::from_dict(doc, &f)));
if style.bold || style.italic {
return false;
}
if !span.chars().any(|c| matches!(u32::from(c), 0x0e00..=0x0e7f))
|| !pdfcraft_fonts::embedding::matching_faces(&source.base_font).iter().any(|f| std::ptr::eq(*f, face))
|| !pdfcraft_fonts::embedding::covers(face.bytes, span)
{
return false;
}
// A registered complete matching program also completes the source subset.
// Do not replace it with bundled outlines merely because the original resource is a subset.
// Only one in the source's style does (`program_candidates`), so only that one counts here.
let family = pdfcraft_fonts::embedding::face_key(&source.base_font);
!fonts.iter().filter_map(|(_, font)| doc.dict(font)).any(|font| {
let metrics = Metrics::from_dict(doc, &font);
pdfcraft_fonts::embedding::face_key(&metrics.base_font) == family
&& FontStyle::of(doc, &font, &metrics) == style
&& pdfcraft_fonts::pdf::embedded_sfnt(doc, &font).is_some_and(|bytes| {
pdfcraft_fonts::embedding::validate(&bytes).is_ok()
&& pdfcraft_fonts::source_shaping::has_thai_layout(&bytes)
&& pdfcraft_fonts::embedding::covers(&bytes, span)
&& span.split(['\n', '\r']).all(|row| pdfcraft_fonts::source_shaping::shape(&bytes, row).is_ok())
})
})
}
/// Use the same Thai script selection and mark/cluster positioning as Add Text, including
/// rows beginning with Latin or digits (guessing from their first character would select Latin).
fn shape_thai_row(text: &str) -> Result<pdfcraft_fonts::source_shaping::ShapedRun, EditError> {
if text.len() > 262_144 || text.contains(['\n', '\r']) {
return Err(EditError::Invalid("shape one bounded text row at a time".into()));
}
let clusters =
pdfcraft_fonts::shape_thai(text).map_err(|e| EditError::Invalid(format!("the matching Thai font cannot shape this text ({e:?})")))?;
let mut glyphs = Vec::new();
let mut advance = 0.0;
for cluster in clusters {
for (i, (id, [x, y])) in cluster.glyphs.into_iter().enumerate() {
if glyphs.len() >= 65_534 {
return Err(EditError::Invalid("shaped row exceeds its geometry budget".into()));
}
glyphs.push(pdfcraft_fonts::source_shaping::ShapedGlyph {
id: id.into(),
position: [advance + x, y],
text: if i == 0 { cluster.text.clone() } else { String::new() },
});
}
advance += cluster.advance;
}
if !advance.is_finite() {
return Err(EditError::Invalid("shaped row exceeds its geometry budget".into()));
}
Ok(pdfcraft_fonts::source_shaping::ShapedRun { glyphs, advance })
}
/// The resources whose complete program may shape text in the face of `base_font`: `source`
/// (the resource `name`) first, then the page's other fonts of that face (at most 128).
fn program_candidates(doc: &Document, fonts: &Dict, name: &str, source: Dict, base_font: &str) -> Vec<(String, Dict)> {
let key = pdfcraft_fonts::embedding::face_key(base_font);
let style = FontStyle::of(doc, &source, &Metrics::from_dict(doc, &source));
let mut candidates = vec![(name.to_owned(), source)];
candidates.extend(
fonts
.iter()
.filter_map(|(n, f)| Some((String::from_utf8_lossy(n).into_owned(), doc.dict(f)?)))
.filter(|(_, f)| {
let m = Metrics::from_dict(doc, f);
pdfcraft_fonts::embedding::face_key(&m.base_font) == key && FontStyle::of(doc, f, &m) == style
})
.take(128),
);
candidates
}
/// A font's weight and slant, for matching subsets and complete programs of one face: a Regular
/// program never completes text that a descriptor marks bold, say.
#[derive(Clone, Copy, Debug, PartialEq)]
struct FontStyle {
bold: bool,
italic: bool,
}
impl FontStyle {
/// Bold from the name or the descriptor's weight ([`Metrics`]); italic from the name or a
/// non-zero `/ItalicAngle`. The descriptor's Italic flag alone isn't trusted: exporters set
/// it on upright faces (a THSarabunPSK subset with `/Flags 96` and `/ItalicAngle 0`).
fn of(doc: &Document, font: &Dict, metrics: &Metrics) -> Self {
let descendant =
font.get(b"DescendantFonts").and_then(|d| doc.resolve(d).as_array().and_then(|a| a.first()).cloned()).and_then(|d| doc.dict(&d));
let descriptor = descendant.as_ref().unwrap_or(font).get(b"FontDescriptor").and_then(|d| doc.dict(d));
let angle = descriptor.as_ref().and_then(|d| d.get(b"ItalicAngle")).and_then(|a| doc.resolve(a).as_f64()).unwrap_or(0.0);
let name = metrics.base_font.to_ascii_lowercase();
Self { bold: metrics.bold, italic: name.contains("italic") || name.contains("oblique") || (angle.is_finite() && angle.abs() > 0.1) }
}
}
/// `text` shaped by `font`'s embedded program with its own layout, when the program covers it and
/// may be embedded again (`embed_mapping` refuses one that may not, so a later program is tried).
fn shape_with_program(doc: &Document, font: &Dict, text: &str) -> Option<(Vec<u8>, pdfcraft_fonts::source_shaping::ShapedRun)> {
let bytes = pdfcraft_fonts::pdf::embedded_sfnt(doc, font)?;
if pdfcraft_fonts::embedding::validate(&bytes).is_err() || !pdfcraft_fonts::embedding::covers(&bytes, text) {
return None;
}
let shaped = pdfcraft_fonts::source_shaping::shape(&bytes, text).ok()?;
Some((bytes, shaped))
}
/// Shape in the source or an explicitly registered matching full face. Never use interface fonts.
fn shape_source(doc: &Document, fonts: &Dict, thai: &ThaiShaping, name: &str, text: &str) -> Result<Option<ShapedText>, EditError> {
if thai.outlined.contains(name) {
let shaped = shape_thai_row(text)?;
return Ok(Some(ShapedText { run: shaped, font: ShapedFont::ThaiOutlines }));
}
if let Some(resource) = thai.program.get(name) {
let font = fonts
.get(resource.as_bytes())
.and_then(|f| doc.dict(f))
.ok_or_else(|| EditError::Invalid("the complete font resource is missing".into()))?;
let family = Metrics::from_dict(doc, &font).base_font;
let (bytes, shaped) = shape_with_program(doc, &font, text)
.ok_or_else(|| EditError::Invalid(format!("{family} cannot shape this text with its own layout; choose a font that covers it")))?;
return Ok(Some(ShapedText { run: shaped, font: ShapedFont::Program { bytes, family } }));
}
if !pdfcraft_fonts::source_shaping::required(text) {
return Ok(None);
}
let source =
fonts.get(name.as_bytes()).and_then(|f| doc.dict(f)).ok_or_else(|| EditError::Invalid("the shaped text has no font resource".into()))?;
let family = Metrics::from_dict(doc, &source).base_font;
let mut candidates = vec![source];
candidates.extend(
fonts
.iter()
.filter_map(|(_, f)| doc.dict(f))
.filter(|f| pdfcraft_fonts::embedding::face_key(&Metrics::from_dict(doc, f).base_font) == pdfcraft_fonts::embedding::face_key(&family)),
);
for font in candidates {
if let Some(bytes) = pdfcraft_fonts::pdf::embedded_sfnt(doc, &font)
&& pdfcraft_fonts::embedding::covers(&bytes, text)
&& let Ok(shaped) = pdfcraft_fonts::source_shaping::shape(&bytes, text)
{
return Ok(Some((bytes, shaped, family)));
// In the face's existing codes first; what they can't show, a complete program of the face
// may (`legacy_thai_font` already gave the span to one that can).
let in_place = (family.rsplit('+').next().unwrap_or(&family).starts_with("THSarabunPSK")
&& text.chars().any(|c| matches!(u32::from(c), 0x0e00..=0x0e7f))
&& has_thai_source_face(&thai.legacy, &pdfcraft_fonts::embedding::face_key(&family)))
.then(|| shape_existing_thai(&thai.legacy, name, text));
let in_place = match in_place {
Some(Ok(shaped)) => return Ok(Some(shaped)),
Some(Err(error)) => Some(error),
None => None,
};
for (_, font) in program_candidates(doc, fonts, name, source, &family) {
if let Some((bytes, shaped)) = shape_with_program(doc, &font, text) {
return Ok(Some(ShapedText { run: shaped, font: ShapedFont::Program { bytes, family } }));
}
}
Err(EditError::Invalid("complex text needs the complete source TrueType/OpenType face; load that font before editing".into()))
if let Some(error) = in_place {
return Err(error);
}
let fallback = pdfcraft_fonts::document_thai_font()
.filter(|_| text.chars().any(|c| matches!(u32::from(c), 0x0e00..=0x0e7f)))
.map(|face| format!("; or select {} in the document-font menu to replace it", face.family))
.unwrap_or_default();
Err(EditError::Invalid(format!(
"complex text in {} needs its complete source TrueType/OpenType font; use Load font to keep it{fallback}",
family.rsplit('+').next().unwrap_or(&family)
)))
}
/// For each output row (a range of the draft's characters), whether it is exactly the source row
@@ -889,10 +1510,12 @@ pub(crate) fn rewrite_styled_block(
max_glyphs,
}
.prepare()?;
let thai = prepared.thai;
let chars = prepared.values;
let substituted = prepared.substituted;
let phrase_size = block.phrase_size();
let size_of = |source: &TextLine, index: usize| {
overrides.get(&index).and_then(|r| r.size).or(style.size).unwrap_or(source.size).max(0.1) / source.origin.k.max(1e-6)
overrides.get(&index).and_then(|r| r.size).or(style.size).or(phrase_size).unwrap_or(source.size).max(0.1) / source.origin.k.max(1e-6)
};
// A source run may carry TJ kerning/justification that is absent from its text state.
// Keep its measured advance when rebuilding the run instead of making it wrap on entry.
@@ -910,7 +1533,8 @@ pub(crate) fn rewrite_styled_block(
spaces.borrow_mut().insert(name.to_owned(), w);
w
};
let calibration = calibration_factors(lines, &metrics, style, &overrides, space_of);
let mut calibration = calibration_factors(lines, &metrics, style, &overrides, space_of);
fit_to_phrase(&mut calibration, lines, phrase_size);
let advance = |item: &(char, usize, String)| -> f64 {
let source = lines.get(item.1).unwrap_or(first);
let Some(m) = metrics.get(&item.2) else { return 0.0 };
@@ -939,7 +1563,8 @@ pub(crate) fn rewrite_styled_block(
}
let segment = items.get(at..end).unwrap_or_default();
let string: String = segment.iter().map(|v| v.0).collect();
if let Some((_, shaped, _)) = shape_source(doc, &fonts, &item.2, &string)? {
if let Some(shaped) = shape_source(doc, &fonts, &thai, &item.2, &string)? {
let shaped = &shaped.run;
let source = lines.get(item.1).unwrap_or(first);
let scale =
overrides.get(&item.1).and_then(|r| r.scale).or(style.scale).map_or(source.origin.state.scale, |s| (s / 100.0).clamp(0.1, 10.0));
@@ -962,7 +1587,7 @@ pub(crate) fn rewrite_styled_block(
};
let (rows, row_ranges, row_lines, row_soft, row_gap) = wrap_styled_rows(&chars, width, measure)?;
let streams = content_streams(doc, &p.dict)?;
let pieces = Pieces::join(&streams.iter().map(|(_, d)| d.as_slice()).collect::<Vec<_>>());
let pieces = crate::text::joined(&streams);
let ops = pieces.parse().ops;
// A row the edit doesn't reach keeps its source operators, so its kerning, TJ word gaps and
// glyph widths stay exactly as they were instead of being rebuilt from the text state. A row
@@ -990,7 +1615,7 @@ pub(crate) fn rewrite_styled_block(
let at = if split { start } else { o.bt_op };
validate_contextual_rows(&rows, &kept)?;
let measured: Vec<_> = rows.iter().map(|row| measure(row)).collect::<Result<_, _>>()?;
let shaped = prepare_shaped_rows(doc, &fonts, &rows, &kept, advance)?;
let shaped = prepare_shaped_rows(doc, &fonts, &thai, &rows, &kept, advance)?;
let ShapedRows { runs: shaped_runs, advances: segment_advances } = shaped;
let glyph_resources = embed_shaped_resources(doc, &mut fonts, &shaped_runs)?;
let n = pdfcraft_content::num;
@@ -1100,67 +1725,19 @@ pub(crate) fn rewrite_styled_block(
// drawn in: a `gs` has no inverse, so one run's ExtGState would stay for the next.
block_ops.push(Op::new("q", vec![]));
block_ops.extend(state.paint_ops(drawn_from, run_ctm));
if let Some((_, shaped, _)) = shaped_runs.get(&(row_index, pos)) {
let resource = glyph_resources.get(&(row_index, pos)).ok_or_else(|| EditError::Invalid("shaped font is missing".into()))?;
let mut properties = Dict::new();
properties.set(b"ActualText".to_vec(), Object::String(PdfString::text(&string)));
block_ops.push(Op::new("BDC", vec![Object::name("Span"), Object::Dict(properties)]));
state.font = Some((resource.clone(), size_of(source, item.1)));
// One text object for the run: each glyph is placed where shaping put it by a TJ
// adjustment from where the glyph before it ends (its advance in the embedded
// font's widths), and raised by the text rise for a mark set above or below
// (ISO 32000-1 §9.4.3, §9.3.7). Font-program codes are separate from Unicode;
// ToUnicode and ActualText keep the logical text.
let size = size_of(source, item.1);
let per_unit = 1000.0 / size;
if !per_unit.is_finite() {
return Err(EditError::Invalid("the shaped text has no size".into()));
}
let shown = fonts
.get(resource.as_bytes())
.and_then(|f| doc.resolve(f).as_dict().cloned())
.map(|d| Metrics::from_dict(doc, &d))
.ok_or_else(|| EditError::Invalid("shaped font is missing".into()))?;
// Spacing is part of each glyph's position, so the run itself has none.
let (char_spacing, word_spacing, rise) = (state.char_spacing, state.word_spacing, state.rise);
state.char_spacing = 0.0;
state.word_spacing = 0.0;
block_ops.push(Op::new("BT", vec![]));
block_ops.extend(state.ops());
block_ops.push(Op::new("Tm", tm.iter().map(|v| n(*v)).collect()));
// `pen`: where the next glyph would start, in thousandths of the font size.
let (mut pen, mut extra, mut raised) = (0.0, 0.0, rise);
let mut shown_glyphs: Vec<Object> = Vec::new();
for (i, glyph) in shaped.glyphs.iter().enumerate() {
let target = (glyph.position[0] * size + extra) * per_unit;
let glyph_rise = rise + glyph.position[1] * size;
if (glyph_rise - raised).abs() > 1e-9 || shown_glyphs.len() >= 4096 {
if !shown_glyphs.is_empty() {
block_ops.push(Op::new("TJ", vec![Object::Array(std::mem::take(&mut shown_glyphs))]));
}
if (glyph_rise - raised).abs() > 1e-9 {
block_ops.push(Op::new("Ts", vec![n(glyph_rise)]));
raised = glyph_rise;
}
}
if (target - pen).abs() > 1e-6 {
shown_glyphs.push(n(pen - target));
}
let code = u16::try_from(i + 1).map_err(|_| EditError::Invalid("shaped glyph limit".into()))?;
shown_glyphs.push(Object::String(PdfString::literal(code.to_be_bytes().to_vec())));
pen = target + shown.width(u32::from(code)) * 1000.0;
extra += char_spacing * glyph.text.chars().count() as f64 + word_spacing * glyph.text.matches(' ').count() as f64;
}
if !shown_glyphs.is_empty() {
block_ops.push(Op::new("TJ", vec![Object::Array(shown_glyphs)]));
}
if (raised - rise).abs() > 1e-9 {
block_ops.push(Op::new("Ts", vec![n(rise)]));
}
block_ops.push(Op::new("ET", vec![]));
(state.char_spacing, state.word_spacing) = (char_spacing, word_spacing);
block_ops.push(Op::new("EMC", vec![]));
segment_width = (shaped.advance * size + extra) * state.scale * source.origin.k;
if let Some(shaped) = shaped_runs.get(&(row_index, pos)) {
let resources = glyph_resources.get(&(row_index, pos)).ok_or_else(|| EditError::Invalid("shaped font is missing".into()))?;
let drawing = ShapedRunDrawing {
doc,
fonts: &fonts,
shaped: &shaped.run,
resources,
text: &string,
matrix: tm,
size: size_of(source, item.1),
thai: string.chars().any(|c| matches!(u32::from(c), 0x0e00..=0x0e7f)),
};
segment_width = drawing.draw(&mut block_ops, &mut state)? * state.scale * source.origin.k;
} else {
block_ops.push(Op::new("BT", vec![]));
block_ops.extend(state.ops());
@@ -1206,7 +1783,7 @@ pub(crate) fn rewrite_styled_block(
// when the rewrite splits one, and a kept row's operators come later in the stream.
let marker = if marked { None } else { paragraph_marker(block, lines, &ops) };
let data = splice_with_marker(&pieces, &ops, marker, at, block_ops, &drop);
let contents = rewritten_contents(doc, &streams, data);
let contents = rewritten_contents(doc, &streams, data)?;
res.set(b"Font".to_vec(), Object::Dict(fonts));
doc.update_dict(p.obj, |d| {
d.set(b"Contents".to_vec(), contents);
crates/edit/src/ungroup.rs
+743
View File
@@ -0,0 +1,743 @@
//! Edit a PDF ▸ Ungroup: draw a Form XObject's content in place of its `Do` (#2964, #3005).
//!
//! Text inside grouped Form artwork can't be edited, because editing rewrites only the page's own
//! content streams. Ungrouping copies the form's content into the page at the same place,
//! wrapped as `q <Matrix> cm <BBox> re W n … Q`, with the form's resources merged into a copy of
//! the page's under names that don't clash. Its text then edits like any other. The form object
//! itself is left alone, so another page or placement that draws it is unchanged. Each resource
//! name is read from the operand the renderer uses ([`resource_operand`]), so a surplus operand
//! before it changes neither the renaming nor the checks below.
//!
//! Drawing a form inline paints the same pixels unless the form is a transparency group whose
//! compositing differs from painting its parts one by one: a knockout group, a group that uses
//! a blend mode other than Normal inside, or a group drawn translucent, blended or under a soft
//! mask, or a group with a blending colour space (`/CS`) other than the page's: its contents
//! composite in that space and are then converted (ISO 32000-2 §11.6.6), inline they would
//! composite in the page's. Those are refused rather than changed, as is a form with resources
//! of its own whose `/DefaultGray`, `/DefaultRGB` or `/DefaultCMYK` differs from the page's, one
//! having it and the other not included (it remaps the device colours painted with it,
//! §8.6.5.6, and inline the page's resources apply), tagged form content (its structure link
//! would be lost) and content whose marked content or text objects don't balance (inlined, they
//! would change the scope of the page's own). A form in a layer (`/OC`) keeps its membership as `/OC … BDC … EMC`.
//!
//! A pattern's matrix maps pattern space to the default space of the content stream that uses it
//! (ISO 32000-2 §8.7.2): inside the form that is the form's space at the `Do`, on the page it is
//! the page's. So each pattern the form selects with `scn`/`SCN` is drawn from a copy whose
//! `/Matrix` also carries the form's placement; one that can't be copied that way is refused.
//!
//! Edit text's preparation of grouped text (`text::forms`) opens each placement with this same
//! code, and decides what to open from what [`Refusal`] and [`Ungrouped::shows_text`] say.
use std::collections::HashMap;
use pdfcraft_content::{Matrix, Op, parse};
use pdfcraft_cos::{DecodeBudget, Dict, Document, Object};
use crate::EditError;
/// The largest form content ungrouped (decoded bytes): the page's own content cap is far above
/// any real form, and a bomb must not be copied into the page. Ungroup reads the form and what it
/// draws within one [`DecodeBudget`] of this size.
pub(crate) const MAX_FORM_BYTES: usize = 64 << 20;
/// Resource categories a content stream refers to by name (ISO 32000-2 §7.8.3, Table 34).
const CATEGORIES: [&[u8]; 7] = [b"Font", b"XObject", b"ExtGState", b"ColorSpace", b"Pattern", b"Shading", b"Properties"];
/// The `/ColorSpace` entries that remap device colours painted with the resources they are in
/// (ISO 32000-2 §8.6.5.6).
const DEFAULT_SPACES: [&[u8]; 3] = [b"DefaultGray", b"DefaultRGB", b"DefaultCMYK"];
/// The most distinct patterns one ungrouped form may select; each becomes a new object.
const MAX_MOVED_PATTERNS: usize = 256;
/// The most entries of one resource category merged: a form or page with more is refused.
const MAX_RESOURCES: usize = 4096;
/// The deepest `q` nesting a form's content may have.
const MAX_SAVES: usize = 4096;
/// A name map for one resource category: form name → name in the page's resources.
type Renames = Vec<(Vec<u8>, Vec<u8>)>;
/// What [`ungroup_ops`] gives back.
pub(crate) struct Ungrouped {
/// The ops that draw the form inline.
pub(crate) ops: Vec<Op>,
/// The page resources with the form's merged in.
pub(crate) resources: Dict,
/// The pattern copies to add to the document, each under the name in the merged `/Pattern`
/// dictionary that the ops already use.
pub(crate) patterns: Vec<(Vec<u8>, Object)>,
/// Whether the form shows text, in its own content or in a Form XObject it draws (at any
/// depth [`MAX_NESTING`] allows). Text inside a tiling pattern or Type 3 glyph doesn't count:
/// inlining the form doesn't make it editable. `None` when something it draws can't be read,
/// is nested too deep, draws itself, or is past the reads or the decode budget allowed.
pub(crate) shows_text: Option<bool>,
}
/// Why [`ungroup_ops`] left a placement grouped.
pub(crate) struct Refusal {
pub(crate) error: EditError,
/// The form itself is the reason (malformed, tagged, a knockout group, and so on), so every
/// placement of it is refused the same way. Otherwise something about this placement or the
/// page so far is: its translucency or matrix, the resources it would merge into (which grow
/// as other groups are prepared), or the decode budget left.
pub(crate) intrinsic: bool,
/// The form's resources need renaming where it is drawn, and it draws something that would
/// look them up by their old names. Names merged in for other forms can make the renaming
/// unnecessary, which [`merges_without_renaming`] checks without reading the content.
pub(crate) renaming: bool,
}
/// Ungroup the form drawn by the `Do` at `page_ops[at]` (the page's streams parsed as one, the
/// indexes `page_images` gives) from the page's `resources`. `page_space` is the page group's
/// blending colour space ([`page_blend_space`]). `placement` maps the form's space to the page's
/// default space at the `Do` (its `/Matrix`, then the CTM there). The form, and the forms and
/// patterns it draws, decode within `budget`.
pub(crate) fn ungroup_ops(
doc: &Document,
resources: &Dict,
page_ops: &[Op],
at: usize,
page_space: Option<&Object>,
placement: Matrix,
budget: &mut DecodeBudget,
) -> Result<Ungrouped, Refusal> {
let refusal = |why: &str, intrinsic: bool| Refusal {
error: EditError::Invalid(format!("this artwork can't be ungrouped: {why}")),
intrinsic,
renaming: false,
};
// Something about this placement or the page so far is the reason.
let refuse = |why: &str| refusal(why, false);
// The form itself is the reason: every placement of it is refused the same way.
let invalid = |why: &str| refusal(why, true);
let op = page_ops.get(at).filter(|op| op.is("Do")).ok_or_else(|| refuse("it is missing"))?;
// The renderer draws a `Do`'s last operand, and the placement was read for its first
// (`page_images`): with more than one they can be different forms.
if op.operands.len() != 1 {
return Err(refuse("it is drawn with an operand too many"));
}
let name = op.name(0).ok_or_else(|| refuse("it is missing"))?;
let (translucent, open) = state_at(doc, resources, page_ops, at);
// Inline, its clip would take in the page's unfinished path and its text objects would be
// nested in the page's.
if open {
return Err(refuse("it is drawn inside an unfinished path or text object"));
}
let xobjects = resources.get(b"XObject").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()).ok_or_else(|| refuse("it is missing"))?;
let form = xobjects.get(name).ok_or_else(|| refuse("it is missing"))?;
let Object::Stream(s) = &*doc.resolve(form) else { return Err(refuse("it is not a Form XObject")) };
if !is_name(doc, &s.dict, b"Subtype", b"Form") {
return Err(refuse("it is not a Form XObject"));
}
let numbers = |key: &[u8], n: usize| -> Option<Vec<f64>> {
let a = s.dict.get(key).map(|o| doc.resolve(o))?;
let a = a.as_array()?;
// Every entry must be a number: a short array padded out by a non-number is unreadable.
let v: Vec<f64> = a.iter().filter_map(|x| doc.resolve(x).as_f64()).collect();
(a.len() == n && v.len() == n && v.iter().all(|x| x.is_finite())).then_some(v)
};
let bbox = numbers(b"BBox", 4).and_then(|b| <[f64; 4]>::try_from(b).ok()).ok_or_else(|| invalid("its bounding box is missing or unreadable"))?;
let matrix = match s.dict.get(b"Matrix") {
None => None,
Some(_) => Some(Matrix(numbers(b"Matrix", 6).and_then(|m| <[f64; 6]>::try_from(m).ok()).ok_or_else(|| invalid("its matrix is unreadable"))?)),
};
let own = s.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned());
if s.dict.get(b"Resources").is_some() && own.is_none() {
return Err(invalid("its resources can't be read"));
}
// A form without its own resources uses the ones it is drawn with (PDF 1.1).
let form_res = own.clone().unwrap_or_else(|| resources.clone());
// A tagged form's content is a structure element's: drawn inline it would lose that link.
if s.dict.get(b"StructParent").is_some() || s.dict.get(b"StructParents").is_some() {
return Err(invalid("it is tagged content (Tags panel); edit its text in a copy without tags"));
}
let group_dict = s.dict.get(b"Group").map(|g| doc.resolve(g)).and_then(|g| g.as_dict().cloned());
if s.dict.get(b"Group").is_some() && group_dict.is_none() {
return Err(invalid("its group dictionary can't be read"));
}
let group = group_dict.filter(|g| is_name(doc, g, b"S", b"Transparency"));
if let Some(group) = &group {
if group.get(b"K").is_some_and(|k| matches!(*doc.resolve(k), Object::Bool(true))) {
return Err(invalid("it is a knockout transparency group"));
}
if translucent {
return Err(refuse("it is a transparency group drawn translucent, blended or under a soft mask"));
}
// Its contents composite in its own colour space (ISO 32000-2 §11.6.6, Table 145 /CS);
// inline they would composite in the page's.
if let Some(cs) = group.get(b"CS")
&& !same_blend_space(doc, cs, page_space)
{
return Err(invalid("it is a transparency group that blends in a colour space of its own"));
}
}
// Strictly: damaged data must not be copied in as a shorter form.
let data = budget.decode_strict(doc, s).map_err(|e| refuse(&format!("its content can't be read ({e})")))?;
let parsed = parse(&data);
if parsed.skipped > 0 {
return Err(invalid("its content has parts that can't be read"));
}
let mut ops = parsed.ops;
// What it draws is read for text whether it is a group or not, and for blend modes only in a
// group: without one, blending inside draws the same inline. The form itself is being read,
// so drawing itself again is content that can't be checked.
let mut walk =
BlendWalk { path: form.as_ref().into_iter().collect(), known: HashMap::new(), reads: 0, budget, text: false, group: group.is_some() };
let unchecked = blends_inside(doc, &form_res, &ops, 0, &mut walk);
if unchecked && group.is_some() {
return Err(refuse("it is a transparency group with blend modes inside, or content inside that can't be checked"));
}
let shows_text = (!unchecked).then_some(walk.text);
// A form's default colour space remaps its device colours (ISO 32000-2 §8.6.5.6); inline,
// the page's resources are the current ones (§7.8.3).
if own.is_some()
&& let Some((key, form_has)) = unshared_default_space(doc, &form_res, resources)
{
let key = String::from_utf8_lossy(key);
return Err(refuse(&if form_has {
format!("it remaps its device colours with a /{key} the page doesn't share")
} else {
format!("the page's /{key} would remap its device colours, which it draws as they are")
}));
}
// Merge the form's resources into a copy of the page's, renaming what clashes.
let mut merged = resources.clone();
let mut renames: Vec<(&[u8], Renames)> = Vec::new();
if own.is_some() {
for cat in CATEGORIES {
let Some(theirs) = form_res.get(cat).map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()) else { continue };
if theirs.len() > MAX_RESOURCES {
return Err(invalid("too many resources"));
}
let mut mine = merged.get(cat).map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()).unwrap_or_default();
if mine.len() > MAX_RESOURCES {
return Err(refuse("too many resources"));
}
let mut map = Renames::new();
let mut n = 0u32;
for (key, value) in theirs.iter() {
// A default colour space checked above is the page's, which applies inline;
// renamed, it would be an entry nothing uses.
if mine.get(key) == Some(value) || (cat == b"ColorSpace" && DEFAULT_SPACES.contains(&&key[..])) {
continue;
}
let fresh = loop {
n = n.saturating_add(1);
let candidate = format!("PCUg{n}").into_bytes();
if mine.get(&candidate).is_none() && theirs.get(&candidate).is_none() {
break candidate;
}
if n == u32::MAX {
return Err(refuse("too many resources"));
}
};
mine.set(fresh.clone(), value.clone());
map.push((key.to_vec(), fresh));
}
// Counted after the merge: a page taken past the limit would refuse every later form,
// even one whose resources are all the page's already.
if mine.len() > MAX_RESOURCES {
return Err(refuse("too many resources"));
}
merged.set(cat.to_vec(), Object::Dict(mine));
if !map.is_empty() {
renames.push((cat, map));
}
}
}
let renamed = |cat: &[u8], name: &[u8]| -> Option<Vec<u8>> {
renames.iter().find(|(c, _)| *c == cat).and_then(|(_, m)| m.iter().find(|(from, _)| from == name)).map(|(_, to)| to.clone())
};
// A nested form, Type 3 font or tiling pattern without resources of its own looks its names
// up in the form's (the renderer chains them to whatever draws it). Drawn from the page, it
// would find the page's, which differ wherever a name was renamed (every form resource that
// isn't already the page's, the nested item's own entry included).
if !renames.is_empty() && ops.iter().any(|op| inherits_resources(doc, &form_res, op)) {
return Err(Refusal { renaming: true, ..refuse("it draws a nested form, Type 3 font or pattern that uses its resources") });
}
let mut depth = 0usize;
// Marked content and text objects must balance within the form's own stream: an extra EMC
// would close one the page opened, and one left open would take in the page's later content.
let mut marked = 0usize;
let mut in_text = false;
for op in &mut ops {
match op.op.as_slice() {
b"BDC" | b"BMC" => marked = marked.saturating_add(1),
b"EMC" => marked = marked.checked_sub(1).ok_or_else(|| invalid("its content ends marked content it didn't begin"))?,
b"BT" if in_text => return Err(invalid("its content begins a text object inside another")),
b"BT" => in_text = true,
b"ET" if !in_text => return Err(invalid("its content ends a text object it didn't begin")),
b"ET" => in_text = false,
_ => {}
}
match op.op.as_slice() {
b"q" if depth >= MAX_SAVES => return Err(invalid("its graphics states are nested too deeply")),
b"q" => depth += 1,
b"Q" => depth = depth.checked_sub(1).ok_or_else(|| invalid("its content restores more graphics states than it saves"))?,
_ => {}
}
let slot = resource_operand(op).map_err(|Unreadable| invalid("its content selects a resource with operands that can't be read"))?;
if let Some((cat, i)) = slot
&& let Some(operand) = op.operands.get_mut(i)
&& let Object::Name(n) = operand
&& let Some(to) = renamed(cat, n)
{
*operand = Object::Name(to);
}
if let Some((dict, _)) = &mut op.inline {
for key in [&b"CS"[..], b"ColorSpace"] {
if let Some(Object::Name(n)) = dict.get(key)
&& let Some(to) = renamed(b"ColorSpace", n)
{
dict.set(key.to_vec(), Object::Name(to));
}
}
}
}
if marked > 0 || in_text {
return Err(invalid("its content leaves marked content or a text object open"));
}
// Patterns keep their place relative to the shapes they fill: each one the form selects is
// drawn from a copy that also carries the placement. Nested forms and patterns inside
// patterns keep their own parents, so only this stream's selections change.
let mut patterns: Vec<(Vec<u8>, Object)> = Vec::new();
// What the copies of tiling patterns may add in all.
let mut pattern_bytes = MAX_FORM_BYTES;
if placement != Matrix::IDENTITY {
let mut table = merged.get(b"Pattern").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()).unwrap_or_default();
let mut moved: Renames = Vec::new();
let mut n = 0u32;
for op in &mut ops {
let Ok(Some((b"Pattern", i))) = resource_operand(op) else { continue };
let Some(Object::Name(from)) = op.operands.get(i) else { continue };
// A name the resources don't have selects nothing, inline or not.
let Some(entry) = table.get(from).cloned() else { continue };
let to = match moved.iter().find(|(f, _)| f == from) {
Some((_, to)) => to.clone(),
None => {
if moved.len() >= MAX_MOVED_PATTERNS {
return Err(refuse("it paints with too many patterns"));
}
let copy =
moved_pattern(doc, &entry, placement).ok_or_else(|| refuse("it paints with a pattern whose position can't be kept"))?;
if let Object::Stream(s) = &copy {
pattern_bytes = pattern_bytes.checked_sub(s.raw.len()).ok_or_else(|| refuse("its pattern copies would be too large"))?;
}
let fresh = loop {
n = n.saturating_add(1);
let candidate = format!("PCUgPat{n}").into_bytes();
if table.get(&candidate).is_none() {
break candidate;
}
if n == u32::MAX {
return Err(refuse("too many resources"));
}
};
// Reserved here; the caller replaces it with the copy's reference.
table.set(fresh.clone(), Object::Null);
patterns.push((fresh.clone(), copy));
moved.push((from.clone(), fresh.clone()));
fresh
}
};
if let Some(operand) = op.operands.get_mut(i) {
*operand = Object::Name(to);
}
}
if !patterns.is_empty() {
merged.set(b"Pattern".to_vec(), Object::Dict(table));
}
}
// A form in a layer (/OC) stays in it: its content is marked with the same membership.
let layer = match s.dict.get(b"OC") {
None => None,
Some(oc) => {
let mut props = merged.get(b"Properties").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()).unwrap_or_default();
let existing = props.iter().find(|(_, v)| *v == oc).map(|(k, _)| k.clone());
let name = match existing {
Some(k) => k,
None => {
let fresh = (1..=u32::MAX)
.map(|n| format!("PCUgOC{n}").into_bytes())
.find(|c| props.get(c).is_none())
.ok_or_else(|| refuse("too many resources"))?;
props.set(fresh.clone(), oc.clone());
merged.set(b"Properties".to_vec(), Object::Dict(props));
fresh
}
};
Some(name)
}
};
let num = |v: f64| Object::Real(v);
let mut out = Vec::new();
if let Some(name) = &layer {
out.push(Op::new("BDC", vec![Object::name("OC"), Object::Name(name.clone())]));
}
out.push(Op::new("q", Vec::new()));
if let Some(m) = matrix {
out.push(Op::new("cm", m.0.iter().copied().map(num).collect()));
}
let [a, b, c, d] = bbox;
let (x0, y0, x1, y1) = (a.min(c), b.min(d), a.max(c), b.max(d));
out.push(Op::new("re", vec![num(x0), num(y0), num(x1 - x0), num(y1 - y0)]));
out.push(Op::new("W", Vec::new()));
out.push(Op::new("n", Vec::new()));
out.extend(ops);
// A path the form leaves unfinished ends with it, not in the page's content after it.
out.push(Op::new("n", Vec::new()));
// A form ends with the state it was drawn with (the Do saves and restores it).
out.extend(std::iter::repeat_with(|| Op::new("Q", Vec::new())).take(depth));
out.push(Op::new("Q", Vec::new()));
if layer.is_some() {
out.push(Op::new("EMC", Vec::new()));
}
Ok(Ungrouped { ops: out, resources: merged, patterns, shows_text })
}
/// Whether form `name` in `resources` would merge into them without a rename: it has resources
/// of its own, and each of their entries is already in `resources` under the same name, as
/// [`ungroup_ops`] compares them. Only dictionaries are read, never the form's content.
pub(crate) fn merges_without_renaming(doc: &Document, resources: &Dict, name: &[u8]) -> bool {
let category = |d: &Dict, cat: &[u8]| d.get(cat).map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned());
let Some(form) = category(resources, b"XObject").and_then(|x| x.get(name).cloned()) else { return false };
let Object::Stream(s) = &*doc.resolve(&form) else { return false };
let Some(own) = s.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()) else { return false };
CATEGORIES.into_iter().all(|cat| {
let Some(theirs) = category(&own, cat) else { return true };
let mine = category(resources, cat).unwrap_or_default();
// Past these sizes `ungroup_ops` refuses the form anyway. Default colour spaces are
// never renamed (`ungroup_ops` checks them separately).
theirs.len() <= MAX_RESOURCES
&& mine.len() <= MAX_RESOURCES
&& theirs.iter().all(|(key, value)| mine.get(key) == Some(value) || (cat == b"ColorSpace" && DEFAULT_SPACES.contains(&&key[..])))
})
}
/// The first of `/DefaultGray`, `/DefaultRGB` and `/DefaultCMYK` that the form's and the page's
/// `/ColorSpace` resources don't share with an equal value (the same object, or the same value
/// resolved), and whether the form has it (else only the page does). A default colour space
/// remaps the device colours painted with the resource dictionary it is in (ISO 32000-2
/// §8.6.5.6, §7.8.3), so inline only the page's would apply: the form's would no longer remap
/// its colours, and the page's would start to.
fn unshared_default_space(doc: &Document, form: &Dict, page: &Dict) -> Option<(&'static [u8], bool)> {
let spaces = |res: &Dict| res.get(b"ColorSpace").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()).unwrap_or_default();
let (theirs, mine) = (spaces(form), spaces(page));
let differs = |key: &[u8]| match (theirs.get(key), mine.get(key)) {
(None, None) => false,
(Some(v), Some(p)) => *doc.resolve(p) != *doc.resolve(v),
_ => true,
};
// The form's own entries are named first.
let form_has = DEFAULT_SPACES.into_iter().find(|key| theirs.get(key).is_some() && differs(key)).map(|key| (key, true));
form_has.or_else(|| DEFAULT_SPACES.into_iter().find(|key| differs(key)).map(|key| (key, false)))
}
/// A copy of pattern `entry` whose `/Matrix` maps its space to the page's default space instead
/// of the form's: its own matrix, then `placement`. `None` when it isn't a readable pattern, its
/// matrix is unreadable, the result isn't finite or a tiling pattern is too large to copy.
fn moved_pattern(doc: &Document, entry: &Object, placement: Matrix) -> Option<Object> {
let resolved = doc.resolve(entry);
let (mut dict, stream) = match &*resolved {
Object::Stream(s) => (s.dict.clone(), Some(s)),
Object::Dict(d) => (d.clone(), None),
_ => return None,
};
let own = match dict.get(b"Matrix") {
None => Matrix::IDENTITY,
Some(m) => {
let m = doc.resolve(m);
let nums: Vec<Object> = m.as_array()?.iter().map(|x| doc.resolve(x).as_ref().clone()).collect();
// A matrix is six numbers: a longer array is unreadable, not cut short.
if nums.len() != 6 {
return None;
}
Matrix::from_operands(&nums)?
}
};
let moved = own.then(&placement);
if !moved.0.iter().all(|v| v.is_finite()) {
return None;
}
dict.set(b"Matrix".to_vec(), Object::Array(moved.0.iter().map(|v| Object::Real(*v)).collect()));
match stream {
Some(s) if s.raw.len() > MAX_FORM_BYTES => None,
Some(s) => Some(Object::Stream(pdfcraft_cos::Stream { dict, raw: s.raw.clone() })),
None => Some(Object::Dict(dict)),
}
}
/// How deep [`blends_inside`] follows nested forms; deeper ones count as blending.
const MAX_NESTING: usize = 8;
/// The most nested forms and tiling patterns [`blends_inside`] reads in all; past that the rest
/// count as blending, so a file can't make the check run for ever.
const MAX_NESTED_READS: usize = 1024;
/// What [`blends_inside`] carries through its nested reads: the forms on the current path (a
/// form drawing itself), the answers for each form with resources of its own (whether it blends
/// or can't be checked, and whether it shows text; neither depends on who draws it), how many
/// reads it has made, the decode budget they share with the form itself, whether the content
/// being walked shows text itself or through a Form XObject it draws, and whether blend modes
/// count (they do in a group).
struct BlendWalk<'a> {
path: Vec<pdfcraft_cos::ObjRef>,
known: HashMap<pdfcraft_cos::ObjRef, (bool, bool)>,
reads: usize,
budget: &'a mut DecodeBudget,
text: bool,
group: bool,
}
/// Whether `op` shows text (ISO 32000-2 §9.4.3, Table 107).
fn shows_text(op: &Op) -> bool {
matches!(op.op.as_slice(), b"Tj" | b"TJ" | b"'" | b"\"")
}
/// An operator whose operands the renderer can't read as it expects, so it skips the operator.
struct Unreadable;
/// Which operand of `op` names a resource, and from which category, read as the renderer reads
/// it: operands are taken from the end, so a surplus operand before them is ignored. `Do`, `gs`,
/// `sh`, `cs` and `CS` take the last operand; `Tf` a name then a size; `BDC` and `DP` a tag then
/// a name or an inline dictionary; `scn` and `SCN` numbers and the last name among them.
/// `Ok(None)` when `op` names no resource; [`Unreadable`] when the renderer would skip it, which
/// [`ungroup_ops`] refuses rather than guess at.
fn resource_operand(op: &Op) -> Result<Option<(&'static [u8], usize)>, Unreadable> {
let operands = &op.operands;
let name_at = |i: Option<usize>| i.is_some_and(|i| matches!(operands.get(i), Some(Object::Name(_))));
let last = operands.len().checked_sub(1);
let (cat, at): (&'static [u8], Option<usize>) = match op.op.as_slice() {
b"Do" => (b"XObject", last),
b"gs" => (b"ExtGState", last),
b"sh" => (b"Shading", last),
b"cs" | b"CS" => (b"ColorSpace", last),
b"Tf" => {
if !last.is_some_and(|i| matches!(operands.get(i), Some(Object::Int(_) | Object::Real(_)))) {
return Err(Unreadable);
}
(b"Font", operands.len().checked_sub(2))
}
b"BDC" | b"DP" => {
if !name_at(operands.len().checked_sub(2)) {
return Err(Unreadable);
}
match last.and_then(|i| operands.get(i)) {
Some(Object::Name(_)) => (b"Properties", last),
Some(Object::Dict(_)) => return Ok(None),
_ => return Err(Unreadable),
}
}
b"scn" | b"SCN" => {
if !operands.iter().all(|o| matches!(o, Object::Int(_) | Object::Real(_) | Object::Name(_))) {
return Err(Unreadable);
}
return Ok(operands.iter().rposition(|o| matches!(o, Object::Name(_))).map(|i| (&b"Pattern"[..], i)));
}
_ => return Ok(None),
};
match at {
Some(i) if name_at(Some(i)) => Ok(Some((cat, i))),
_ => Err(Unreadable),
}
}
/// Whether `op` draws a Form XObject (`Do`), Type 3 font (`Tf`) or tiling pattern (`scn`/`SCN`) from
/// `resources` that has no `/Resources` of its own, and so inherits the ones it is drawn with.
fn inherits_resources(doc: &Document, resources: &Dict, op: &Op) -> bool {
let Ok(Some((cat @ (b"XObject" | b"Font" | b"Pattern"), i))) = resource_operand(op) else { return false };
let Some(Object::Name(name)) = op.operands.get(i) else { return false };
let Some(entry) = resources.get(cat).map(|d| doc.resolve(d)).and_then(|d| d.as_dict().and_then(|d| d.get(name).cloned())) else {
return false;
};
let dict = match &*doc.resolve(&entry) {
Object::Stream(s) => s.dict.clone(),
Object::Dict(d) => d.clone(),
_ => return false,
};
let drawn = match cat {
b"XObject" => is_name(doc, &dict, b"Subtype", b"Form"),
b"Font" => is_name(doc, &dict, b"Subtype", b"Type3"),
// A shading pattern has no content of its own.
_ => int_of(doc, &dict, b"PatternType") == Some(1),
};
drawn && dict.get(b"Resources").is_none()
}
/// Whether `ops` (drawn with `resources`) blend with something other than Normal, which counts
/// only in a group (`walk.group`): a `gs` they use, or one used by a form or pattern they draw, at
/// any depth. A shading pattern blends through its own `/ExtGState`, a tiling pattern through its
/// content. A nested form or pattern that can't be read, is nested too deep, draws itself or is
/// past the reads or the budget allowed counts as blending, so the caller refuses rather than
/// guesses. An operator the renderer skips ([`Unreadable`]) draws nothing and is skipped. Sets
/// `walk.text` when `ops` show text, or a Form XObject they draw does; text in a tiling pattern's
/// cell is not the page's text and is not counted.
fn blends_inside(doc: &Document, resources: &Dict, ops: &[Op], depth: usize, walk: &mut BlendWalk<'_>) -> bool {
let category = |cat: &[u8]| resources.get(cat).map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()).unwrap_or_default();
let (states, xobjects, patterns) = (category(b"ExtGState"), category(b"XObject"), category(b"Pattern"));
let group = walk.group;
let state_blends = |g: Option<&Object>| group && g.map(|g| doc.resolve(g)).and_then(|g| g.as_dict().cloned()).is_some_and(|gs| blends(doc, &gs));
if ops.iter().any(shows_text) {
walk.text = true;
}
for op in ops {
let Ok(Some((cat, i))) = resource_operand(op) else { continue };
let Some(name) = op.name(i) else { continue };
let o = match cat {
b"ExtGState" if state_blends(states.get(name)) => return true,
b"XObject" => xobjects.get(name),
b"Pattern" => patterns.get(name),
_ => continue,
};
let Some(o) = o else { continue };
let resolved = doc.resolve(o);
let nested = match &*resolved {
Object::Stream(st) if op.is("Do") && is_name(doc, &st.dict, b"Subtype", b"Form") => st,
Object::Stream(st) if !op.is("Do") && int_of(doc, &st.dict, b"PatternType") == Some(1) => st,
// A shading pattern.
Object::Stream(st) if !op.is("Do") => {
if state_blends(st.dict.get(b"ExtGState")) {
return true;
}
continue;
}
Object::Dict(d) if !op.is("Do") => {
if state_blends(d.get(b"ExtGState")) {
return true;
}
continue;
}
_ => continue,
};
let r = o.as_ref();
let drawn_form = op.is("Do");
if let Some((known, text)) = r.and_then(|r| walk.known.get(&r)).copied() {
if known {
return true;
}
walk.text |= drawn_form && text;
continue;
}
if depth >= MAX_NESTING || r.is_some_and(|r| walk.path.contains(&r)) || walk.reads >= MAX_NESTED_READS {
return true;
}
walk.reads += 1;
let Ok(data) = walk.budget.decode_strict(doc, nested) else { return true };
let own_resources = nested.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned());
let has_own = own_resources.is_some();
let own = own_resources.unwrap_or_else(|| resources.clone());
let inner = parse(&data);
if inner.skipped > 0 {
return true;
}
if let Some(r) = r {
walk.path.push(r);
}
let outer_text = std::mem::replace(&mut walk.text, false);
let blends = blends_inside(doc, &own, &inner.ops, depth + 1, walk);
let inner_text = walk.text;
walk.text = outer_text || (drawn_form && inner_text);
if let Some(r) = r {
walk.path.retain(|x| *x != r);
if has_own {
walk.known.insert(r, (blends, inner_text));
}
}
if blends {
return true;
}
}
false
}
/// The blending colour space of the page group (ISO 32000-2 §11.4.7): the page's transparency
/// `/Group /CS`, resolved. `None` when the page names none, and the PDF processor chooses.
pub(crate) fn page_blend_space(doc: &Document, page: &Dict) -> Option<Object> {
let group = page.get(b"Group").map(|g| doc.resolve(g)).and_then(|g| g.as_dict().cloned()).filter(|g| is_name(doc, g, b"S", b"Transparency"))?;
group.get(b"CS").map(|cs| (*doc.resolve(cs)).clone())
}
/// Whether a group blending in `cs` composites as the page does: in the page group's own space
/// when it names one, else in DeviceRGB, the space viewers composite a page in on screen.
fn same_blend_space(doc: &Document, cs: &Object, page_space: Option<&Object>) -> bool {
let cs = doc.resolve(cs);
match page_space {
Some(page) => *cs == *page,
None => cs.as_name() == Some(b"DeviceRGB"),
}
}
/// Whether `d`'s `key` is the name `want`, directly or through an indirect object (any value may
/// be one, ISO 32000-2 §7.3.10).
fn is_name(doc: &Document, d: &Dict, key: &[u8], want: &[u8]) -> bool {
d.get(key).is_some_and(|v| doc.resolve(v).as_name() == Some(want))
}
/// `d`'s integer `key`, directly or through an indirect object.
fn int_of(doc: &Document, d: &Dict, key: &[u8]) -> Option<i64> {
d.get(key).and_then(|v| doc.resolve(v).as_int())
}
/// Whether an ExtGState blends with something other than Normal.
fn blends(doc: &Document, gs: &Dict) -> bool {
let Some(bm) = gs.get(b"BM").map(|b| doc.resolve(b)) else { return false };
let first = match &*bm {
Object::Array(a) => a.first().map(|x| doc.resolve(x)),
_ => Some(bm.clone()),
};
!first.is_some_and(|b| matches!(b.as_name(), Some(b"Normal" | b"Compatible")))
}
/// The state at op `at` of the page's streams parsed as one: whether the graphics state is
/// translucent, blended or soft-masked there, following `q`/`Q` and `gs` (each `gs` changes only
/// the parameters it sets), which a transparency group composites as a whole; and whether a path
/// or a text object is open there.
fn state_at(doc: &Document, resources: &Dict, ops: &[Op], at: usize) -> (bool, bool) {
let states = resources.get(b"ExtGState").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()).unwrap_or_default();
// Fill alpha below one, stroke alpha below one, a soft mask, a blend mode other than Normal.
let mut stack: Vec<[bool; 4]> = Vec::new();
let mut now = [false; 4];
let (mut path, mut text) = (false, false);
for op in ops.iter().take(at) {
match op.op.as_slice() {
b"m" | b"l" | b"c" | b"v" | b"y" | b"re" | b"h" | b"W" | b"W*" => path = true,
b"S" | b"s" | b"f" | b"F" | b"f*" | b"B" | b"B*" | b"b" | b"b*" | b"n" => path = false,
b"BT" => text = true,
b"ET" => text = false,
b"q" => stack.push(now),
// A Q with nothing to restore is ignored, as renderers ignore it.
b"Q" => now = stack.pop().unwrap_or(now),
b"gs" => {
let Ok(Some((_, i))) = resource_operand(op) else { continue };
let Some(gs) = op.name(i).and_then(|n| states.get(n)).map(|g| doc.resolve(g)).and_then(|g| g.as_dict().cloned()) else {
continue;
};
let alpha = |key: &[u8]| gs.get(key).map(|v| doc.resolve(v)).and_then(|v| v.as_f64()).map(|a| a < 1.0);
if let Some(v) = alpha(b"ca") {
now[0] = v;
}
if let Some(v) = alpha(b"CA") {
now[1] = v;
}
if let Some(m) = gs.get(b"SMask").map(|v| doc.resolve(v)) {
now[2] = m.as_name() != Some(b"None");
}
if gs.get(b"BM").is_some() {
now[3] = blends(doc, &gs);
}
}
_ => {}
}
}
(now.iter().any(|v| *v), path || text)
}
crates/engine/src/compare.rs
+26 -10
View File
@@ -206,6 +206,20 @@ pub struct OfficeExport {
/// `Err(false)`: it can't be encoded).
type PreparedImage = Result<(&'static str, Arc<[u8]>), bool>;
/// Displayed points per user-space unit on `info`'s page (`/UserUnit`), 1 when the page has no
/// usable size.
pub(crate) fn display_unit(info: &pdfcraft_render::PageInfo) -> f64 {
let crop_width = f64::from(info.crop[2] - info.crop[0]);
let display_width = f64::from(if info.rotation.is_multiple_of(180) { info.width } else { info.height });
if crop_width > 0.0 && crop_width.is_finite() && display_width.is_finite() { display_width / crop_width } else { 1.0 }
}
/// Bold and italic as a font's name says them (`Arial-BoldItalicMT`).
pub(crate) fn weight_and_slant(base_font: &str) -> (bool, bool) {
let f = base_font.to_ascii_lowercase();
(f.contains("bold") || f.contains("black") || f.contains("heavy"), f.contains("italic") || f.contains("oblique"))
}
impl crate::Document {
/// The pages as paragraphs and images (for Word, HTML and RTF export), including what
/// form XObjects draw.
@@ -247,9 +261,7 @@ impl crate::Document {
let b = info.user_to_view(r[2] as f32, r[3] as f32);
[a[0].min(b[0]) as f64, (info.height - a[1].max(b[1])) as f64, a[0].max(b[0]) as f64, (info.height - a[1].min(b[1])) as f64]
};
let crop_width = f64::from(info.crop[2] - info.crop[0]);
let display_width = f64::from(if info.rotation % 180 == 0 { info.width } else { info.height });
let unit = if crop_width > 0.0 && crop_width.is_finite() && display_width.is_finite() { display_width / crop_width } else { 1.0 };
let unit = display_unit(info);
let reading = pdfcraft_edit::reading_blocks_checked(cos, i).unwrap_or_default();
let forms_unread = reading.truncated;
let blocks = reading
@@ -257,7 +269,7 @@ impl crate::Document {
.into_iter()
.filter(|b| !b.text.trim().is_empty())
.map(|b| {
let f = b.base_font.to_ascii_lowercase();
let (bold, italic) = weight_and_slant(&b.base_font);
// `text` is trimmed; the editor text keeps the source's own spaces.
let space_around = [b.editor_text.starts_with(char::is_whitespace), b.editor_text.ends_with(char::is_whitespace)];
pdfcraft_export::Block {
@@ -265,8 +277,8 @@ impl crate::Document {
text: b.text,
rect: visible_rect(b.rect),
size: b.size * unit,
bold: f.contains("bold") || f.contains("black") || f.contains("heavy"),
italic: f.contains("italic") || f.contains("oblique"),
bold,
italic,
color: b.color,
rotation: b.rotation - f64::from(info.rotation),
}
@@ -301,7 +313,9 @@ impl crate::Document {
}
// The page colour (Edit a PDF ▸ Background, a producer's page fill) for formats that keep one.
let background = pdfcraft_edit::page_background(cos, i).ok().flatten();
pdfcraft_export::Page { width: info.width as f64, height: info.height as f64, blocks, images, background }
// The ruled lines tell a drawn table from text that merely lines up (#5149).
let rules = pdfcraft_edit::page_rules(cos, i).unwrap_or_default().into_iter().map(visible_rect).collect();
pdfcraft_export::Page { width: info.width as f64, height: info.height as f64, blocks, images, background, rules }
})
.collect();
(pages, truncated)
@@ -319,11 +333,13 @@ impl crate::Document {
let (pages, truncated_pages) =
self.export_pages_checked(!matches!(format, OfficeFormat::Rtf | OfficeFormat::Markdown), MAX_EXPORT_IMAGE_BYTES);
let title = self.info.title.clone().unwrap_or_else(|| self.name.trim_end_matches(".pdf").to_string());
// Word and HTML name the document's language (`/Lang`), so spelling is checked in it.
let language = self.initial_view().language;
let bytes = match format {
OfficeFormat::Docx => pdfcraft_export::docx(&pages, &title),
OfficeFormat::Docx => pdfcraft_export::docx_with_language(&pages, &title, language.as_deref()),
OfficeFormat::Pptx => pdfcraft_export::pptx(&pages, &title),
OfficeFormat::Html => pdfcraft_export::html(&pages, &title).into_bytes(),
OfficeFormat::HtmlSimple => pdfcraft_export::html_simple(&pages, &title).into_bytes(),
OfficeFormat::Html => pdfcraft_export::html_with_language(&pages, &title, language.as_deref()).into_bytes(),
OfficeFormat::HtmlSimple => pdfcraft_export::html_simple_with_language(&pages, &title, language.as_deref()).into_bytes(),
OfficeFormat::Rtf => pdfcraft_export::rtf(&pages).into_bytes(),
OfficeFormat::Markdown => pdfcraft_export::markdown(&pages).into_bytes(),
};
crates/engine/src/doc_state.rs
+3
View File
@@ -26,6 +26,9 @@ pub enum ImageEdit {
Arrange(ArrangeOrder),
/// Remove any private crop on the image.
RemoveCrop,
/// Grouped Form artwork: draw its content in place, so its text and pictures can be edited
/// one by one (#2964).
Ungroup,
}
/// Forms ▸ Merge data files into spreadsheet: the field values of each file (FDF, XFDF or a
crates/engine/src/document.rs
+110 -20
View File
@@ -8,7 +8,8 @@ use pdfcraft_render::{DocInfo, RenderConfig, RenderPool};
use crate::history::Editor;
use crate::{
CalculationOrder, CheckStyle, DocId, Edit, EditError, EditableObject, FormFieldKind, Hidden, InitialView, Label, MarkKind, SecuritySummary,
SignatureInfo, TabOrder, TrustStore, XfaLayout, a11y, form_appearance, guard, js, measure, optimize, renderer_password, unrestricted,
SignatureInfo, StatusCaptions, TabOrder, TrustStore, XfaLayout, a11y, form_appearance, guard, js, measure, optimize, renderer_password,
signatures_of, unrestricted,
};
#[cfg(test)]
use crate::{record_link_scan, record_marker_scan};
@@ -19,8 +20,9 @@ pub struct Document {
pub path: Option<String>,
/// The working file: what Save writes.
pub bytes: Arc<Vec<u8>>,
/// What is displayed (the page view, `page_render`, page images): `bytes`, plus appearances
/// drawn for display only for comments that have none ([`display_bytes`]). Never saved.
/// What is displayed (the page view, `page_render`, page images): `bytes`, plus what is drawn
/// for display only ([`display_bytes`]): appearances for comments that have none, and the
/// states of named-layer signatures. Never saved.
pub display: Arc<Vec<u8>>,
pub info: DocInfo,
pub renderer: RenderPool,
@@ -41,6 +43,9 @@ pub struct Document {
/// Signature fields, validated against the session's trust store (Signatures panel).
pub signatures: Arc<Vec<SignatureInfo>>,
pub(crate) trust: Arc<TrustStore>,
/// The words signature status layers are drawn with (the session's, see
/// [`Session::set_signature_captions`](crate::Session::set_signature_captions)).
pub(crate) captions: Arc<StatusCaptions>,
pub(crate) sig_cache: Arc<pdfcraft_sign::DigestCache>,
pub(crate) editor: Option<Editor>,
pub(crate) config: RenderConfig,
@@ -255,6 +260,15 @@ impl Document {
pdfcraft_create::image_file(&editor.cos, img.object.ok_or("the image has no object")?)
}
/// Edit text: whether grouped artwork `index` on `page` (0-based, as [`Self::page_images`]
/// lists them) shows text that [`Edit::PrepareText`](crate::Edit::PrepareText) would make
/// editable. Reads the page without changing it; `false` for a logo, a group that can't be
/// opened or a page that can't be read.
pub fn grouped_text_at(&self, page: usize, index: usize) -> bool {
let Some(editor) = self.editor.as_ref() else { return false };
crate::guard(|| pdfcraft_edit::text::forms::grouped_text_at(&editor.cos, page, index)).is_ok_and(|r| r.is_ok_and(|yes| yes))
}
/// Edit a PDF ▸ Edit text: the paragraphs on `page` (0-based).
pub fn text_blocks(&self, page: usize) -> Vec<pdfcraft_edit::TextBlock> {
self.editor.as_ref().and_then(|e| e.text_blocks(page).ok()).map(|b| b.as_ref().clone()).unwrap_or_default()
@@ -631,6 +645,12 @@ impl Document {
pub fn repair_log(&self) -> Vec<String> {
self.editor.as_ref().map(|e| e.cos.repair_log().to_vec()).unwrap_or_default()
}
/// Objects of the file that could not be read and read as null, one line each (see
/// [`pdfcraft_cos::Document::unreadable_objects`]): only those read so far.
pub fn unreadable_objects(&self) -> Vec<String> {
self.editor.as_ref().map(|e| e.cos.unreadable_objects()).unwrap_or_default()
}
}
/// A title that names no document: what a web browser writes when it saves a blank page or a
@@ -643,25 +663,84 @@ pub(crate) fn is_placeholder_title(title: &str) -> bool {
address || t == "untitled"
}
/// The working file plus appearances for comments that have none, and for form fields the file
/// leaves to the viewer (#3470). Signature artwork is preserved; validation badges belong to the
/// viewer and never enter these bytes, which image/text exports and Print as Image also consume.
pub(crate) fn display_bytes(editor: Option<&mut Editor>, bytes: &Arc<Vec<u8>>) -> Arc<Vec<u8>> {
/// What is displayed: the working file plus, in memory only and never saved, the appearances
/// PdfCraft draws for comments that have none ([`pdfcraft_annot::with_missing_appearances`]) and
/// for form fields the file leaves to the viewer (#3470), and the signature states drawn into
/// named-layer signature appearances ([`pdfcraft_sign::show_status_layers`]). The working file is
/// unchanged, and so is everything that reads it (saving, signatures, …). Image and text exports
/// and Print as Image read these bytes, so they show the page as displayed; validation badges
/// belong to the viewer and never enter them.
///
/// `signatures` is input and output: each one whose state gets drawn comes back marked
/// (`verdict_shown`). A mark describes the bytes returned, so every mark is cleared first and a
/// copy that cannot be written takes its marks with it.
pub(crate) fn display_bytes(
editor: Option<&mut Editor>,
bytes: &Arc<Vec<u8>>,
signatures: &mut [SignatureInfo],
captions: &StatusCaptions,
) -> Arc<Vec<u8>> {
for sig in signatures.iter_mut() {
sig.verdict_shown = false;
}
let Some(e) = editor else { return bytes.clone() };
let comments = pdfcraft_annot::with_missing_appearances_cached(&e.cos, &mut e.appearances);
let fields = pdfcraft_forms::with_field_appearances_cached(comments.as_ref().unwrap_or(&e.cos), &mut e.field_appearances);
let Some(display) = fields.or(comments) else {
if let Some(mut display) = fields.or(comments) {
// Written whether or not a signature changed: the copy exists for the comments and fields.
let _ = pdfcraft_sign::show_status_layers(&mut display, signatures, captions);
return write_display_copy(e, bytes, &display, signatures);
}
// No comment or field needed an appearance, so there is no copy yet: build one only when a
// signature's state would change it.
if !pdfcraft_sign::has_status_layers(&e.cos, signatures) {
return bytes.clone();
};
match write_incremental(&display, &SaveOptions::default()) {
}
let mut display = e.cos.clone();
if !pdfcraft_sign::show_status_layers(&mut display, signatures, captions) {
return bytes.clone();
}
write_display_copy(e, bytes, &display, signatures)
}
/// Write the display copy, or keep the working file when it cannot be written. The page then
/// shows the file's own artwork, so the marks are cleared. Losing the copy is a repair, so it is
/// logged, once however many refreshes lose it again.
pub(crate) fn write_display_copy(
e: &mut Editor,
bytes: &Arc<Vec<u8>>,
display: &pdfcraft_cos::Document,
signatures: &mut [SignatureInfo],
) -> Arc<Vec<u8>> {
match write_incremental(display, &SaveOptions::default()) {
Ok(written) => Arc::new(written),
Err(err) => {
e.cos.note_repair(format!("missing comment or field appearances could not be drawn ({err})"));
for sig in signatures.iter_mut() {
sig.verdict_shown = false;
}
let line = format!("the display copy could not be written, so pages show the file as stored ({err})");
if !e.cos.repair_log().contains(&line) {
e.cos.note_repair(line);
}
bytes.clone()
}
}
}
/// Carry the marks across a revalidation that leaves the display copy alone, when it may: only
/// when every verdict is the one the copy was drawn with. Returns whether it did; otherwise the
/// copy has to be drawn again.
pub(crate) fn carry_verdict_marks(previous: &[SignatureInfo], next: &mut [SignatureInfo]) -> bool {
let unchanged = previous.len() == next.len()
&& previous.iter().zip(next.iter()).all(|(p, n)| p.field == n.field && p.revision == n.revision && p.status == n.status);
if unchanged {
for (p, n) in previous.iter().zip(next.iter_mut()) {
n.verdict_shown = p.verdict_shown;
}
}
unchanged
}
/// `comments` (the display copy with comment appearances, if it needed any; else the document
/// itself) with its form fields drawn from their values where the file leaves that to the viewer.
fn with_field_appearances(comments: Option<pdfcraft_cos::Document>, doc: &pdfcraft_cos::Document) -> Option<pdfcraft_cos::Document> {
@@ -688,17 +767,28 @@ pub(crate) fn info_fields<'a>(info: &'a mut DocInfo, cos: &pdfcraft_cos::Documen
]
}
/// The copy PDFCraft displays when opening `bytes`: with appearances drawn, in memory only, for
/// comments that have none and form fields the file leaves to the viewer. Render paths outside the app, such as the CLI, use this so their page
/// images agree with the app and independent viewers. It never changes the source document, and an
/// unparseable file, a write failure or a parser panic falls back to the original bytes.
/// The copy PDFCraft displays when opening `bytes` with no trusted certificates and English
/// captions ([`display_bytes`]): appearances drawn, in memory only, for comments that have none
/// and form fields the file leaves to the viewer, and the state of each named-layer signature.
/// Render paths outside the app, such as the CLI, use this so their page images agree with the app
/// and independent viewers. It never changes the source document: a file with nothing to draw, an
/// unparseable file, a write failure or a parser panic gives back the original bytes.
pub fn display_bytes_for(bytes: Arc<Vec<u8>>, password: Option<&str>) -> Arc<Vec<u8>> {
guard(|| {
pdfcraft_cos::Document::open_with_password(bytes.clone(), password)
.ok()
.and_then(|cos| with_field_appearances(pdfcraft_annot::with_missing_appearances(&cos), &cos))
.and_then(|display| write_incremental(&display, &SaveOptions::default()).ok())
.map(Arc::new)
let cos = pdfcraft_cos::Document::open_with_password(bytes.clone(), password).ok()?;
let mut signatures = signatures_of(&cos, &bytes, &TrustStore::default(), &pdfcraft_sign::DigestCache::default());
let layered = pdfcraft_sign::has_status_layers(&cos, &signatures);
let (mut display, added) = match with_field_appearances(pdfcraft_annot::with_missing_appearances(&cos), &cos) {
Some(display) => (display, true),
None if layered => (cos, false),
None => return None,
};
// As in the app, a layer recognised but not drawn after all makes no copy.
let drawn = layered && pdfcraft_sign::show_status_layers(&mut display, &mut signatures, &StatusCaptions::default());
if !(added || drawn) {
return None;
}
write_incremental(&display, &SaveOptions::default()).ok().map(Arc::new)
})
.ok()
.flatten()
crates/engine/src/edit.rs
+4
View File
@@ -466,6 +466,10 @@ pub enum Edit {
/// Formatting changes (font, size, colour, alignment); default keeps the paragraph's.
style: pdfcraft_edit::BlockStyle,
},
/// Expose supported grouped source text on one page for direct editing.
PrepareText {
page: usize,
},
/// An explicit open-licensed full font program, completing subsets or selected by a user.
RegisterTextFont {
page: usize,
crates/engine/src/edit/labels.rs
+3
View File
@@ -111,6 +111,7 @@ pub const EDIT_LABELS: &[&str] = &[
"Delete document JavaScript",
"Edit document JavaScript",
"Edit text",
"Prepare grouped text",
"Load document font",
"Move object",
"Move objects",
@@ -269,6 +270,7 @@ impl Edit {
Edit::SetDocumentScript { script: None, .. } => Label::of("Delete document JavaScript"),
Edit::SetDocumentScript { .. } => Label::of("Edit document JavaScript"),
Edit::EditTextLine { .. } | Edit::EditTextBlock { .. } => Label::of("Edit text"),
Edit::PrepareText { .. } => Label::of("Prepare grouped text"),
Edit::RegisterTextFont { .. } => Label::of("Load document font"),
Edit::MoveObjects { objects, .. } => plural("Move object", "Move objects", objects.len()),
Edit::EditPageImage { change, .. } => Label::of(match change {
@@ -286,6 +288,7 @@ impl Edit {
pdfcraft_edit::ArrangeOrder::SendToBack => "Send to back",
},
ImageEdit::RemoveCrop => "Remove crop",
ImageEdit::Ungroup => "Ungroup artwork",
}),
Edit::AddHeaderFooter { replace: false, .. } => Label::of("Add header & footer"),
Edit::AddHeaderFooter { .. } => Label::of("Update header & footer"),
crates/engine/src/edit/run.rs
+16 -1
View File
@@ -148,6 +148,7 @@ pub(crate) fn check_permission(edit: &Edit, p: &pdfcraft_cos::Permissions) -> Re
| Edit::RemoveStandardId { .. }
| Edit::EditTextLine { .. }
| Edit::EditTextBlock { .. }
| Edit::PrepareText { .. }
| Edit::RegisterTextFont { .. }
| Edit::EditPageImage { .. }
| Edit::Flatten { .. }
@@ -193,6 +194,9 @@ pub(crate) struct EditCtx {
/// The overlay streams the edit's applied redactions added, which alone the renderer's check
/// leaves out.
pub(crate) redaction_overlays: Vec<pdfcraft_cos::ObjRef>,
/// What [`Edit::PrepareText`] prepared, repaired and left grouped, for
/// [`Session::prepare_text`](crate::Session::prepare_text).
pub(crate) prepared: Option<pdfcraft_edit::text::forms::Preparation>,
}
impl EditCtx {
@@ -222,6 +226,7 @@ impl EditCtx {
rng: getrandom::fill,
renamed_fields: Vec::new(),
redaction_overlays: Vec::new(),
prepared: None,
}
}
@@ -306,6 +311,7 @@ fn run_edit_page_image(doc: &mut pdfcraft_cos::Document, page: usize, index: usi
ImageEdit::Delete => pdfcraft_edit::ImageChange::Delete,
ImageEdit::Arrange(order) => pdfcraft_edit::ImageChange::Arrange(*order),
ImageEdit::RemoveCrop => pdfcraft_edit::ImageChange::RemoveCrop,
ImageEdit::Ungroup => pdfcraft_edit::ImageChange::Ungroup,
};
let page_model = pdfcraft_model::pages(doc)
.into_iter()
@@ -411,7 +417,7 @@ fn run_edit_page_image(doc: &mut pdfcraft_cos::Document, page: usize, index: usi
ai.image = new_image;
pdfcraft_edit::update_content(doc, page, added_idx, &AddedContent::Image(ai))?;
}
ImageEdit::Crop(_) | ImageEdit::RemoveCrop | ImageEdit::RotateBy(_) | ImageEdit::Arrange(_) | ImageEdit::Delete => {
ImageEdit::Crop(_) | ImageEdit::RemoveCrop | ImageEdit::RotateBy(_) | ImageEdit::Arrange(_) | ImageEdit::Delete | ImageEdit::Ungroup => {
pdfcraft_edit::change_image(doc, page, index, &c)?;
}
}
@@ -726,6 +732,15 @@ pub(crate) fn run_edit(doc: &mut pdfcraft_cos::Document, edit: &Edit, cx: &mut E
Edit::EditTextBlock { page, block, text, style } => {
pdfcraft_edit::rewrite_block(doc, *page, *block, Some(text), style)?;
}
Edit::PrepareText { page } => {
let result = pdfcraft_edit::text::forms::prepare(doc, *page)?;
if result.expanded == 0 && result.repaired_fonts == 0 {
return Err(EditError::Invalid(
result.refused.first().cloned().unwrap_or_else(|| "this page has no supported grouped text to prepare".into()),
));
}
cx.prepared = Some(result);
}
Edit::RegisterTextFont { page, bytes } => {
pdfcraft_edit::text::register_text_font(doc, *page, bytes)?;
}
crates/engine/src/export.rs
+6 -5
View File
@@ -1,8 +1,8 @@
//! Export a PDF ▸ Image and Text (execution plan M10.3, the first formats).
//!
//! [`Exporter`] renders pages to PNG, JPEG or TIFF at a resolution and extracts reading-order
//! text, from a
//! document's working file (so unsaved edits and hidden layers are respected, as on screen).
//! text, from the copy a document displays ([`Document::export_source`]): its working file, so
//! unsaved edits and hidden layers are respected, with what is drawn for display only.
use std::collections::BTreeMap;
use std::panic::{AssertUnwindSafe, catch_unwind};
@@ -67,9 +67,10 @@ pub struct ExportSource {
impl Document {
/// What an export reads, for exporting on another thread. These are the displayed
/// bytes, so they may carry missing comment appearances PdfCraft drew. Signature
/// artwork stays original; viewer validation badges are not document content and
/// do not enter image/text exports or Print as Image. Saving uses the working file.
/// bytes, so they may carry appearances PdfCraft drew that the file itself does not hold:
/// missing comment appearances, and the signature states of named-layer signature
/// appearances. Exports and Print as Image follow the page as shown; the viewer's badges
/// are not document content and never enter them. Saving uses the working file.
pub fn export_source(&self) -> ExportSource {
ExportSource {
bytes: self.display.clone(),
crates/engine/src/history.rs
+1
View File
@@ -81,6 +81,7 @@ pub(crate) fn scope_of(edit: &Edit) -> Scope {
Edit::MoveObjects { .. }
| Edit::EditTextLine { .. }
| Edit::EditTextBlock { .. }
| Edit::PrepareText { .. }
| Edit::EditPageImage { .. }
| Edit::EnhanceScan { .. }
| Edit::AddText { .. }
crates/engine/src/js.rs
+6 -2
View File
@@ -429,12 +429,16 @@ impl Session {
let mut out = Vec::new();
for page in list {
let Some(info) = doc.info.pages.get(page) else { continue };
let res = r.render(pdfcraft_render::RenderRequest { page, kind: pdfcraft_render::RequestKind::Text, scale: 1.0, ..Default::default() });
let shapes = pdfcraft_forms::detect::page_shapes(cos, page);
let existing: Vec<[f64; 4]> = doc.form.iter().flat_map(|f| f.widgets.iter()).filter(|w| w.page == Some(page)).map(|w| w.rect).collect();
let (found, turn) = if shapes.boxes.is_empty() && shapes.rules.is_empty() && crate::scan_fields::is_scan(doc, page, info) {
self.detect_on_scan(id, &mut r, page, res.text.as_deref(), &existing, &taken)
// Labels come from the page's own text. Words that only /ActualText names (a stamp
// drawn as outlines) are no labels, and leave the scan to be read.
let own = crate::ocr::own_text(&mut r, page).ok();
self.detect_on_scan(id, &mut r, page, own.as_ref(), &existing, &taken)
} else {
let res =
r.render(pdfcraft_render::RenderRequest { page, kind: pdfcraft_render::RequestKind::Text, scale: 1.0, ..Default::default() });
let words = res.text.map(|t| detection_words(&t, info)).unwrap_or_default();
(pdfcraft_forms::detect::detect(&words, &shapes, &existing, &taken), 0)
};
crates/engine/src/lib.rs
+7 -3
View File
@@ -44,6 +44,7 @@ mod pictures;
pub mod print_raster;
pub mod printcheck;
mod protection;
mod rich_copy;
pub mod scan_clean;
mod scan_colours;
mod scan_fields;
@@ -68,6 +69,7 @@ pub use links::{
MAX_REMOTE_DOCUMENT_BYTES, RemoteDocumentPathError, read_remote_document, remote_document_path_is_absolute, resolve_remote_document_path,
};
pub use pdfcraft_edit::strip_subset_tag;
pub use rich_copy::RichCopy;
pub use signature_image::{ImageSignaturePreview, SignatureImage};
pub use signature_scan::{PhotoRect, ScanError, SignaturePhoto, Turn};
pub use stamp_preview::{CommentLayer, StampPreview};
@@ -91,7 +93,8 @@ pub use pdfcraft_edit::added::{
};
pub use pdfcraft_edit::paragraph_align;
pub use pdfcraft_edit::{
Added, AddedImage, AddedText, Align as TextAlign, Background, Content as AddedContent, Family as FontFamily, HeaderFooter, MarkKind, Watermark,
Added, AddedImage, AddedText, Align as TextAlign, Background, Content as AddedContent, Family as FontFamily, HeaderFooter, Ink as TextInk,
MarkKind, Watermark,
};
pub use pdfcraft_forms::{
BorderStyle, CalculatedField, CalculationIssue, CalculationOrder, CheckStyle, Field as FormField, FieldAction, FieldChange, FieldFont,
@@ -126,8 +129,9 @@ pub use pdfcraft_redact::patterns::{
MAX_WORDS as REDACT_MAX_WORDS, PATTERNS as REDACT_PATTERNS, Pattern as RedactPattern, find as find_pattern, word_list as redact_word_list,
};
pub use pdfcraft_redact::sanitize::{HIDDEN, Hidden};
pub use pdfcraft_redact::user_regex::{UserRegex as RedactRegex, VALIDATORS as REDACT_VALIDATORS, Validator as RedactValidator};
pub use pdfcraft_sign as sign;
pub use pdfcraft_sign::{SignOptions, SignatureInfo, Status as SignatureStatus, TrustStore};
pub use pdfcraft_sign::{SignOptions, SignatureInfo, Status as SignatureStatus, StatusCaptions, TrustStore};
pub use pdfcraft_xfdf::Format as DataFormat;
pub use pdfcraft_xfdf::{Outline as CommentOutline, comment_count, comment_outlines, comments_bounds, is_xfdf_comments};
@@ -138,7 +142,7 @@ pub use comments::{SummarySort, comment_kind, comment_summary, comment_type_name
#[cfg(test)]
use document::page_runs;
pub use document::{Document, EditNotice, TextNotice, display_bytes_for, render_password_for};
use document::{display_bytes, info_fields, recovered_render_password, same_pages, shown_under_redactions, undrawn_notices};
use document::{carry_verdict_marks, display_bytes, info_fields, recovered_render_password, same_pages, shown_under_redactions, undrawn_notices};
pub use edit::{BATCH_LABELS, EDIT_LABELS, Edit};
pub use error::{EditError, SourceProblem};
pub use fill_sign::{displayed_offset, text_box_at, typed_signature_graphic, typed_signature_shape};
crates/engine/src/ocr.rs
+423 -158
View File
@@ -11,10 +11,12 @@ use std::sync::{Arc, Mutex};
#[cfg(not(target_arch = "wasm32"))]
use std::time::{Duration, Instant};
use pdfcraft_render::text::TextFilter;
use pdfcraft_render::{PageInfo, PageRenderer, RenderConfig, RenderRequest, RequestKind};
pub use pdfcraft_ocr::{
ConfidenceBand, LANGUAGES, MergeStrategy, Models, Ocr, OcrError, PlacedWord, Recognizer, band_for, is_suspect, low_confidence,
ConfidenceBand, DETECTION_MODEL, LANGUAGES, Line, MergeStrategy, Models, Ocr, OcrError, OcrImage, PlacedWord, RECOGNITION_MODEL, Recognizer,
StrategyOutcome, Word, band_for, is_suspect, lines_from_json, low_confidence,
};
use crate::{DocId, Edit, EditError, Session};
@@ -161,6 +163,18 @@ const ALREADY_ON_PAGE: f32 = 0.5;
/// is a resolution on paper this size or smaller.
const SHEET: f32 = 842.0;
/// The text of `page`'s own: its glyphs, without the words drawn as outlines or pictures that only
/// `/ActualText` names.
pub(crate) fn own_text(r: &mut PageRenderer, page: usize) -> Result<pdfcraft_render::PageText, String> {
r.filtered_text(page, TextFilter { painted: true, ..Default::default() })
}
/// Whether `page` has text of its own ([`own_text`]). A stamp drawn as outlines on a scan still
/// leaves the scan to read; its words are not read twice, since they count as already on the page.
fn has_own_text(r: &mut PageRenderer, page: usize) -> bool {
own_text(r, page).is_ok_and(|t| t.glyphs.iter().any(|g| !g.text.trim().is_empty()))
}
/// How much of `word` the `text` boxes cover, from 0 to 1. Boxes are `[x0, y0, x1, y1]` in page
/// view space. Overlaps between the text boxes are counted twice (glyphs rarely overlap), so the
/// sum is capped at the whole word.
@@ -244,6 +258,23 @@ struct EngineCache {
ocrs: Option<Arc<dyn Recognizer>>,
}
/// The engines built so far (see [`recognizers`] and [`install`]).
static ENGINES: Mutex<Option<EngineCache>> = Mutex::new(None);
/// Share an ocrs recogniser loaded some other way than from the model files [`Models::find`]
/// looks for: the web build downloads the models and loads them with [`Ocr::from_bytes`]. From
/// then on [`recognizers`] hands it out and the engine counts as available.
pub fn install(ocr: Ocr) -> Arc<dyn Recognizer> {
let ocrs: Arc<dyn Recognizer> = Arc::new(pdfcraft_ocr::OcrsRecognizer::from_ocr(ocr));
ENGINES.lock().unwrap_or_else(|e| e.into_inner()).get_or_insert_with(|| EngineCache { ocrs: None }).ocrs = Some(ocrs.clone());
ocrs
}
/// Whether an ocrs recogniser has been loaded (from the model files, or by [`install`]).
fn ocrs_loaded() -> bool {
ENGINES.lock().unwrap_or_else(|e| e.into_inner()).as_ref().is_some_and(|c| c.ocrs.is_some())
}
/// How long a negative engine probe (no program there) is remembered. A short TTL, not
/// forever: a tesseract installed while the app runs is found by the next job, without a
/// restart. An engine that was found is remembered for the session.
@@ -395,10 +426,11 @@ fn models_installed_cached() -> bool {
found
}
/// The web build has no disk search worth caching.
/// The web build has no model files: the models count as installed once [`install`] has
/// loaded the ones it downloaded.
#[cfg(target_arch = "wasm32")]
fn models_installed_cached() -> bool {
Models::find().is_some()
ocrs_loaded()
}
/// The program's `--version` line, remembered per program path for the session (`ocr_status`
@@ -460,7 +492,7 @@ pub fn engines_status() -> Vec<EngineStatus> {
/// takes a location, because an agent must not choose which program runs; the tools report the
/// default settings.)
pub fn engines_status_for(settings: &OcrSettings) -> Vec<EngineStatus> {
let ocrs = Models::find().is_some();
let ocrs = ocrs_loaded() || Models::find().is_some();
let mut out = vec![EngineStatus { id: "ocrs", available: ocrs, why_not: (!ocrs).then(models_missing), program: None, version: None }];
#[cfg(not(target_arch = "wasm32"))]
{
@@ -511,7 +543,6 @@ pub fn recognizers(settings: &OcrSettings) -> Result<Recognizers, String> {
let configured = configured_tesseract(settings);
let tesseract =
if matches!(settings.engine, EngineChoice::Tesseract | EngineChoice::Auto) { tesseract_engine(configured.as_deref()) } else { None };
static ENGINES: Mutex<Option<EngineCache>> = Mutex::new(None);
let mut slot = ENGINES.lock().unwrap_or_else(|e| e.into_inner());
let cache = slot.get_or_insert_with(|| EngineCache { ocrs: None });
if cache.ocrs.is_none() {
@@ -554,7 +585,7 @@ fn models_missing() -> String {
/// never waits.
pub fn available() -> bool {
let configured = std::env::var_os("PDFCRAFT_TESSERACT").map(|v| v.to_string_lossy().into_owned());
Models::find().is_some() || tesseract_engine(configured.as_deref()).is_some()
ocrs_loaded() || Models::find().is_some() || tesseract_engine(configured.as_deref()).is_some()
}
/// Why `engine` cannot run right now, named for the UI's tooltips (the Engine dropdown
@@ -690,137 +721,226 @@ impl OcrJob {
/// Read the pages. `progress(done, total)` is called before each page; returning `false`
/// stops (the pages read so far are returned).
pub fn run(self, recognizers: &Recognizers, mut progress: impl FnMut(usize, usize) -> bool) -> Vec<OcrPage> {
let config =
RenderConfig { password: self.password.as_deref().map(Arc::from), hide_comments: self.settings.native_text_only, ..Default::default() };
let mut r = PageRenderer::new(self.bytes.clone(), config);
let mut out = Vec::new();
let total = self.pages.len();
let options = pdfcraft_ocr::RecognizeOptions { language: self.settings.language.clone() };
for (done, &page) in self.pages.iter().enumerate() {
if !progress(done, total) {
break;
let mut steps = self.steps();
let mut out = Vec::new();
while progress(steps.done(), total) {
match steps.next_page() {
None => break,
Some(Err(skipped)) => out.push(skipped),
Some(Ok(shot)) => out.push(shot.read(recognizers)),
}
let Some(info) = self.infos.get(page) else { continue };
let skip = |why: &str| OcrPage { page, words: Vec::new(), already_on_page: 0, skipped: Some(why.into()), notes: Vec::new() };
// The boxes of the page's own text (an earlier OCR layer's included), in view space.
let existing: Vec<[f32; 4]> = r
.render(RenderRequest { page, kind: RequestKind::Text, scale: 1.0, ..Default::default() })
.text
.map(|t| t.glyphs.iter().filter(|g| !g.text.trim().is_empty()).map(|g| g.rect).collect())
.unwrap_or_default();
if self.settings.skip_text_pages && !self.settings.force_ocr && !existing.is_empty() {
out.push(skip("the page already contains text"));
continue;
}
// A scan is read at its own resolution (enlarging it only blurs the letters), at
// most the chosen one.
let scale = self.render_scale(page);
let shot = r.render(RenderRequest { page, scale, ..Default::default() });
if let Some(e) = shot.error {
out.push(skip(&e));
continue;
}
let full = match pdfcraft_ocr::OcrImage::new(shot.width, shot.height, shot.rgba.to_vec()) {
Ok(img) => img,
Err(e) => {
out.push(skip(&e.to_string()));
continue;
}
};
// The optional region: crop the rendered page, clamped; an empty region skips the
// page with a named reason (nothing was asked of it).
let (image, offset) = match self.settings.region {
None => (full, [0.0f32; 2]),
Some(region) => match clamp_region(region, shot.width, shot.height) {
Some([x0, y0, x1, y1]) => match crop(&full, x0, y0, x1, y1) {
Some(c) => (c, [x0 as f32, y0 as f32]),
None => {
out.push(skip("the selected region is empty"));
continue;
}
},
None => {
out.push(skip("the selected region is outside the page"));
continue;
}
},
};
// Preprocessing sits between the page raster and recognition; its word boxes come
// back in processed coordinates and are mapped to the ORIGINAL raster's before
// placement. The render dpi is the real one (pixels per point × 72), never a
// fixed value.
let pre = pdfcraft_ocr::preprocess::preprocess(
image,
scale * 72.0,
&pdfcraft_ocr::preprocess::PreprocessOptions {
auto_rotate: self.settings.auto_rotate,
deskew: self.settings.deskew,
denoise: self.settings.denoise,
binarize: self.settings.binarize,
},
);
let (image, inverse) = match pre {
Ok(p) => (p.image, p.inverse_transform),
Err(e) => {
out.push(skip(&e.to_string()));
continue;
}
};
let outcome = match pdfcraft_ocr::recognize_with_strategy(
recognizers.primary.as_ref(),
recognizers.secondary.as_deref(),
self.settings.strategy,
&image,
&options,
) {
Ok(o) => o,
Err(e) => {
out.push(skip(&e.to_string()));
continue;
}
};
let scale = shot.width as f32 / info.width.max(1e-3);
let to_user = |x: f32, y: f32| {
// processed pixels → original raster pixels (the exact inverse) → past the
// region crop → page view space.
let (ox, oy) = inverse.apply(f64::from(x), f64::from(y));
let [u, v] = info.view_to_user((ox as f32 + offset[0]) / scale, (oy as f32 + offset[1]) / scale);
[u as f64, v as f64]
};
// Validate recognition geometry and compare against existing text in view space,
// undoing preprocessing and region offsets before testing overlap.
let (fresh, already): (Vec<_>, Vec<_>) = outcome.lines.iter().flat_map(|l| &l.words).filter(|w| box_writable(w)).partition(|w| {
let [l, t, r, b] = w.rect;
let bounds = [(l, t), (r, t), (r, b), (l, b)].into_iter().fold(
[f32::INFINITY, f32::INFINITY, f32::NEG_INFINITY, f32::NEG_INFINITY],
|mut bounds, (x, y)| {
let (x, y) = inverse.apply(f64::from(x), f64::from(y));
let x = (x as f32 + offset[0]) / scale;
let y = (y as f32 + offset[1]) / scale;
bounds[0] = bounds[0].min(x);
bounds[1] = bounds[1].min(y);
bounds[2] = bounds[2].max(x);
bounds[3] = bounds[3].max(y);
bounds
},
);
covered(bounds, &existing) < ALREADY_ON_PAGE
});
let words = fresh.into_iter().map(|w| PlacedWord::place(w, to_user)).collect::<Vec<_>>();
let mut notes = outcome.notes;
// Only visible editable output needs display glyphs. Searchable output preserves
// every Unicode character with empty Type3 programs and ToUnicode maps.
if self.settings.output_mode == OutputMode::EditableText && !words.is_empty() {
let missing = pdfcraft_fonts::TextPlan::for_words(words.iter().map(|w| w.text.as_str())).missing();
if missing > 0 {
notes.push(format!("{missing} characters had no font that could show them and appear as ? in the output"));
}
}
out.push(OcrPage { page, words, already_on_page: already.len(), skipped: None, notes });
}
progress(total, total);
out
}
/// Read the pages one at a time, preparing each page's image and recognising it separately:
/// the web build prepares pages on the page and recognises them in Web Workers.
pub fn steps(self) -> OcrSteps {
let config =
RenderConfig { password: self.password.as_deref().map(Arc::from), hide_comments: self.settings.native_text_only, ..Default::default() };
let renderer = PageRenderer::new(self.bytes.clone(), config);
OcrSteps { job: self, renderer, next: 0 }
}
}
/// [`OcrJob`] a page at a time: [`OcrSteps::next_page`] renders the next page, crops it to the
/// region and preprocesses it; an engine reads [`OcrShot::image`], and [`OcrShot::place`] puts
/// the words found on the page ([`OcrShot::read`] does both with the job's engines).
pub struct OcrSteps {
job: OcrJob,
renderer: PageRenderer,
next: usize,
}
impl OcrSteps {
/// The number of pages to read.
pub fn total(&self) -> usize {
self.job.pages.len()
}
/// The number of pages prepared (or skipped) so far.
pub fn done(&self) -> usize {
self.next
}
/// The settings the job reads with.
pub fn settings(&self) -> &OcrSettings {
&self.job.settings
}
/// Prepare the next page for recognition. `Err` is a page that isn't read (it already has
/// text, or couldn't be rendered or prepared), with the reason; `None` once every page is
/// done.
pub fn next_page(&mut self) -> Option<Result<OcrShot, OcrPage>> {
loop {
let page = *self.job.pages.get(self.next)?;
self.next += 1;
let Some(info) = self.job.infos.get(page).cloned() else { continue };
return Some(self.prepare(page, info));
}
}
fn prepare(&mut self, page: usize, info: PageInfo) -> Result<OcrShot, OcrPage> {
let settings = &self.job.settings;
let skip = |why: &str| OcrPage { page, words: Vec::new(), already_on_page: 0, skipped: Some(why.into()), notes: Vec::new() };
let r = &mut self.renderer;
// The boxes of all text on the page, in view space: an earlier OCR layer's, and words
// only /ActualText names, so neither is read again.
let existing: Vec<[f32; 4]> = r
.render(RenderRequest { page, kind: RequestKind::Text, scale: 1.0, ..Default::default() })
.text
.map(|t| t.glyphs.iter().filter(|g| !g.text.trim().is_empty()).map(|g| g.rect).collect())
.unwrap_or_default();
if settings.skip_text_pages && !settings.force_ocr && !existing.is_empty() && has_own_text(r, page) {
return Err(skip("the page already contains text"));
}
// A scan is read at its own resolution (enlarging it only blurs the letters), at
// most the chosen one.
let scale = self.job.render_scale(page);
let shot = r.render(RenderRequest { page, scale, ..Default::default() });
if let Some(e) = shot.error {
return Err(skip(&e));
}
let full = pdfcraft_ocr::OcrImage::new(shot.width, shot.height, shot.rgba.to_vec()).map_err(|e| skip(&e.to_string()))?;
// The optional region: crop the rendered page, clamped; an empty region skips the
// page with a named reason (nothing was asked of it).
let (image, offset) = match settings.region {
None => (full, [0.0f32; 2]),
Some(region) => match clamp_region(region, shot.width, shot.height) {
Some([x0, y0, x1, y1]) => match crop(&full, x0, y0, x1, y1) {
Some(c) => (c, [x0 as f32, y0 as f32]),
None => return Err(skip("the selected region is empty")),
},
None => return Err(skip("the selected region is outside the page")),
},
};
// Preprocessing sits between the page raster and recognition; its word boxes come
// back in processed coordinates and are mapped to the ORIGINAL raster's before
// placement. The render dpi is the real one (pixels per point × 72), never a
// fixed value.
let pre = pdfcraft_ocr::preprocess::preprocess(
image,
scale * 72.0,
&pdfcraft_ocr::preprocess::PreprocessOptions {
auto_rotate: settings.auto_rotate,
deskew: settings.deskew,
denoise: settings.denoise,
binarize: settings.binarize,
},
)
.map_err(|e| skip(&e.to_string()))?;
Ok(OcrShot {
page,
image: Some(pre.image),
inverse: pre.inverse_transform,
offset,
pixels_per_point: shot.width as f32 / info.width.max(1e-3),
info,
existing,
strategy: settings.strategy,
options: pdfcraft_ocr::RecognizeOptions { language: settings.language.clone() },
output_mode: settings.output_mode,
})
}
}
/// A page prepared for recognition: the image an engine reads, and what placing the words
/// found in it needs.
pub struct OcrShot {
pub page: usize,
/// The page's pixels as the engines read them: rendered, cropped to the region and
/// preprocessed (`None` once [`OcrShot::take_image`] has handed them on).
image: Option<OcrImage>,
/// Processed pixels → the rendered raster's (the exact inverse of preprocessing).
inverse: pdfcraft_ocr::preprocess::InverseTransform,
/// Where the region crop starts in the rendered raster.
offset: [f32; 2],
/// The rendered raster's pixels per point of page view space.
pixels_per_point: f32,
info: PageInfo,
/// The boxes of the page's own text, in view space.
existing: Vec<[f32; 4]>,
strategy: MergeStrategy,
options: pdfcraft_ocr::RecognizeOptions,
output_mode: OutputMode,
}
impl OcrShot {
/// What the engines read (`None` once taken).
pub fn image(&self) -> Option<&OcrImage> {
self.image.as_ref()
}
/// Hand the image on (the web build sends it to a worker), keeping what placing the words
/// needs: the page's pixels need not stay in memory while the worker reads them.
pub fn take_image(&mut self) -> Option<OcrImage> {
self.image.take()
}
/// Recognise the image with `recognizers` (per the job's strategy) and place the words.
pub fn read(&self, recognizers: &Recognizers) -> OcrPage {
let Some(image) = &self.image else { return self.place(Err("the page's image was handed to another reader".into())) };
let outcome = pdfcraft_ocr::recognize_with_strategy(
recognizers.primary.as_ref(),
recognizers.secondary.as_deref(),
self.strategy,
image,
&self.options,
);
self.place(outcome.map_err(|e| e.to_string()))
}
/// Put the lines recognised in [`Self::image`] on the page (or skip the page with the
/// error). A single engine's reading, which is all the web build's workers run, is
/// `StrategyOutcome { lines, ..Default::default() }`: every strategy reads that way when
/// there is no second engine.
pub fn place(&self, outcome: Result<StrategyOutcome, String>) -> OcrPage {
let page = self.page;
let outcome = match outcome {
Ok(o) => o,
Err(e) => return OcrPage { page, words: Vec::new(), already_on_page: 0, skipped: Some(e), notes: Vec::new() },
};
let (info, inverse, offset, scale) = (&self.info, self.inverse, self.offset, self.pixels_per_point);
let to_user = |x: f32, y: f32| {
// processed pixels → original raster pixels (the exact inverse) → past the
// region crop → page view space.
let (ox, oy) = inverse.apply(f64::from(x), f64::from(y));
let [u, v] = info.view_to_user((ox as f32 + offset[0]) / scale, (oy as f32 + offset[1]) / scale);
[u as f64, v as f64]
};
// Validate recognition geometry and compare against existing text in view space,
// undoing preprocessing and region offsets before testing overlap.
let (fresh, already): (Vec<_>, Vec<_>) = outcome.lines.iter().flat_map(|l| &l.words).filter(|w| box_writable(w)).partition(|w| {
let [l, t, r, b] = w.rect;
let bounds = [(l, t), (r, t), (r, b), (l, b)].into_iter().fold(
[f32::INFINITY, f32::INFINITY, f32::NEG_INFINITY, f32::NEG_INFINITY],
|mut bounds, (x, y)| {
let (x, y) = inverse.apply(f64::from(x), f64::from(y));
let x = (x as f32 + offset[0]) / scale;
let y = (y as f32 + offset[1]) / scale;
bounds[0] = bounds[0].min(x);
bounds[1] = bounds[1].min(y);
bounds[2] = bounds[2].max(x);
bounds[3] = bounds[3].max(y);
bounds
},
);
covered(bounds, &self.existing) < ALREADY_ON_PAGE
});
let words = fresh.into_iter().map(|w| PlacedWord::place(w, to_user)).collect::<Vec<_>>();
let mut notes = outcome.notes;
// Only visible editable output needs display glyphs. Searchable output preserves
// every Unicode character with empty Type3 programs and ToUnicode maps.
if self.output_mode == OutputMode::EditableText && !words.is_empty() {
let missing = pdfcraft_fonts::TextPlan::for_words(words.iter().map(|w| w.text.as_str())).missing();
if missing > 0 {
notes.push(format!("{missing} characters had no font that could show them and appear as ? in the output"));
}
}
OcrPage { page, words, already_on_page: already.len(), skipped: None, notes }
}
}
/// Whether a word's pixel box may reach the content stream: width and height (right − left,
@@ -1081,36 +1201,70 @@ pub fn recognize_file(
recognizers: &Recognizers,
progress: impl FnMut(usize, usize) -> bool,
) -> Result<FileResult, String> {
let mut s = Session::new();
let id = s.open(name, None, bytes.clone(), password).map_err(|e| e.to_string())?;
if let Some(why) = s.get(id).and_then(|d| d.read_only_reason.clone()) {
return Err(why);
match FileJob::open(name, bytes, password, settings)? {
Opened::Skipped(result) => Ok(result),
Opened::Read(mut file) => {
let job = file.job.take().ok_or("the document could not be read")?;
let pages = job.run(recognizers, progress);
file.finish(pages)
}
}
let job = s.ocr_job(id, &[], settings.clone()).ok_or("the document could not be read")?;
// The skip-files probe runs before any engine work: a file with text needs no engine at
// all, and the original bytes come back unchanged.
if settings.skip_text_files && !settings.force_ocr && job.has_text() {
let pages = job
.pages
.iter()
.map(|&page| OcrPage {
already_on_page: 0,
page,
words: Vec::new(),
skipped: Some("the file already contains text".into()),
notes: Vec::new(),
})
.collect();
return Ok(FileResult { bytes, pages, file_skipped: Some("the file already contains text".into()) });
}
/// [`recognize_file`] in two halves, for callers that read the pages themselves (the web build
/// reads them in Web Workers): [`FileJob::open`], read `job`, then [`FileJob::finish`].
pub struct FileJob {
session: Session,
id: DocId,
bytes: Arc<Vec<u8>>,
/// The pages to read (taken by the caller).
pub job: Option<OcrJob>,
}
/// What [`FileJob::open`] found: a file to read, or one the skip-files option passes over
/// (its result is final, and no engine is needed).
pub enum Opened {
Read(Box<FileJob>),
Skipped(FileResult),
}
impl FileJob {
pub fn open(name: &str, bytes: Arc<Vec<u8>>, password: Option<&str>, settings: OcrSettings) -> Result<Opened, String> {
let mut session = Session::new();
let id = session.open(name, None, bytes.clone(), password).map_err(|e| e.to_string())?;
if let Some(why) = session.get(id).and_then(|d| d.read_only_reason.clone()) {
return Err(why);
}
let job = session.ocr_job(id, &[], settings.clone()).ok_or("the document could not be read")?;
// The skip-files probe runs before any engine work: a file with text needs no engine at
// all, and the original bytes come back unchanged.
if settings.skip_text_files && !settings.force_ocr && job.has_text() {
let pages = job
.pages
.iter()
.map(|&page| OcrPage {
already_on_page: 0,
page,
words: Vec::new(),
skipped: Some("the file already contains text".into()),
notes: Vec::new(),
})
.collect();
return Ok(Opened::Skipped(FileResult { bytes, pages, file_skipped: Some("the file already contains text".into()) }));
}
Ok(Opened::Read(Box::new(FileJob { session, id, bytes, job: Some(job) })))
}
let pages = job.run(recognizers, progress);
if s.apply_ocr(id, &pages).map_err(|e| e.to_string())? == 0 {
return Ok(FileResult { bytes, pages, file_skipped: None });
/// Add the words found and save: the searchable file, or the original when nothing was found.
pub fn finish(mut self, pages: Vec<OcrPage>) -> Result<FileResult, String> {
if self.session.apply_ocr(self.id, &pages).map_err(|e| e.to_string())? == 0 {
return Ok(FileResult { bytes: self.bytes, pages, file_skipped: None });
}
// Checked like a save before the caller writes it (#2166).
let bytes = self.session.save_bytes(self.id).map_err(|e| e.to_string())?;
self.session.check_save(self.id, &bytes).map_err(|e| e.to_string())?;
Ok(FileResult { bytes, pages, file_skipped: None })
}
// Checked like a save before the caller writes it (#2166).
let bytes = s.save_bytes(id).map_err(|e| e.to_string())?;
s.check_save(id, &bytes).map_err(|e| e.to_string())?;
Ok(FileResult { bytes, pages, file_skipped: None })
}
#[cfg(test)]
@@ -1118,6 +1272,34 @@ mod tests {
use super::*;
use crate::export;
/// One 300 × 300 page drawing `content`, with Helvetica as `/F1`.
fn page(content: &str) -> Arc<Vec<u8>> {
let pdf = format!(
"%PDF-1.7\n1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj\n2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj\n\
3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >> endobj\n\
4 0 obj << /Length {} >> stream\n{content}\nendstream endobj\n\
5 0 obj << /Type /Font /Subtype /Type1 /BaseFont /Helvetica >> endobj\ntrailer << /Root 1 0 R >>\n%%EOF",
content.len()
);
Arc::new(pdf.into_bytes())
}
#[test]
fn a_stamp_drawn_as_outlines_leaves_a_scan_to_read() {
let stamp = page("/Span <</ActualText (APPROVED)>> BDC 100 40 60 20 re f EMC");
let mut r = PageRenderer::new(stamp, RenderConfig::default());
assert!(!has_own_text(&mut r, 0), "words only /ActualText names are no text of the page's own");
// They still count as on the page, so reading the scan doesn't add them twice.
let text = r.render(RenderRequest { page: 0, kind: RequestKind::Text, scale: 1.0, ..Default::default() }).text.unwrap();
assert_eq!(text.plain_text(), "APPROVED");
}
#[test]
fn printed_glyphs_count_as_own_text() {
let mut r = PageRenderer::new(page("BT /F1 12 Tf 20 150 Td (Total) Tj ET"), RenderConfig::default());
assert!(has_own_text(&mut r, 0));
}
/// #1028: a recognised word over the page's own text is that text again; one in an empty
/// part of the page (a picture) is new.
#[test]
@@ -1597,6 +1779,89 @@ mod tests {
assert_eq!(again.pages[0].skipped.as_deref(), Some("the page already contains text"));
}
/// The web build prepares pages with [`OcrJob::steps`] and has a worker read each image,
/// its lines coming back as JSON: the pages it ends with are the ones [`OcrJob::run`] gives,
/// with a region crop and preprocessing between the render and the engine.
#[test]
fn reading_page_by_page_matches_a_run() {
let mut s = Session::new();
let bytes = s.create_from_text("t", "One page\nanother page").unwrap();
let id = s.open("t.pdf", None, bytes, None).unwrap();
let settings = OcrSettings { force_ocr: true, deskew: true, binarize: true, region: Some([40.0, 30.0, 900.0, 700.0]), ..Default::default() };
let (r, fake) = fake_recognizers();
let run = s.ocr_job(id, &[], settings.clone()).unwrap().run(&r, |_, _| true);
assert!(run.iter().all(|p| p.skipped.is_none() && p.words.len() == 1), "{run:?}");
let mut steps = s.ocr_job(id, &[], settings).unwrap().steps();
assert_eq!((steps.done(), steps.total()), (0, run.len()));
let mut stepped = Vec::new();
while let Some(next) = steps.next_page() {
let shot = next.unwrap();
let json = pdfcraft_ocr::lines_to_json(&fake.recognize(shot.image().unwrap(), &pdfcraft_ocr::RecognizeOptions::default()).unwrap());
let lines = lines_from_json(&json, "fake").map_err(|e| e.to_string());
stepped.push(shot.place(lines.map(|lines| StrategyOutcome { lines, ..Default::default() })));
}
assert_eq!(steps.done(), run.len());
assert_eq!(stepped, run);
// An engine's failure skips the page with the reason, as in a run.
let mut steps = s.ocr_job(id, &[0], OcrSettings { force_ocr: true, ..Default::default() }).unwrap().steps();
let mut shot = steps.next_page().unwrap().unwrap();
let page = shot.place(Err("worker gone".into()));
assert_eq!(page.skipped.as_deref(), Some("worker gone"));
// Once the image is handed on, it can't be read here as well.
assert!(shot.take_image().is_some() && shot.image().is_none());
assert!(shot.read(&r).skipped.is_some());
assert!(steps.next_page().is_none());
}
/// A page the job doesn't read comes out of [`OcrSteps::next_page`] as skipped, with its
/// reason, before any engine is involved.
#[test]
fn page_by_page_skips_text_pages_without_an_engine() {
let mut s = Session::new();
let bytes = s.create_from_text("t", "A page with text").unwrap();
let id = s.open("t.pdf", None, bytes, None).unwrap();
let mut steps = s.ocr_job(id, &[], OcrSettings::default()).unwrap().steps();
let Some(Err(page)) = steps.next_page() else { panic!("the text page should be skipped") };
assert_eq!(page.skipped.as_deref(), Some("the page already contains text"));
}
/// The web build has no model files: a recogniser loaded from downloaded bytes is installed
/// and from then on is the engine every job reads with.
#[test]
fn an_installed_recognizer_is_available_and_handed_out() {
let Some(models) = Models::find().filter(|m| m.detection.ends_with(DETECTION_MODEL)) else {
eprintln!("skipped: ONNX OCR models not installed");
return;
};
let ocr = Ocr::from_bytes(std::fs::read(&models.detection).unwrap(), std::fs::read(&models.recognition).unwrap()).unwrap();
let installed = install(ocr);
assert!(installed.available());
assert!(available());
let r = recognizers(&OcrSettings::default()).unwrap();
assert!(Arc::ptr_eq(&r.primary, &installed), "the installed recogniser is the one handed out");
assert!(engines_status().iter().any(|e| e.id == "ocrs" && e.available));
}
/// The batch's halves: a file to read opens with its job, and [`FileJob::finish`] saves
/// the searchable copy; a file the skip-files option passes over is final on opening.
#[test]
fn a_file_job_reads_in_two_halves() {
let s = Session::new();
let bytes = s.create_from_text("t", "Already typed here").unwrap();
let (r, fake) = fake_recognizers();
let skip = OcrSettings { skip_text_files: true, ..Default::default() };
let Ok(Opened::Skipped(out)) = FileJob::open("t.pdf", bytes.clone(), None, skip) else { panic!("the file should be skipped") };
assert_eq!(out.file_skipped.as_deref(), Some("the file already contains text"));
let force = OcrSettings { force_ocr: true, ..Default::default() };
let Ok(Opened::Read(mut file)) = FileJob::open("t.pdf", bytes.clone(), None, force.clone()) else { panic!("the file should be read") };
let pages = file.job.take().unwrap().run(&r, |_, _| true);
let out = file.finish(pages).unwrap();
assert_eq!((fake.calls(), out.words(), out.file_skipped.clone()), (1, 1, None));
let whole = recognize_file("t.pdf", bytes, None, force, &r, |_, _| true).unwrap();
// The saved bytes carry a modification time, so two saves a second apart differ there alone.
assert_eq!((out.pages, out.file_skipped), (whole.pages, whole.file_skipped), "the same as reading it in one call");
}
/// Two inputs of one name (from different folders, or differing only in case) would
/// write the same output: both are found, each once.
#[test]
crates/engine/src/open.rs
+24 -8
View File
@@ -7,8 +7,8 @@ use pdfcraft_render::{DocInfo, OpenError, RenderConfig, RenderPool, inspect};
use crate::history::{Editor, Keys, TextBlocksCache};
use crate::{
AppearanceCache, DocId, Document, Session, TrustStore, XFA_SCRIPTS_OFF, XfaLayout, display_bytes, display_config, guard, info_fields, js,
note_warnings, recovered_render_password, render_threads, same_pages, signatures_of, xfa, xfa_layout, xfa_values_from_datasets,
AppearanceCache, DocId, Document, Session, StatusCaptions, TrustStore, XFA_SCRIPTS_OFF, XfaLayout, display_bytes, display_config, guard,
info_fields, js, note_warnings, recovered_render_password, render_threads, same_pages, signatures_of, xfa, xfa_layout, xfa_values_from_datasets,
};
/// Immutable session state needed while a document is prepared away from its owning thread.
@@ -17,6 +17,7 @@ pub struct OpenContext {
now: Option<i64>,
js_off: bool,
trust: Arc<TrustStore>,
captions: Arc<StatusCaptions>,
xfa_time_scale: u32,
}
@@ -54,7 +55,13 @@ impl OpenContext {
impl Session {
/// Snapshot the session state that affects open-time scripts, dates and signature trust.
pub fn open_context(&self) -> OpenContext {
OpenContext { now: self.now(), js_off: self.js_off, trust: self.trust.clone(), xfa_time_scale: self.xfa_time_scale.unwrap_or(1) }
OpenContext {
now: self.now(),
js_off: self.js_off,
trust: self.trust.clone(),
captions: self.captions.clone(),
xfa_time_scale: self.xfa_time_scale.unwrap_or(1),
}
}
/// Prepare a document without adding it to this session.
@@ -74,6 +81,14 @@ impl Session {
let id = DocId(self.next_id);
let mut document = prepared.document;
document.id = id;
// Trust or the interface language may have changed while it was prepared on another
// thread.
if !Arc::ptr_eq(&document.trust, &self.trust) {
document.captions = self.captions.clone();
Self::revalidate(&mut document, self.trust.clone());
} else if *document.captions != *self.captions {
Self::draw_captions(&mut document, self.captions.clone());
}
self.docs.push(document);
id
}
@@ -253,7 +268,7 @@ impl Session {
// XFA notes survive the document being re-read after edits.
let xfa_warnings: Vec<String> = if info.xfa.is_some() { info.warnings.clone() } else { Vec::new() };
let mut document =
Self::build_document(&context.trust, DocumentOpenData { name, path, bytes, info, structure: cos, render_password, password, xfa });
Self::build_document(context, DocumentOpenData { name, path, bytes, info, structure: cos, render_password, password, xfa });
// A script that ran away on open turns the form's scripts off for this document.
document.xfa_template = if ran_away { None } else { xfa_template };
document.js_output.append(script_output);
@@ -266,7 +281,7 @@ impl Session {
}
/// The last step of opening: build the document record and register it.
fn build_document(trust: &Arc<TrustStore>, data: DocumentOpenData<'_>) -> Document {
fn build_document(context: &OpenContext, data: DocumentOpenData<'_>) -> Document {
let DocumentOpenData { name, path, bytes, mut info, structure: cos, render_password, password, xfa } = data;
let config = RenderConfig { password: render_password.as_deref().map(Arc::from), ..Default::default() };
let (mut editor, read_only_reason) = match cos {
@@ -293,7 +308,7 @@ impl Session {
Err(_) => (None, Some("the document structure could not be read for editing".into())),
};
let sig_cache = Arc::new(pdfcraft_sign::DigestCache::default());
let signatures = editor.as_ref().map(|e| signatures_of(&e.cos, &bytes, trust, &sig_cache)).unwrap_or_default();
let mut signatures = editor.as_ref().map(|e| signatures_of(&e.cos, &bytes, &context.trust, &sig_cache)).unwrap_or_default();
// The inspection reads only `/Info`; a file stating its title or author only in XMP still
// shows them (#2373).
if let Some(e) = &editor {
@@ -303,7 +318,7 @@ impl Session {
}
}
}
let display = display_bytes(editor.as_mut(), &bytes);
let display = display_bytes(editor.as_mut(), &bytes, &mut signatures, &context.captions);
let renderer = RenderPool::new(display.clone(), render_threads(), display_config(&config));
let (form, adopted) = editor
.as_ref()
@@ -341,7 +356,8 @@ impl Session {
marker_scan: OnceLock::new(),
links: OnceLock::new(),
signatures: Arc::new(signatures),
trust: trust.clone(),
trust: context.trust.clone(),
captions: context.captions.clone(),
sig_cache,
editor,
config,
crates/engine/src/rich_copy.rs
+389
View File
@@ -0,0 +1,389 @@
//! Copy with formatting: the selected glyphs of a page as paragraphs of styled spans and an
//! HTML fragment ([`pdfcraft_export::clip_html`]) for the clipboard, next to the plain text.
//!
//! Glyphs come from the text layer (view space); style comes from the page's reading blocks,
//! which never mix styles, mapped into view space. A glyph takes the style of the block its
//! centre lies in; a glyph in no block is copied unstyled.
use pdfcraft_export::{ClipBuilder, ClipStyle, ClipText, clip_html};
use pdfcraft_render::{PageInfo, PageText};
use crate::EditError;
use crate::compare::{display_unit, weight_and_slant};
/// Block tests spent matching glyphs to blocks for one copy; past it the rest is unstyled.
const MAX_MATCH_STEPS: usize = 20_000_000;
/// How far outside a block's box (points) a glyph centre still belongs to it.
const SLACK: f32 = 0.5;
/// Selected text with its formatting.
#[derive(Clone, Debug, Default, PartialEq)]
pub struct RichCopy {
/// The paragraphs and spans.
pub text: ClipText,
/// The HTML fragment for the clipboard (empty when nothing was selected).
pub html: String,
/// Styles were left out: the document has no editor to read them, reading stopped at its
/// limits on this page, or matching glyphs to blocks ran out of steps.
pub styles_incomplete: bool,
/// A cap left part of the selection out of `text` and `html`; the plain text has all of it.
pub truncated: bool,
}
struct StyledBlock {
rect: [f32; 4],
style: ClipStyle,
}
impl crate::Document {
/// Selected `glyphs` of `page` (indices into `text`, in copy order) with their formatting.
/// Refused when the document does not allow copying.
pub fn rich_copy(&self, page: usize, text: &PageText, glyphs: &[usize]) -> Result<RichCopy, EditError> {
self.rich_copy_within(page, text, glyphs, MAX_MATCH_STEPS)
}
/// [`Self::rich_copy`] with `steps` block tests to match glyphs to blocks.
fn rich_copy_within(&self, page: usize, text: &PageText, glyphs: &[usize], mut steps: usize) -> Result<RichCopy, EditError> {
self.check_copying()?;
let (blocks, mut incomplete) = self.styled_blocks(page);
let mut nested = vec![None; blocks.len()];
let mut b = ClipBuilder::new();
// The previous glyph's line, block and box, and whether it was a hyphen.
let mut prev: Option<(usize, Option<usize>, [f32; 4], bool)> = None;
for &i in glyphs {
let (Some(g), Some(&line)) = (text.glyphs.get(i), text.line_of.get(i)) else { continue };
let hint = prev.and_then(|(_, b, _, _)| b).filter(|&k| !holds_smaller(&blocks, &mut nested, k, &mut steps));
let found = block_at(&blocks, g.rect, hint, &mut steps).unwrap_or_else(|| {
incomplete = true;
None
});
let line = usize::try_from(line).unwrap_or(usize::MAX);
if let Some((prev_line, prev_block, prev_rect, hyphen)) = prev {
if line != prev_line {
let style = |k: Option<usize>| k.and_then(|k| blocks.get(k)).map(|s| &s.style);
if found.is_none() && prev_block.is_none() {
b.line_break();
} else if found == prev_block || wraps(style(prev_block), style(found), prev_rect, g.rect) {
// A paragraph wrapping: one space, none after a hyphen.
if !hyphen {
b.space();
}
} else {
b.paragraph();
}
} else if text.space_before.get(i).copied().unwrap_or(false) {
b.space();
}
}
let style = found.and_then(|k| blocks.get(k)).map(|s| &s.style);
if !b.push(&g.text, style.unwrap_or(&ClipStyle::default())) {
break;
}
prev = Some((line, found, g.rect, g.text.ends_with('-')));
}
Ok(finish(b, incomplete))
}
/// `page`'s reading blocks in view space with their style, and whether reading stopped at
/// its limits.
fn styled_blocks(&self, page: usize) -> (Vec<StyledBlock>, bool) {
let Some(info) = self.info.pages.get(page) else { return (Vec::new(), false) };
let Some(editor) = self.editor.as_ref() else { return (Vec::new(), true) };
let Ok(reading) = pdfcraft_edit::reading_blocks_checked(&editor.cos, page) else { return (Vec::new(), true) };
let unit = display_unit(info);
let blocks = reading
.blocks
.into_iter()
.filter(|b| !b.text.trim().is_empty())
.map(|b| {
let (bold, italic) = weight_and_slant(&b.base_font);
let style = ClipStyle {
families: pdfcraft_export::font_family(&b.base_font),
size: b.size * unit,
bold: b.bold || bold,
italic: b.italic || italic,
color: b.color,
};
StyledBlock { rect: view_rect(info, b.rect), style }
})
.collect();
(blocks, reading.truncated)
}
}
/// A user-space box as a view-space box (points from the displayed page's top-left, y down).
fn view_rect(info: &PageInfo, r: [f64; 4]) -> [f32; 4] {
let a = info.user_to_view(r[0] as f32, r[1] as f32);
let b = info.user_to_view(r[2] as f32, r[3] as f32);
[a[0].min(b[0]), a[1].min(b[1]), a[0].max(b[0]), a[1].max(b[1])]
}
/// The copy `b` collected, with its HTML; `truncated` when a cap cut the text or the HTML.
fn finish(b: ClipBuilder, styles_incomplete: bool) -> RichCopy {
let text = b.finish();
let (html, cut) = clip_html(&text);
RichCopy { html, truncated: text.truncated || cut, text, styles_incomplete }
}
/// Whether the next line starts inside the same paragraph although its first glyph is in another
/// block: blocks split where the style changes, so a style change at a line's end is still a
/// wrap when the size stays and the lines sit no further apart than their own height allows.
/// Two blocks of the same style, a change of size, or a gap of over half a line are paragraphs.
fn wraps(prev: Option<&ClipStyle>, next: Option<&ClipStyle>, prev_rect: [f32; 4], rect: [f32; 4]) -> bool {
let (Some(prev), Some(next)) = (prev, next) else { return false };
let diff = (prev.size - next.size).abs();
if prev == next || diff.is_nan() || diff > 0.01 {
return false;
}
let height = (prev_rect[3] - prev_rect[1]).max(rect[3] - rect[1]);
// The next line starts just below the last one; a line far above it is another column.
let gap = rect[1] - prev_rect[3];
height > 0.0 && gap.is_finite() && gap >= -height / 2.0 && gap <= height / 2.0
}
/// Whether a block smaller than block `k` overlaps it, so a glyph inside `k` may belong to that
/// one instead. Worked out once per block (`nested` keeps it), at a step per block tested; when
/// the steps run out the answer is yes, so the full search decides.
fn holds_smaller(blocks: &[StyledBlock], nested: &mut [Option<bool>], k: usize, steps: &mut usize) -> bool {
if let Some(Some(known)) = nested.get(k) {
return *known;
}
let Some(outer) = blocks.get(k) else { return true };
let area = |r: &[f32; 4]| (r[2] - r[0]) * (r[3] - r[1]);
let overlaps = |r: &[f32; 4]| r[0] < outer.rect[2] && r[2] > outer.rect[0] && r[1] < outer.rect[3] && r[3] > outer.rect[1];
let mut found = false;
for (j, b) in blocks.iter().enumerate() {
if *steps == 0 {
return true;
}
*steps -= 1;
if j != k && overlaps(&b.rect) && area(&b.rect) < area(&outer.rect) {
found = true;
break;
}
}
if let Some(slot) = nested.get_mut(k) {
*slot = Some(found);
}
found
}
/// The block whose box holds the centre of `glyph` (view space): `hint` first (a block no
/// smaller one overlaps), then the smallest that holds it. Each block tested costs a step;
/// `None` when the steps ran out first.
fn block_at(blocks: &[StyledBlock], glyph: [f32; 4], hint: Option<usize>, steps: &mut usize) -> Option<Option<usize>> {
let (x, y) = ((glyph[0] + glyph[2]) / 2.0, (glyph[1] + glyph[3]) / 2.0);
let holds = |r: &[f32; 4]| x >= r[0] - SLACK && x <= r[2] + SLACK && y >= r[1] - SLACK && y <= r[3] + SLACK;
let area = |r: &[f32; 4]| (r[2] - r[0]) * (r[3] - r[1]);
if let Some(k) = hint
&& *steps > 0
{
*steps -= 1;
if blocks.get(k).is_some_and(|b| holds(&b.rect)) {
return Some(Some(k));
}
}
let mut best: Option<(usize, f32)> = None;
for (k, b) in blocks.iter().enumerate() {
if *steps == 0 {
return None;
}
*steps -= 1;
if holds(&b.rect) && best.is_none_or(|(_, a)| area(&b.rect) < a) {
best = Some((k, area(&b.rect)));
}
}
Some(best.map(|(k, _)| k))
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
use pdfcraft_export::{ClipBuilder, ClipStyle};
use crate::Session;
/// A heading in bold, a body paragraph of two lines with HTML-special characters, and a red
/// word on the body's first line.
fn styled_page() -> Vec<u8> {
let content = b"BT /F1 24 Tf 20 250 Td (Title) Tj ET \
BT /F2 12 Tf 20 200 Td (Body text) Tj ET \
BT 1 0 0 rg /F2 12 Tf 90 200 Td (red) Tj ET \
BT /F2 12 Tf 20 186 Td (a<b & c) Tj ET";
page_with(content)
}
/// A one-page PDF drawing `content` with `/F1` Helvetica-Bold and `/F2` Helvetica.
fn page_with(content: &[u8]) -> Vec<u8> {
pdfcraft_testkit::Pdf::new()
.obj("<< /Type /Catalog /Pages 2 0 R >>")
.obj("<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 300 300] >>")
.obj("<< /Type /Page /Parent 2 0 R /Contents 4 0 R /Resources << /Font << /F1 5 0 R /F2 6 0 R >> >> >>")
.stream("", content)
.obj("<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica-Bold >>")
.obj("<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>")
.build()
}
fn page_text(s: &Session, id: crate::DocId) -> Arc<pdfcraft_render::PageText> {
let doc = s.get(id).unwrap();
let mut r = pdfcraft_render::PageRenderer::new(doc.bytes.clone(), Default::default());
let request = pdfcraft_render::RenderRequest { page: 0, kind: pdfcraft_render::RequestKind::Text, scale: 1.0, ..Default::default() };
r.render(request).text.unwrap()
}
#[test]
fn copy_keeps_headings_body_and_colour_as_html() {
let mut s = Session::new();
let id = s.open("styled.pdf", None, Arc::new(styled_page()), None).unwrap();
let text = page_text(&s, id);
let all: Vec<usize> = (0..text.glyphs.len()).collect();
let copy = s.get(id).unwrap().rich_copy(0, &text, &all).unwrap();
let html = &copy.html;
assert!(!copy.styles_incomplete);
assert!(
html.starts_with("<p style=\"margin:0\"><span style=\"font-family:'Helvetica';font-size:24pt\"><strong>Title</strong></span></p>\n"),
"{html}"
);
assert!(html.contains("font-size:12pt\">Body text "), "{html}");
assert!(html.contains("font-size:12pt;color:#FF0000\">red</span>"), "{html}");
assert!(html.contains("a&lt;b &amp; c"), "{html}");
assert!(!html.contains("<b &"), "{html}");
assert_eq!(copy.text.paragraphs.len(), html.matches("<p ").count());
assert!(copy.text.paragraphs.len() >= 2, "{html}");
// The heading and the body are separate paragraphs; the red word shares the body's.
let body = copy.text.paragraphs.iter().find(|p| p.iter().any(|s| s.text.contains("Body"))).unwrap();
assert!(body.iter().any(|s| s.text == "red" && s.style.color == [1.0, 0.0, 0.0]), "{body:?}");
// A part of the selection: only those glyphs.
let part = s.get(id).unwrap().rich_copy(0, &text, &all[..5]).unwrap();
assert_eq!(part.text.paragraphs.len(), 1);
assert!(part.html.contains("<strong>Title</strong>"), "{}", part.html);
// Nothing selected, and indices past the text: an empty copy, not a panic.
let none = s.get(id).unwrap().rich_copy(0, &text, &[]).unwrap();
assert!(none.html.is_empty() && none.text.paragraphs.is_empty());
let past = s.get(id).unwrap().rich_copy(7, &text, &[usize::MAX, text.glyphs.len()]).unwrap();
assert!(past.html.is_empty());
}
#[test]
fn running_out_of_match_steps_copies_the_rest_unstyled() {
let mut s = Session::new();
let id = s.open("styled.pdf", None, Arc::new(styled_page()), None).unwrap();
let text = page_text(&s, id);
let all: Vec<usize> = (0..text.glyphs.len()).collect();
let doc = s.get(id).unwrap();
let full = doc.rich_copy(0, &text, &all).unwrap();
// Enough for the first glyphs only: the text is all there, the rest has no style.
let short = doc.rich_copy_within(0, &text, &all, 8).unwrap();
assert!(short.styles_incomplete);
let plain = |c: &super::RichCopy| c.text.paragraphs.iter().flatten().map(|s| s.text.clone()).collect::<String>();
assert_eq!(plain(&short).replace([' ', '\n'], ""), plain(&full).replace([' ', '\n'], ""));
let last = short.text.paragraphs.last().and_then(|p| p.last()).unwrap();
assert_eq!(last.style, pdfcraft_export::ClipStyle::default(), "{last:?}");
assert!(short.html.starts_with("<p style=\"margin:0\"><span style=\"font-family:'Helvetica';font-size:24pt\"><strong>T"), "{}", short.html);
// No steps at all: nothing styled, still no panic.
let none = doc.rich_copy_within(0, &text, &all, 0).unwrap();
assert!(none.styles_incomplete && !none.html.contains("font-size"), "{}", none.html);
// A page without blocks to match needs no steps.
let page_less = doc.rich_copy_within(5, &text, &all, 0).unwrap();
assert!(!page_less.styles_incomplete);
}
fn copy_all(s: &Session, id: crate::DocId) -> super::RichCopy {
let text = page_text(s, id);
let all: Vec<usize> = (0..text.glyphs.len()).collect();
s.get(id).unwrap().rich_copy(0, &text, &all).unwrap()
}
#[test]
fn a_style_change_at_a_line_end_stays_in_its_paragraph() {
let content = b"BT /F2 12 Tf 20 200 Td (plain then) Tj ET \
BT /F1 12 Tf 85 200 Td (bold) Tj ET \
BT /F2 12 Tf 20 186 Td (continues here) Tj ET \
BT /F2 12 Tf 20 120 Td (Another paragraph) Tj ET";
let mut s = Session::new();
let id = s.open("wrap.pdf", None, Arc::new(page_with(content)), None).unwrap();
let copy = copy_all(&s, id);
let text = |p: &Vec<pdfcraft_export::ClipSpan>| p.iter().map(|s| s.text.as_str()).collect::<String>();
let paragraphs: Vec<String> = copy.text.paragraphs.iter().map(text).collect();
assert!(paragraphs.iter().any(|p| p.contains("bold") && p.contains("continues")), "{paragraphs:?}");
assert!(paragraphs.iter().any(|p| p.contains("Another") && !p.contains("continues")), "a wide gap is still a paragraph: {paragraphs:?}");
}
#[test]
fn a_style_change_into_the_next_column_starts_a_paragraph() {
let content = b"BT /F2 12 Tf 20 200 Td (left top) Tj ET \
BT /F2 12 Tf 20 186 Td (left end) Tj ET \
BT /F1 12 Tf 170 200 Td (Right head) Tj ET \
BT /F2 12 Tf 170 186 Td (right body) Tj ET";
let mut s = Session::new();
let id = s.open("columns.pdf", None, Arc::new(page_with(content)), None).unwrap();
let copy = copy_all(&s, id);
let text = |p: &Vec<pdfcraft_export::ClipSpan>| p.iter().map(|s| s.text.as_str()).collect::<String>();
let paragraphs: Vec<String> = copy.text.paragraphs.iter().map(text).collect();
assert!(paragraphs.iter().any(|p| p.contains("Right")), "{paragraphs:?}");
assert!(!paragraphs.iter().any(|p| p.contains("left end") && p.contains("Right")), "{paragraphs:?}");
}
#[test]
fn a_styled_word_inside_a_paragraph_keeps_its_style() {
let content = b"BT /F2 12 Tf 20 200 Td (first row of the text) Tj ET \
BT /F2 12 Tf 20 186 Td (second) Tj ET \
BT /F1 12 Tf 62 186 Td (bold) Tj ET \
BT /F2 12 Tf 92 186 Td (row) Tj ET";
let mut s = Session::new();
let id = s.open("nested.pdf", None, Arc::new(page_with(content)), None).unwrap();
let copy = copy_all(&s, id);
let spans: Vec<_> = copy.text.paragraphs.iter().flatten().collect();
assert!(spans.iter().any(|s| s.text.contains("bold") && s.style.bold), "{spans:?}");
assert!(spans.iter().any(|s| s.text.contains("second") && !s.style.bold), "{spans:?}");
}
#[test]
fn a_capped_copy_is_marked_truncated() {
let (plain, bold) = (ClipStyle::default(), ClipStyle { bold: true, ..ClipStyle::default() });
let mut b = ClipBuilder::new();
for k in 0..=pdfcraft_export::MAX_CLIP_SPANS {
b.push("x", if k % 2 == 0 { &plain } else { &bold });
}
let copy = super::finish(b, false);
assert!(copy.truncated && copy.text.truncated);
let mut b = ClipBuilder::new();
b.push("x", &plain);
assert!(!super::finish(b, false).truncated);
}
#[test]
fn without_an_editor_the_styles_are_incomplete() {
let mut s = Session::new();
let id = s.open("styled.pdf", None, Arc::new(styled_page()), None).unwrap();
let text = page_text(&s, id);
let all: Vec<usize> = (0..text.glyphs.len()).collect();
s.doc_mut(id).unwrap().editor = None;
let copy = s.get(id).unwrap().rich_copy(0, &text, &all).unwrap();
assert!(copy.styles_incomplete && !copy.html.contains("font-size"), "{}", copy.html);
assert!(!copy.truncated);
}
#[test]
fn copy_with_formatting_is_refused_when_copying_is_withheld() {
let mut cos = pdfcraft_cos::Document::open(Arc::new(styled_page())).unwrap();
cos.set_encryption(&pdfcraft_cos::NewEncryption {
algorithm: pdfcraft_cos::Algorithm::Aes256,
user_password: "",
owner_password: "owner",
permissions: 0b0100, // print only
encrypt_metadata: true,
seed: [7; 32],
})
.unwrap();
let bytes = Arc::new(pdfcraft_cos::write_full(&cos, &Default::default()).unwrap());
let mut s = Session::new();
let id = s.open("locked.pdf", None, bytes, None).unwrap();
let text = pdfcraft_render::PageText { glyphs: Vec::new(), line_of: Vec::new(), space_before: Vec::new() };
let err = s.get(id).unwrap().rich_copy(0, &text, &[]).unwrap_err();
assert!(matches!(err, crate::EditError::NotPermitted(_)), "{err:?}");
}
}
crates/engine/src/session.rs
+7 -5
View File
@@ -11,11 +11,11 @@ use crate::edit::{EditCtx, check_permission, run_edit};
use crate::history::{Keys, MAX_UNDO, Scope, keys_after, scope_of, uses_scripts};
use crate::{
BookmarkPage, BookmarkRow, CombineSource, Combined, DataFormat, DocId, Document, Edit, EditError, Hidden, ImageResolution,
LOW_RESOLUTION_PRINT_DPI, Label, ObjectKind, OptimizeReport, RecoverySnapshot, SignOptions, SourceKind, SplitPart, SummarySort, TrustStore,
XFA_REMOVED_PREFIX, XFA_SCRIPTS_OFF, apply_layer_state, comment_list, comment_summary, decode_text, display_bytes, display_config, guard,
info_fields, js, local_utc_offset, note_warnings, open_source, open_source_with, optimize, outline_rows, parse_ymd, print, print_raster,
render_threads, scan_colours, set_default_layer_state, shown_under_redactions, signatures_of, source_kind, sync_xfa_state, undrawn_notices,
use_layer_choices, writable, xfa, xfa_sync_datasets,
LOW_RESOLUTION_PRINT_DPI, Label, ObjectKind, OptimizeReport, RecoverySnapshot, SignOptions, SourceKind, SplitPart, StatusCaptions, SummarySort,
TrustStore, XFA_REMOVED_PREFIX, XFA_SCRIPTS_OFF, apply_layer_state, carry_verdict_marks, comment_list, comment_summary, decode_text,
display_bytes, display_config, guard, info_fields, js, local_utc_offset, note_warnings, open_source, open_source_with, optimize, outline_rows,
parse_ymd, print, print_raster, render_threads, scan_colours, set_default_layer_state, shown_under_redactions, signatures_of, source_kind,
sync_xfa_state, undrawn_notices, use_layer_choices, writable, xfa, xfa_sync_datasets,
};
mod editing;
@@ -36,6 +36,8 @@ pub struct Session {
utc_offset: Option<fn() -> i64>,
/// Certificates trusted for signing (Acrobat: Trusted Certificates).
pub(crate) trust: Arc<TrustStore>,
/// The words signature status layers are drawn with, in the interface language.
pub(crate) captions: Arc<StatusCaptions>,
/// Preferences ▸ JavaScript ▸ Enable Acrobat JavaScript, inverted (on by default).
pub(crate) js_off: bool,
/// Preferences ▸ Date format, when not the default (see [`dates`]).
crates/engine/src/session/editing.rs
+39 -12
View File
@@ -2,9 +2,9 @@
use super::check_refresh_panic;
use super::{
Arc, DocId, Document, Edit, EditCtx, EditError, Label, MAX_UNDO, ObjectKind, OnceLock, RenderPool, SaveOptions, Scope, Session,
XFA_REMOVED_PREFIX, XFA_SCRIPTS_OFF, check_permission, comment_list, display_bytes, display_config, guard, info_fields, inspect, js, keys_after,
note_warnings, render_threads, run_edit, scan_colours, scope_of, shown_under_redactions, signatures_of, sync_xfa_state, undrawn_notices,
uses_scripts, write_incremental, xfa, xfa_sync_datasets,
XFA_REMOVED_PREFIX, XFA_SCRIPTS_OFF, carry_verdict_marks, check_permission, comment_list, display_bytes, display_config, guard, info_fields,
inspect, js, keys_after, note_warnings, render_threads, run_edit, scan_colours, scope_of, shown_under_redactions, signatures_of, sync_xfa_state,
undrawn_notices, uses_scripts, write_incremental, xfa, xfa_sync_datasets,
};
use crate::{EditNotice, MarkKind};
@@ -20,6 +20,19 @@ fn deletes_pages(edit: &Edit) -> bool {
impl Session {
/// Apply an edit. On success the previous state is undoable and the view data is refreshed.
pub fn apply(&mut self, id: DocId, edit: Edit) -> Result<(), EditError> {
self.apply_reporting(id, edit).map(|_| ())
}
/// Edit a PDF ▸ Edit text: make the grouped text on `page` (0-based) editable, as one undoable
/// [`Edit::PrepareText`], and say what it prepared, which fonts' maps it repaired and which
/// groups it left as they were (and why). The counts are this edit's own, not a preview's.
pub fn prepare_text(&mut self, id: DocId, page: usize) -> Result<pdfcraft_edit::text::forms::Preparation, EditError> {
self.apply_reporting(id, Edit::PrepareText { page })?
.ok_or_else(|| EditError::Invalid("preparing grouped text gave no result; the document was not changed".into()))
}
/// [`Session::apply`], giving back what an [`Edit::PrepareText`] did (`None` for other edits).
fn apply_reporting(&mut self, id: DocId, edit: Edit) -> Result<Option<pdfcraft_edit::text::forms::Preparation>, EditError> {
let now = self.now();
let today = self.today();
let js_off = self.js_off;
@@ -61,7 +74,7 @@ impl Session {
};
// Nothing to bake: leave undo history and the dirty flag alone.
if matches!(&edit, Edit::FlattenFillSign) && !pdfcraft_annot::has_visible_fill_sign(&editor.cos) {
return Ok(());
return Ok(None);
}
let mut next = editor.cos.clone();
if !js_off && uses_scripts(&edit) {
@@ -228,7 +241,7 @@ impl Session {
doc.xfa_scripts_off = true;
doc.js_output.errors.push(XFA_SCRIPTS_OFF.into());
}
Ok(())
Ok(cx.prepared)
}
pub fn undo(&mut self, id: DocId) -> Result<Label, EditError> {
@@ -342,7 +355,14 @@ impl Session {
#[cfg(test)]
check_refresh_panic();
if !doc.signatures.is_empty() {
let signatures = signatures_of(&editor.cos, &bytes, &doc.trust, &doc.sig_cache);
let mut signatures = signatures_of(&editor.cos, &bytes, &doc.trust, &doc.sig_cache);
// The display copy is kept for a metadata edit, and so are its marks, unless the
// edit changed a verdict (a certification can forbid it): the page then draws the
// new one.
if !carry_verdict_marks(&doc.signatures, &mut signatures) {
doc.display = display_bytes(Some(editor), &bytes, &mut signatures, &doc.captions);
doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config));
}
doc.signatures = Arc::new(signatures);
}
doc.bytes = bytes;
@@ -386,8 +406,8 @@ impl Session {
}
doc.info.file_size = bytes.len();
doc.form = form;
let signatures = if doc.signatures.is_empty() { Vec::new() } else { signatures_of(&editor.cos, &bytes, &doc.trust, &doc.sig_cache) };
doc.display = display_bytes(Some(editor), &bytes);
let mut signatures = if doc.signatures.is_empty() { Vec::new() } else { signatures_of(&editor.cos, &bytes, &doc.trust, &doc.sig_cache) };
doc.display = display_bytes(Some(editor), &bytes, &mut signatures, &doc.captions);
doc.signatures = Arc::new(signatures);
doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config));
doc.bytes = bytes;
@@ -444,8 +464,8 @@ impl Session {
doc.marker_scan = OnceLock::new();
}
doc.info.file_size = bytes.len();
let signatures = if doc.signatures.is_empty() { Vec::new() } else { signatures_of(&editor.cos, &bytes, &doc.trust, &doc.sig_cache) };
doc.display = display_bytes(Some(editor), &bytes);
let mut signatures = if doc.signatures.is_empty() { Vec::new() } else { signatures_of(&editor.cos, &bytes, &doc.trust, &doc.sig_cache) };
doc.display = display_bytes(Some(editor), &bytes, &mut signatures, &doc.captions);
doc.signatures = Arc::new(signatures);
doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config));
doc.bytes = bytes;
@@ -467,6 +487,13 @@ impl Session {
doc.renderer = RenderPool::new_inline(Arc::new(Vec::new()), doc.config.clone());
doc.display = original.clone();
doc.bytes = original;
// The page shows the original's own artwork again, so nothing counts as drawn, even if
// the refresh that follows fails.
if doc.signatures.iter().any(|s| s.verdict_shown) {
for sig in Arc::make_mut(&mut doc.signatures) {
sig.verdict_shown = false;
}
}
#[cfg(test)]
check_refresh_panic();
}
@@ -528,8 +555,8 @@ impl Session {
if invalidate_links {
doc.links = OnceLock::new();
}
let signatures = signatures_of(&editor.cos, &bytes, &doc.trust, &doc.sig_cache);
doc.display = display_bytes(Some(editor), &bytes);
let mut signatures = signatures_of(&editor.cos, &bytes, &doc.trust, &doc.sig_cache);
doc.display = display_bytes(Some(editor), &bytes, &mut signatures, &doc.captions);
doc.signatures = Arc::new(signatures);
doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config));
doc.bytes = bytes;
crates/engine/src/session/files.rs
+5
View File
@@ -170,6 +170,11 @@ impl Session {
if !doc.config.layers.is_empty() {
set_default_layer_state(&mut cos, &doc.info.layers);
}
// Signature states as the screen shows them, so not when its display copy could not be
// written. This copy is only printed, never saved.
if doc.signatures.iter().any(|s| s.verdict_shown) {
let _ = pdfcraft_sign::show_status_layers(&mut cos, &mut doc.signatures.to_vec(), &doc.captions);
}
pdfcraft_print::impose(&cos, settings).map_err(|e| EditError::Print(e.to_string()))
}
crates/engine/src/session/save.rs
+125 -12
View File
@@ -1,6 +1,6 @@
use super::{
Arc, DocId, EditError, Keys, OpenError, RecoverySnapshot, RenderPool, SaveOptions, Session, SignOptions, TrustStore, display_bytes,
display_config, guard, render_threads, signatures_of, writable, write_full, write_incremental, xfa_sync_datasets,
Arc, DocId, Document, EditError, Keys, OpenError, RecoverySnapshot, RenderPool, SaveOptions, Session, SignOptions, StatusCaptions, TrustStore,
display_bytes, display_config, guard, render_threads, signatures_of, writable, write_full, write_incremental, xfa_sync_datasets,
};
impl Session {
@@ -242,17 +242,55 @@ impl Session {
pub fn set_trusted_certificates(&mut self, certs: Vec<pdfcraft_sign::Certificate>) {
self.trust = Arc::new(TrustStore { certs });
for doc in &mut self.docs {
doc.trust = self.trust.clone();
let Some(editor) = doc.editor.as_mut() else { continue };
// Keep verdict refresh and renderer refresh together. Viewer caches are
// invalidated by the UI trust path; trust alone changes no document artwork.
let signatures = if doc.signatures.is_empty() { Vec::new() } else { signatures_of(&editor.cos, &doc.bytes, &doc.trust, &doc.sig_cache) };
doc.display = display_bytes(Some(editor), &doc.bytes);
doc.signatures = Arc::new(signatures);
doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config));
Self::revalidate(doc, self.trust.clone());
}
}
/// Validate `doc`'s signatures against `trust`, and rebuild its display copy and renderer
/// with them: the verdicts choose what the copy draws. An unsigned document has nothing to
/// validate and keeps both. Viewer caches are invalidated by the UI trust path.
pub(crate) fn revalidate(doc: &mut Document, trust: Arc<TrustStore>) {
doc.trust = trust;
if doc.signatures.is_empty() {
return;
}
let Some(editor) = doc.editor.as_mut() else { return };
let mut signatures = signatures_of(&editor.cos, &doc.bytes, &doc.trust, &doc.sig_cache);
doc.display = display_bytes(Some(editor), &doc.bytes, &mut signatures, &doc.captions);
doc.signatures = Arc::new(signatures);
doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config));
}
/// The words the caption of a named-layer signature appearance is drawn with
/// ([`pdfcraft_sign::show_status_layers`]), in the interface language. When they change, every
/// open document's display copy and renderer are rebuilt; like trust, this is not an edit.
/// Returns whether they changed.
pub fn set_signature_captions(&mut self, captions: StatusCaptions) -> bool {
if *self.captions == captions {
return false;
}
self.captions = Arc::new(captions);
for doc in &mut self.docs {
Self::draw_captions(doc, self.captions.clone());
}
true
}
/// Draw `doc`'s signature status layers with `captions`: the display copy and renderer are
/// rebuilt when it draws a signature's state (working file, generation and dirty unchanged).
/// Whether a state is drawn does not depend on the words: a caption that can't be drawn in
/// them is drawn in English.
pub(crate) fn draw_captions(doc: &mut Document, captions: Arc<StatusCaptions>) {
doc.captions = captions;
if !doc.signatures.iter().any(|s| s.verdict_shown) {
return;
}
let Some(editor) = doc.editor.as_mut() else { return };
let signatures = Arc::make_mut(&mut doc.signatures);
doc.display = display_bytes(Some(editor), &doc.bytes, signatures, &doc.captions);
doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config));
}
/// The signing time as a PDF date in local time with its offset (`D:…+02'00'`).
pub fn signing_date(&self) -> String {
let offset = self.local_offset();
@@ -276,7 +314,7 @@ impl Session {
opts.date = self.signing_date();
}
// A typed Password-field value is not signed into the file (#3926), as it isn't saved.
let cos = writable(&editor.cos)?;
let cos = drawn_for_first_signature(writable(&editor.cos)?);
Ok(Arc::new(pdfcraft_sign::sign(&cos, id, &opts)?))
}
@@ -294,7 +332,7 @@ impl Session {
if opts.date.is_empty() {
opts.date = self.signing_date();
}
let cos = writable(&editor.cos)?;
let cos = drawn_for_first_signature(writable(&editor.cos)?);
let signed = Arc::new(pdfcraft_sign::sign_with_timestamp(&cos, id, &opts, tsa)?);
// Checked like a save before anyone writes it (#2166).
self.check_save(doc, &signed)?;
@@ -336,3 +374,78 @@ impl Session {
Ok(())
}
}
/// `cos` with its form fields drawn, when the signature about to be made is the document's first
/// and the form sets `/NeedAppearances` (#5939). That signature removes the flag, after which
/// viewers show each field's stored appearance; a filled field whose stored appearance draws no
/// text (an empty placeholder such as `/Tx BMC EMC`, or a stale look) would go blank. So every
/// text and choice field is drawn from its value first, as the document is displayed
/// ([`pdfcraft_forms::with_field_appearances`]), and the signature covers that look.
///
/// One exception: where PdfCraft's drawing would show a character as `?` (the field's font can't
/// hold it) and the field's own appearance draws text, the field keeps its own appearance, which
/// the program that filled it drew with a font that has the character. A field whose own
/// appearance draws no text gets PdfCraft's drawing, `?` and all. Later signatures change
/// nothing here: redrawing would count as a change to the earlier signatures.
fn drawn_for_first_signature(cos: std::borrow::Cow<'_, pdfcraft_cos::Document>) -> std::borrow::Cow<'_, pdfcraft_cos::Document> {
use pdfcraft_forms::FieldKind;
if !pdfcraft_forms::need_appearances(&cos) || pdfcraft_sign::has_signatures(&cos) {
return cos;
}
let Some(mut drawn) = pdfcraft_forms::with_field_appearances(&cos) else { return cos };
for f in pdfcraft_forms::fields(&cos) {
if !matches!(f.kind, FieldKind::Text | FieldKind::Combo | FieldKind::List) {
continue;
}
for w in &f.widgets {
let own = cos.get(w.obj).as_dict().filter(|d| draws_text(&cos, d)).and_then(|d| d.get(b"AP").cloned());
// `.1`: the first character the drawing showed as `?`.
if let Some(ap) = own
&& pdfcraft_forms::appearance::field_appearance_drawn(&cos, &f, w, &f.value, true).1.is_some()
{
drawn.update_dict(w.obj, |d| d.set(b"AP".to_vec(), ap)).ok();
}
}
}
std::borrow::Cow::Owned(drawn)
}
/// Whether the widget's normal appearance stream (`/AP /N`) shows text, itself or through the
/// form XObjects it paints (a background-only one doesn't count).
fn draws_text(cos: &pdfcraft_cos::Document, widget: &pdfcraft_cos::Dict) -> bool {
let ap = widget.get(b"AP").map(|ap| cos.resolve(ap));
let normal = ap.as_deref().and_then(|ap| ap.as_dict()).and_then(|ap| ap.get(b"N")).map(|n| cos.resolve(n));
let Some(pdfcraft_cos::Object::Stream(s)) = normal.as_deref() else { return false };
stream_draws_text(cos, s, &mut 16)
}
/// Whether the form stream `s` shows text, following `Do` into its form XObjects. `budget` caps
/// the streams opened along the way; running out (deep nesting, a cycle, many XObjects) counts
/// as text, so the field keeps its own appearance as it would without the look inside.
fn stream_draws_text(cos: &pdfcraft_cos::Document, s: &pdfcraft_cos::Stream, budget: &mut u8) -> bool {
let Ok(data) = s.decoded() else { return false };
let resources = s.dict.get(b"Resources").map(|r| cos.resolve(r));
let xobjects = resources.as_deref().and_then(|r| r.as_dict()).and_then(|r| r.get(b"XObject")).map(|x| cos.resolve(x));
for op in pdfcraft_content::ops(&data) {
match op.op.as_slice() {
b"Tj" | b"TJ" | b"'" | b"\"" => return true,
b"Do" => {
let name = op.operands.first().and_then(|n| n.as_name());
let x = name.and_then(|n| xobjects.as_deref().and_then(|x| x.as_dict()).and_then(|x| x.get(n))).map(|x| cos.resolve(x));
let Some(pdfcraft_cos::Object::Stream(f)) = x.as_deref() else { continue };
if f.dict.name(b"Subtype") != Some(b"Form") {
continue;
}
if *budget == 0 {
return true;
}
*budget -= 1;
if stream_draws_text(cos, f, budget) {
return true;
}
}
_ => {}
}
}
false
}
crates/engine/src/tests.rs
+816 -1
View File
@@ -1,6 +1,6 @@
use std::sync::Arc;
use super::document::is_placeholder_title;
use super::document::{is_placeholder_title, write_display_copy};
use super::edit::{EditCtx, check_permission, run_edit};
use super::*;
use pdfcraft_render::{Layer, LayerOp, OpenError, ReadingColors};
@@ -2699,6 +2699,126 @@ fn a_signed_file_carries_no_password_field_value() {
assert!(!appears_in_pdf(&signed, "*******"), "nor the typed length");
}
/// A form with `/NeedAppearances true` whose filled fields have stored appearances that draw no
/// text: `name` an empty placeholder, `town` one too with a value Helvetica can't hold (`ň`),
/// `city` such a value with its own appearance that draws it (#5939), `street` no appearance at
/// all, `river` a value Helvetica can't hold (`ř`) with an appearance that paints only a
/// background form XObject, and `lake` such a value drawn by a form XObject its appearance
/// paints.
fn need_appearances_form() -> Vec<u8> {
let stream =
|content: &str| format!("<< /Type /XObject /Subtype /Form /BBox [0 0 160 20] /Length {} >>\nstream\n{content}\nendstream", content.len());
let objs = [
"<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [4 0 R 5 0 R 6 0 R 11 0 R 12 0 R 15 0 R] /NeedAppearances true /DA (/Helv 0 Tf 0 g) /DR << /Font << /Helv 7 0 R >> >> >> >>".to_string(),
"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 200 300] >>".into(),
"<< /Type /Page /Parent 2 0 R /Annots [4 0 R 5 0 R 6 0 R 11 0 R 12 0 R 15 0 R] >>".into(),
"<< /Type /Annot /Subtype /Widget /FT /Tx /T (name) /V (Ada Lovelace) /Rect [20 250 180 270] /P 3 0 R /AP << /N 8 0 R >> >>".into(),
"<< /Type /Annot /Subtype /Widget /FT /Tx /T (town) /V <FEFF0050006C007A00650148> /Rect [20 200 180 220] /P 3 0 R /AP << /N 9 0 R >> >>".into(),
"<< /Type /Annot /Subtype /Widget /FT /Tx /T (city) /V <FEFF00420072006E006F00200159> /Rect [20 150 180 170] /P 3 0 R /AP << /N 10 0 R >> >>".into(),
"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>".into(),
stream("/Tx BMC EMC"),
stream("/Tx BMC EMC"),
stream("/Tx BMC BT /Helv 10 Tf 2 5 Td (Brno r) Tj ET EMC"),
"<< /Type /Annot /Subtype /Widget /FT /Tx /T (street) /V (Main Street) /Rect [20 100 180 120] /P 3 0 R >>".into(),
"<< /Type /Annot /Subtype /Widget /FT /Tx /T (river) /V <FEFF004F0159006500630068> /Rect [20 50 180 70] /P 3 0 R /AP << /N 13 0 R >> >>".into(),
"<< /Type /XObject /Subtype /Form /BBox [0 0 160 20] /Resources << /XObject << /Bg 14 0 R >> >> /Length 10 >>\nstream\nq /Bg Do Q\nendstream".into(),
stream("0.9 g 0 0 160 20 re f"),
"<< /Type /Annot /Subtype /Widget /FT /Tx /T (lake) /V <FEFF004F0159006500630068> /Rect [20 20 180 40] /P 3 0 R /AP << /N 16 0 R >> >>".into(),
"<< /Type /XObject /Subtype /Form /BBox [0 0 160 20] /Resources << /XObject << /Fm 17 0 R >> >> /Length 10 >>\nstream\nq /Fm Do Q\nendstream".into(),
stream("BT /Helv 10 Tf 2 5 Td (Orech) Tj ET"),
];
let mut out = b"%PDF-1.7\n".to_vec();
let mut offsets = Vec::new();
for (i, o) in objs.iter().enumerate() {
offsets.push(out.len());
out.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes());
}
let xref = out.len();
out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes());
for o in offsets {
out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes());
}
out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes());
out
}
/// The decoded normal appearance of the widget `num` in `doc`.
fn normal_appearance(doc: &pdfcraft_cos::Document, num: u32) -> String {
let widget = doc.get(pdfcraft_cos::ObjRef::new(num, 0));
let ap = doc.resolve(widget.as_dict().unwrap().get(b"AP").unwrap());
let n = doc.resolve(ap.as_dict().unwrap().get(b"N").unwrap());
let pdfcraft_cos::Object::Stream(s) = &*n else { panic!("{n:?}") };
String::from_utf8_lossy(&s.decoded().unwrap()).into_owned()
}
/// #5939 review: the first signature removes `/NeedAppearances`, so the fields are drawn from
/// their values first; without that, `name` and `town` would show their empty placeholders. A
/// field whose own appearance draws its value keeps it rather than get a `?` for `ř`.
#[test]
fn the_first_signature_draws_the_fields_need_appearances_left_to_the_viewer() {
let p12 = std::fs::read(concat!(env!("CARGO_MANIFEST_DIR"), "/../sign/tests/data/ec-p256.p12")).unwrap();
let digital_id = sign::pkcs12::open(&p12, "test").unwrap();
// After the test certificate's start date (2026-10-02).
let mut s = Session::new().with_clock(|| 1_800_000_000);
let id = s.open("form.pdf", None, Arc::new(need_appearances_form()), None).unwrap();
let signed = s.sign(id, &digital_id, SignOptions { rect: None, ..SignOptions::default() }).unwrap();
let doc = pdfcraft_cos::Document::open(signed.clone()).unwrap();
assert!(!pdfcraft_forms::need_appearances(&doc));
assert!(normal_appearance(&doc, 4).contains("(Ada Lovelace) Tj"), "{}", normal_appearance(&doc, 4));
assert!(normal_appearance(&doc, 5).contains("(Plze?) Tj"), "{}", normal_appearance(&doc, 5));
assert!(normal_appearance(&doc, 6).contains("(Brno r) Tj"), "{}", normal_appearance(&doc, 6));
assert!(normal_appearance(&doc, 11).contains("(Main Street) Tj"), "{}", normal_appearance(&doc, 11));
// Its own appearance only paints a background, so `river` gets the drawing, `?` and all.
assert!(normal_appearance(&doc, 12).contains("(O?ech) Tj"), "{}", normal_appearance(&doc, 12));
let city = |d: &pdfcraft_cos::Document| d.get(pdfcraft_cos::ObjRef::new(6, 0)).as_dict().unwrap().get(b"AP").cloned();
let original = pdfcraft_cos::Document::open(Arc::new(need_appearances_form())).unwrap();
assert_eq!(city(&doc), city(&original), "city keeps its own appearance");
let lake = |d: &pdfcraft_cos::Document| d.get(pdfcraft_cos::ObjRef::new(15, 0)).as_dict().unwrap().get(b"AP").cloned();
assert_eq!(lake(&doc), lake(&original), "lake keeps the appearance its form XObject draws");
s.mark_signed(id, signed, None).unwrap();
s.set_trusted_certificates(vec![digital_id.certificate.clone()]);
let sig = &s.get(id).unwrap().signatures[0];
assert_eq!((sig.signed, sig.status, sig.modification.clone()), (true, SignatureStatus::Valid, sign::Modification::None));
}
/// A later signature draws nothing and leaves the flag: either would be a change to what the
/// first signature covers.
#[test]
fn a_later_signature_leaves_need_appearances_forms_alone() {
let p12 = std::fs::read(concat!(env!("CARGO_MANIFEST_DIR"), "/../sign/tests/data/ec-p256.p12")).unwrap();
let digital_id = sign::pkcs12::open(&p12, "test").unwrap();
let mut s = Session::new().with_clock(|| 1_800_000_000);
let id = s.open("form.pdf", None, Arc::new(need_appearances_form()), None).unwrap();
let first = s.sign(id, &digital_id, SignOptions { rect: None, ..SignOptions::default() }).unwrap();
// The flag set again after the first signature, and `name` back to its empty placeholder.
let mut cos = pdfcraft_cos::Document::open(first).unwrap();
let root = cos.root().unwrap();
cos.update_dict(root, |c| {
if let Some(pdfcraft_cos::Object::Dict(af)) = c.get_mut(b"AcroForm") {
af.set(b"NeedAppearances".to_vec(), pdfcraft_cos::Object::Bool(true));
}
})
.unwrap();
let placeholder = cos.add(pdfcraft_cos::Object::Stream(pdfcraft_cos::Stream::from_raw(Default::default(), b"/Tx BMC EMC".to_vec())));
cos.update_dict(pdfcraft_cos::ObjRef::new(4, 0), |w| {
let mut ap = pdfcraft_cos::Dict::new();
ap.set(b"N".to_vec(), pdfcraft_cos::Object::Ref(placeholder));
w.set(b"AP".to_vec(), pdfcraft_cos::Object::Dict(ap));
})
.unwrap();
let flagged =
Arc::new(pdfcraft_cos::write_incremental(&cos, &pdfcraft_cos::SaveOptions { object_streams: false, ..Default::default() }).unwrap());
let id = s.open("flagged.pdf", None, flagged.clone(), None).unwrap();
let second = s.sign(id, &digital_id, SignOptions { rect: None, ..SignOptions::default() }).unwrap();
let (before, after) = (pdfcraft_cos::Document::open(flagged).unwrap(), pdfcraft_cos::Document::open(second.clone()).unwrap());
assert!(pdfcraft_forms::need_appearances(&after));
let widget = |d: &pdfcraft_cos::Document| d.get(pdfcraft_cos::ObjRef::new(4, 0));
assert_eq!(widget(&after), widget(&before));
assert_eq!(normal_appearance(&after, 4), "/Tx BMC EMC");
s.mark_signed(id, second, None).unwrap();
assert_eq!(s.get(id).unwrap().signatures.iter().filter(|x| x.signed).count(), 2);
}
/// #3944: a value typed into an XFA `<passwordEdit/>` field stays in the open document (both the
/// widget and the live XFA datasets, even across multiple edits and across Save + `mark_saved`),
/// but is written to no file — neither on the widget nor in the XFA datasets stream.
@@ -3116,6 +3236,26 @@ fn a_word_in_a_font_set_by_an_extgstate_is_redacted() {
);
}
/// A word drawn as outlines reads through its /ActualText. A mark over part of it applies (the
/// outlines are clipped), and the name that would still read the whole word is gone from the page
/// and from the saved file.
#[test]
fn a_word_drawn_as_outlines_is_redacted_with_its_name() {
let mut s = Session::new().with_clock(|| 1_700_000_000);
let page = one_line_page("", "/Span <</ActualText (SECRET42)>> BDC 20 150 m 140 150 l 140 170 l 20 170 l h f EMC");
let id = s.open("outlines.pdf", None, Arc::new(page), None).unwrap();
assert_eq!(page_texts(&s, id), ["SECRET42"]);
let shape =
Shape::Redact { quads: vec![pdfcraft_annot::rect_quad([10.0, 140.0, 90.0, 180.0])], overlay: String::new(), look: Default::default() };
let mark =
Edit::AddAnnotation(NewAnnotation { page: 0, style: Style::default_for(&shape), shape, contents: String::new(), author: "Ada".into() });
s.apply(id, mark).unwrap();
s.apply(id, Edit::ApplyRedactions { pages: None }).unwrap();
assert_eq!(page_texts(&s, id), [""]);
let saved = s.save_bytes(id).unwrap();
assert!(!saved.windows(b"SECRET42".len()).any(|w| w == b"SECRET42"), "the name is not left in the file");
}
#[test]
fn redaction_marks_apply_for_good_and_undo() {
let (mut s, id) = session_with(2);
@@ -4182,6 +4322,7 @@ fn pasted_added_content_keeps_its_style_and_picture() {
italic: true,
size: 12.0,
color: [0.8, 0.1, 0.2],
ink: None,
align: TextAlign::Center,
aligns: Vec::new(),
char_spacing: 0.0,
@@ -4854,6 +4995,17 @@ fn bytes_has(haystack: &[u8], needle: &[u8]) -> bool {
/// ranges and embed a detached CMS over them. The signed bytes keep the
/// prepared appearance, so they validate exactly like a product signer's file.
fn sign_prepared(doc: &mut pdfcraft_cos::Document, id: &sign::pkcs12::DigitalId, date: &str) -> Vec<u8> {
sign_prepared_with(doc, id, date, |_| {})
}
/// [`sign_prepared`], with `value` adding to the signature dictionary first (a certification's
/// `/Reference`, say).
fn sign_prepared_with(
doc: &mut pdfcraft_cos::Document,
id: &sign::pkcs12::DigitalId,
date: &str,
value: impl FnOnce(&mut pdfcraft_cos::Dict),
) -> Vec<u8> {
use pdfcraft_cos::{Dict, Object, PdfString};
let reserve = 8192 + id.certificate.raw.len() + id.chain.iter().map(|c| c.raw.len()).sum::<usize>();
let mut v = Dict::new();
@@ -4863,6 +5015,7 @@ fn sign_prepared(doc: &mut pdfcraft_cos::Document, id: &sign::pkcs12::DigitalId,
v.set(b"ByteRange".to_vec(), Object::Array([0].iter().chain(LAYERED_BR_MARK.iter()).map(|n| Object::Int(*n)).collect()));
v.set(b"Contents".to_vec(), Object::String(PdfString { bytes: vec![0; reserve], hex: true }));
v.set(b"M".to_vec(), PdfString::literal(date.as_bytes().to_vec()));
value(&mut v);
let sig = doc.add(Object::Dict(v));
// The field by name, never by number: a full save (encryption, repairs) is
// free to renumber objects, and a hardcoded reference would stamp the value
@@ -5103,6 +5256,442 @@ fn real_layered_signatures_round_trip_behind_a_user_password() {
assert_eq!(t.get(tid).unwrap().signatures[0].status, SignatureStatus::Valid);
}
/// A one-page file whose unsigned signature field carries a named-layer appearance, in the
/// geometry of a real one scaled by 4 (every word and drawing synthetic): /N draws /FRM, which
/// draws n0–n4; n1 a yellow question mark and n4 "Signature Not Verified". Object numbers:
/// 1 catalog, 2 pages, 3 page (400×300), 4 field, 5 AcroForm, 6 /N, 7 /FRM, 8–12 n0–n4,
/// 13 Helvetica, 14 page content. `sign_prepared` adds the value.
fn named_layer_signable_fixture() -> Vec<u8> {
named_layer_signable_fixture_with("[0 0 100 100]")
}
/// [`named_layer_signable_fixture`] with `n1_bbox` as its n1 layer's `/BBox`.
fn named_layer_signable_fixture_with(n1_bbox: &str) -> Vec<u8> {
fn form(bbox: &str, resources: &str, ops: &str) -> String {
format!("<< /Type /XObject /Subtype /Form /BBox {bbox} /Resources {resources} /Length {} >>\nstream\n{ops}\nendstream", ops.len())
}
let helv = "<< /Font << /F1 13 0 R >> >>";
let glyph = "1.08 0 0 1.08 46 6 cm";
let objs = [
"<< /Type /Catalog /Pages 2 0 R /AcroForm 5 0 R >>".to_string(),
"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_string(),
"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 400 300] /Contents 14 0 R /Resources << /Font << /F1 13 0 R >> >> /Annots [4 0 R] >>"
.to_string(),
"<< /FT /Sig /T (Signature1) /Type /Annot /Subtype /Widget /F 132 /Rect [100 100 300 220] /P 3 0 R /AP << /N 6 0 R >> >>".to_string(),
"<< /Fields [4 0 R] /SigFlags 3 >>".to_string(),
form("[0 0 200 120]", "<< /XObject << /FRM 7 0 R >> >>", "q 1 0 0 1 0 0 cm /FRM Do Q"),
form(
"[0 0 200 120]",
"<< /XObject << /n0 8 0 R /n1 9 0 R /n2 10 0 R /n3 11 0 R /n4 12 0 R >> >>",
&format!("q /n0 Do Q q {glyph} /n1 Do Q q /n2 Do Q q {glyph} /n3 Do Q q /n4 Do Q"),
),
form("[0 0 100 100]", "<< >>", "% DSBlank"),
form(
n1_bbox,
"<< >>",
"q 1 0.84 0.2 rg 0.55 0.4 0 RG 3 w 30 72 m 30 88 42 98 56 98 c 72 98 84 88 84 74 c 66 50 l 58 28 l 44 28 l 44 48 l h B 44 6 14 14 re B Q",
),
form("[0 0 200 120]", helv, "BT /F1 13.6 Tf 1 0 0 1 8 62.4 Tm (Digitally signed by) Tj 1 0 0 1 8 48.8 Tm (Test Signer) Tj ET"),
form("[0 0 100 100]", "<< >>", "% DSBlank"),
form("[0 84 200 120]", helv, "BT /F1 18.8 Tf 1 0 0 1 8 93.2 Tm (Signature Not Verified) Tj ET"),
"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>".to_string(),
"<< /Length 44 >>\nstream\nBT /F1 14 Tf 20 270 Td (Contract text) Tj ET\nendstream".to_string(),
];
let mut out = b"%PDF-1.7\n".to_vec();
let mut offs = Vec::new();
for (k, o) in objs.iter().enumerate() {
offs.push(out.len());
out.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", k + 1).as_bytes());
}
let start = out.len();
out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes());
for o in &offs {
out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes());
}
out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{start}\n%%EOF", objs.len() + 1).as_bytes());
out
}
/// The status colours as the regenerated icons draw them.
const VALID_GREEN: [u8; 3] = [0x2D, 0x9D, 0x78];
const UNKNOWN_AMBER: [u8; 3] = [0xE6, 0x86, 0x19];
const INVALID_RED: [u8; 3] = [0xD7, 0x37, 0x3F];
/// How many pixels of `rgba` are within 40 of `rgb` on every channel. A regenerated icon's
/// stroke covers thousands at 1 px/pt; the fixture holds none of these colours otherwise.
fn pixels_near(rgba: &[u8], rgb: [u8; 3]) -> usize {
rgba.as_chunks::<4>().0.iter().filter(|p| p.iter().zip(rgb).all(|(a, b)| a.abs_diff(b) <= 40)).count()
}
/// The displayed page at 1 px/pt.
fn displayed_pixels(s: &Session, id: DocId) -> Vec<u8> {
let doc = s.get(id).unwrap();
let mut renderer = pdfcraft_render::PageRenderer::new(doc.display.clone(), doc.config.clone());
let r = renderer.render(pdfcraft_render::RenderRequest { page: 0, scale: 1.0, ..Default::default() });
assert!(r.error.is_none(), "{:?}", r.error);
r.rgba.to_vec()
}
fn exported_text(s: &Session, id: DocId) -> String {
export::Exporter::new(s.get(id).unwrap()).text_of(&[0]).unwrap()
}
/// The test identity and the certificate that makes it trusted.
fn test_identity() -> (sign::pkcs12::DigitalId, sign::x509::Certificate) {
let p12 = std::fs::read(concat!(env!("CARGO_MANIFEST_DIR"), "/../sign/tests/data/ec-p256.p12")).unwrap();
let id = sign::pkcs12::open(&p12, "test").unwrap();
let anchor = id.chain.first().cloned().unwrap_or_else(|| id.certificate.clone());
(id, anchor)
}
/// A named-layer signature through `Session`: the unknown state drawn on opening, the valid one
/// once trusted, the signed bytes saved unchanged and still valid on reopening.
fn check_named_layer_states(signed: Arc<Vec<u8>>, password: Option<&str>, anchor: &sign::x509::Certificate) {
let mut s = Session::new().with_clock(|| 1_800_000_000);
let id = s.open("named.pdf", None, signed.clone(), password).unwrap();
let sig = &s.get(id).unwrap().signatures[0];
assert_eq!(sig.status, SignatureStatus::Unknown, "untrusted at first");
assert!(sig.verdict_shown, "the appearance shows the unknown state");
let text = exported_text(&s, id);
assert!(text.contains("Validity unknown") && !text.contains("Not Verified"), "{text:?}");
let amber = pixels_near(&displayed_pixels(&s, id), UNKNOWN_AMBER);
assert!(amber > 500, "{amber} amber pixels");
s.set_trusted_certificates(vec![anchor.clone()]);
let doc = s.get(id).unwrap();
assert_eq!(doc.signatures[0].status, SignatureStatus::Valid);
assert!(doc.signatures[0].verdict_shown);
assert!(!doc.dirty, "trust is not an edit");
let exported = exported_text(&s, id);
assert!(exported.contains("Signature valid"), "{exported}");
let page = displayed_pixels(&s, id);
assert!(pixels_near(&page, VALID_GREEN) > 500 && pixels_near(&page, UNKNOWN_AMBER) < 50, "the page redraws for the new state");
// Saving writes the working file, which never held the drawn state.
let saved = s.save_bytes(id).unwrap();
assert!(saved == signed, "an unchanged save is the signed file exactly");
let mut t = Session::new().with_clock(|| 1_800_000_000);
t.set_trusted_certificates(vec![anchor.clone()]);
let tid = t.open("named-saved.pdf", None, saved, password).unwrap();
let sig = &t.get(tid).unwrap().signatures[0];
assert!(sig.status == SignatureStatus::Valid && sig.verdict_shown);
let exported = exported_text(&t, tid);
assert!(exported.contains("Signature valid"), "{exported}");
}
#[test]
fn named_layer_signatures_show_their_state_through_trust_save_and_reopen() {
let (id, anchor) = test_identity();
let mut doc = pdfcraft_cos::Document::open(Arc::new(named_layer_signable_fixture())).unwrap();
let signed = Arc::new(sign_prepared(&mut doc, &id, "D:20261002120000+01'00'"));
check_named_layer_states(signed, None, &anchor);
}
#[test]
fn named_layer_signatures_round_trip_behind_a_user_password() {
let (id, anchor) = test_identity();
let mut doc = pdfcraft_cos::Document::open(Arc::new(named_layer_signable_fixture())).unwrap();
doc.set_encryption(&pdfcraft_cos::NewEncryption {
algorithm: pdfcraft_cos::Algorithm::Aes128,
user_password: "user",
owner_password: "owner",
permissions: -1,
encrypt_metadata: true,
seed: [7; 32],
})
.unwrap();
let encrypted = Arc::new(pdfcraft_cos::write_full(&doc, &pdfcraft_cos::SaveOptions::default()).unwrap());
let mut doc = pdfcraft_cos::Document::open_with_password(encrypted, Some("user")).unwrap();
let signed = Arc::new(sign_prepared(&mut doc, &id, "D:20261002120000+01'00'"));
check_named_layer_states(signed, Some("user"), &anchor);
}
#[test]
fn a_tampered_named_layer_signature_shows_the_invalid_state() {
let (id, _) = test_identity();
let mut doc = pdfcraft_cos::Document::open(Arc::new(named_layer_signable_fixture())).unwrap();
let mut signed = sign_prepared(&mut doc, &id, "D:20261002120000+01'00'");
let at = signed.windows(15).position(|w| w == b"(Contract text)").unwrap();
signed[at + 10] = b'X';
let mut s = Session::new().with_clock(|| 1_800_000_000);
let id = s.open("tampered.pdf", None, Arc::new(signed), None).unwrap();
let sig = &s.get(id).unwrap().signatures[0];
assert!(sig.status == SignatureStatus::Invalid && sig.verdict_shown);
let exported = exported_text(&s, id);
assert!(exported.contains("Signature invalid"), "{exported}");
let red = pixels_near(&displayed_pixels(&s, id), INVALID_RED);
assert!(red > 500, "{red} red pixels");
}
/// A trusted named-layer signature.
fn trusted_named_layer_session() -> (Session, DocId) {
let (id, anchor) = test_identity();
let mut doc = pdfcraft_cos::Document::open(Arc::new(named_layer_signable_fixture())).unwrap();
let signed = Arc::new(sign_prepared(&mut doc, &id, "D:20261002120000+01'00'"));
let mut s = Session::new().with_clock(|| 1_800_000_000);
let vid = s.open("named.pdf", None, signed, None).unwrap();
s.set_trusted_certificates(vec![anchor]);
assert_eq!(s.get(vid).unwrap().signatures[0].status, SignatureStatus::Valid);
(s, vid)
}
#[test]
fn named_layer_states_reach_exports_and_both_print_modes() {
// What is drawn on the page is what leaves it.
let (s, id) = trusted_named_layer_session();
let doc = s.get(id).unwrap();
let source = doc.export_source();
let mut file = export::Exporter::from_source(export::ExportSource { bytes: doc.bytes.clone(), ..source.clone() });
let mut shown = export::Exporter::from_source(source);
assert_ne!(shown.png(0, 72.0).unwrap(), file.png(0, 72.0).unwrap(), "image export follows the display copy");
let text = shown.text_of(&[0]).unwrap();
assert!(text.contains("Signature valid") && !text.contains("Not Verified"), "{text:?}");
let mut settings = print::Settings { pages: vec![0], paper: (400.0, 300.0), ..Default::default() };
let (_, _, vector) = printed_pixels(s.print_pdf(id, &settings).unwrap());
settings.raster = Some(print::Raster { dpi: 72.0, grayscale: false });
let (_, _, raster) = printed_pixels(s.print_pdf(id, &settings).unwrap());
for (mode, pixels) in [("vector", &vector), ("raster", &raster)] {
assert!(pixels_near(pixels, VALID_GREEN) > 500, "{mode} print draws the valid state");
assert!(pixels_near(pixels, [255, 214, 51]) < 50, "{mode} print drops the stored question mark");
}
}
#[test]
fn named_layers_show_beside_comments_that_need_appearances() {
// A comment without an appearance makes the display copy for its own sake; the signature's
// state is drawn into that same copy.
use pdfcraft_cos::{ObjRef, Object};
let (signer, anchor) = test_identity();
let mut cos = pdfcraft_cos::Document::open(Arc::new(named_layer_signable_fixture())).unwrap();
let square = cos.add(pdfcraft_cos::Lexer::new(b"<< /Type /Annot /Subtype /Square /Rect [10 10 60 60] /C [1 0 0] >>", 0).object().unwrap());
cos.update_dict(ObjRef::new(3, 0), |page| page.set(b"Annots".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(4, 0)), Object::Ref(square)])))
.unwrap();
let signed = Arc::new(sign_prepared(&mut cos, &signer, "D:20261002120000+01'00'"));
let mut s = Session::new().with_clock(|| 1_800_000_000);
let id = s.open("commented.pdf", None, signed, None).unwrap();
s.set_trusted_certificates(vec![anchor]);
let shown = pdfcraft_cos::Document::open(s.get(id).unwrap().display.clone()).unwrap();
assert!(shown.get(square).as_dict().is_some_and(|d| d.contains(b"AP")), "the comment's appearance is in the same copy");
assert!(s.get(id).unwrap().signatures[0].verdict_shown);
let green = pixels_near(&displayed_pixels(&s, id), VALID_GREEN);
assert!(green > 500, "{green} green pixels");
}
#[test]
fn named_layers_show_beside_fields_the_viewer_draws() {
// A text field without an appearance makes the display copy for its own sake; the
// signature's state is drawn into that same copy.
use pdfcraft_cos::{Lexer, ObjRef, Object};
let (signer, anchor) = test_identity();
let mut cos = pdfcraft_cos::Document::open(Arc::new(named_layer_signable_fixture())).unwrap();
let field = cos.add(
Lexer::new(
b"<< /Type /Annot /Subtype /Widget /FT /Tx /T (name) /V (Test Signer) /DA (/F1 12 Tf 0 g) /Rect [20 20 380 40] /F 4 /P 3 0 R >>",
0,
)
.object()
.unwrap(),
);
let both = Object::Array(vec![Object::Ref(ObjRef::new(4, 0)), Object::Ref(field)]);
cos.update_dict(ObjRef::new(3, 0), |page| page.set(b"Annots".to_vec(), both.clone())).unwrap();
let resources = Lexer::new(b"<< /Font << /F1 13 0 R >> >>", 0).object().unwrap();
cos.update_dict(ObjRef::new(5, 0), |form| {
form.set(b"Fields".to_vec(), both);
form.set(b"DR".to_vec(), resources);
})
.unwrap();
let signed = Arc::new(sign_prepared(&mut cos, &signer, "D:20261002120000+01'00'"));
let mut s = Session::new().with_clock(|| 1_800_000_000);
let id = s.open("filled.pdf", None, signed, None).unwrap();
s.set_trusted_certificates(vec![anchor]);
let shown = pdfcraft_cos::Document::open(s.get(id).unwrap().display.clone()).unwrap();
assert!(shown.get(field).as_dict().is_some_and(|d| d.contains(b"AP")), "the field's appearance is in the same copy");
assert!(s.get(id).unwrap().signatures[0].verdict_shown);
let green = pixels_near(&displayed_pixels(&s, id), VALID_GREEN);
assert!(green > 500, "{green} green pixels");
}
#[test]
fn a_metadata_edit_keeps_the_regenerated_state() {
let (mut s, id) = trusted_named_layer_session();
s.apply(id, Edit::SetInfo { key: "Title".into(), value: "Contract".into() }).unwrap();
let doc = s.get(id).unwrap();
assert!(doc.signatures[0].verdict_shown, "the page still draws the state");
let exported = exported_text(&s, id);
assert!(exported.contains("Signature valid"), "{exported}");
}
#[test]
fn a_metadata_edit_that_changes_a_verdict_redraws_it() {
// A certification that allows no changes: setting the title invalidates the signature, and
// the page has to say so rather than keep the valid state it drew.
let (signer, anchor) = test_identity();
let mut cos = pdfcraft_cos::Document::open(Arc::new(named_layer_signable_fixture())).unwrap();
let signed = sign_prepared_with(&mut cos, &signer, "D:20261002120000+01'00'", |v| {
let reference =
pdfcraft_cos::Lexer::new(b"[<< /Type /SigRef /TransformMethod /DocMDP /TransformParams << /Type /TransformParams /P 1 /V /1.2 >> >>]", 0)
.object()
.unwrap();
v.set(b"Reference".to_vec(), reference);
});
let mut s = Session::new().with_clock(|| 1_800_000_000);
let id = s.open("certified.pdf", None, Arc::new(signed), None).unwrap();
s.set_trusted_certificates(vec![anchor]);
assert_eq!(s.get(id).unwrap().signatures[0].status, SignatureStatus::Valid);
s.apply(id, Edit::SetInfo { key: "Title".into(), value: "Changed".into() }).unwrap();
let doc = s.get(id).unwrap();
assert_eq!(doc.signatures[0].status, SignatureStatus::Invalid);
assert!(doc.signatures[0].verdict_shown);
let exported = exported_text(&s, id);
assert!(exported.contains("Signature invalid"), "{exported}");
let red = pixels_near(&displayed_pixels(&s, id), INVALID_RED);
assert!(red > 500, "{red} red pixels");
}
#[test]
fn verdict_marks_carry_over_only_while_every_verdict_is_unchanged() {
let (s, id) = trusted_named_layer_session();
let previous = s.get(id).unwrap().signatures.to_vec();
assert!(previous[0].verdict_shown);
let mut same = previous.clone();
same[0].verdict_shown = false;
assert!(carry_verdict_marks(&previous, &mut same) && same[0].verdict_shown);
let mut changed = previous.clone();
(changed[0].verdict_shown, changed[0].status) = (false, SignatureStatus::Invalid);
assert!(!carry_verdict_marks(&previous, &mut changed) && !changed[0].verdict_shown);
let mut later = previous.clone();
(later[0].verdict_shown, later[0].revision) = (false, previous[0].revision + 1);
assert!(!carry_verdict_marks(&previous, &mut later) && !later[0].verdict_shown);
}
#[test]
fn releasing_the_working_file_drops_the_marks() {
// The page shows the original's own artwork until the refresh finishes.
let (mut s, id) = trusted_named_layer_session();
let doc = s.docs.iter_mut().find(|d| d.id == id).unwrap();
assert!(doc.signatures[0].verdict_shown);
Session::release_working(doc);
assert!(!doc.signatures[0].verdict_shown);
}
#[test]
fn background_open_uses_the_current_trust_at_commit() {
let (signer, anchor) = test_identity();
let mut cos = pdfcraft_cos::Document::open(Arc::new(named_layer_signable_fixture())).unwrap();
let signed = Arc::new(sign_prepared(&mut cos, &signer, "D:20261002120000+01'00'"));
let mut s = Session::new().with_clock(|| 1_800_000_000);
let prepared = s.open_context().prepare_open("background.pdf", None, signed, None).unwrap();
s.set_trusted_certificates(vec![anchor]);
let id = s.commit_open(prepared);
let sig = &s.get(id).unwrap().signatures[0];
assert!(sig.status == SignatureStatus::Valid && sig.verdict_shown);
let exported = exported_text(&s, id);
assert!(exported.contains("Signature valid"), "{exported}");
}
#[test]
fn background_open_uses_the_current_trust_and_language_at_commit() {
// Both changed while the document was prepared: the state and its words are both the new ones.
let (signer, anchor) = test_identity();
let mut cos = pdfcraft_cos::Document::open(Arc::new(named_layer_signable_fixture())).unwrap();
let signed = Arc::new(sign_prepared(&mut cos, &signer, "D:20261002120000+01'00'"));
let mut s = Session::new().with_clock(|| 1_800_000_000);
let prepared = s.open_context().prepare_open("background.pdf", None, signed, None).unwrap();
s.set_trusted_certificates(vec![anchor]);
s.set_signature_captions(StatusCaptions {
valid: "Signatur gültig".into(),
unknown: "Gültigkeit unbekannt".into(),
invalid: "Signatur ungültig".into(),
});
let id = s.commit_open(prepared);
assert_eq!(s.get(id).unwrap().signatures[0].status, SignatureStatus::Valid);
let exported = exported_text(&s, id);
assert!(exported.contains("Signatur gültig"), "{exported}");
}
#[test]
fn printing_draws_signature_states_only_where_the_screen_does() {
let (mut s, id) = trusted_named_layer_session();
let settings = print::Settings { pages: vec![0], paper: (400.0, 300.0), ..Default::default() };
// The screen falls back to the file's artwork when its display copy cannot be written, and
// takes the marks with it; print follows.
let doc = s.docs.iter_mut().find(|d| d.id == id).unwrap();
let mut cleared = doc.signatures.to_vec();
cleared[0].verdict_shown = false;
doc.signatures = Arc::new(cleared);
let (_, _, printed) = printed_pixels(s.print_pdf(id, &settings).unwrap());
let green = pixels_near(&printed, VALID_GREEN);
assert!(green < 50, "{green} green pixels");
}
#[test]
fn a_failed_display_write_drops_the_marks() {
let (mut s, id) = trusted_named_layer_session();
let doc = s.docs.iter_mut().find(|d| d.id == id).unwrap();
let mut signatures = doc.signatures.to_vec();
assert!(signatures[0].verdict_shown);
// A stream-limited document is read-only: the writer refuses it.
let unwritable = pdfcraft_cos::Document::open_with_stream_limit(doc.bytes.clone(), None, 1).unwrap();
let editor = doc.editor.as_mut().unwrap();
let shown = write_display_copy(editor, &doc.bytes, &unwritable, &mut signatures);
assert!(Arc::ptr_eq(&shown, &doc.bytes), "the page falls back to the file's own artwork");
assert!(!signatures[0].verdict_shown, "so nothing counts as drawn");
// Logged once, however many refreshes lose the copy again.
write_display_copy(editor, &doc.bytes, &unwritable, &mut signatures);
let logged = editor.cos.repair_log().iter().filter(|l| l.contains("the display copy could not be written")).count();
assert_eq!(logged, 1);
}
#[test]
fn background_open_uses_the_current_caption_language_at_commit() {
let (mut s, id) = trusted_named_layer_session();
let bytes = s.get(id).unwrap().bytes.clone();
let prepared = s.open_context().prepare_open("background.pdf", None, bytes, None).unwrap();
s.set_signature_captions(StatusCaptions {
valid: "Signatur gültig".into(),
unknown: "Gültigkeit unbekannt".into(),
invalid: "Signatur ungültig".into(),
});
let later = s.commit_open(prepared);
let exported = exported_text(&s, later);
assert!(exported.contains("Signatur gültig"), "{exported}");
assert!(s.get(later).unwrap().signatures[0].verdict_shown);
assert!(!s.get(later).unwrap().dirty);
}
#[test]
fn caption_language_follows_the_session_captions() {
let (mut s, id) = trusted_named_layer_session();
let generation = s.get(id).unwrap().generation;
let german = StatusCaptions { valid: "Signatur gültig".into(), unknown: "Gültigkeit unbekannt".into(), invalid: "Signatur ungültig".into() };
s.set_signature_captions(german);
let doc = s.get(id).unwrap();
assert!(!doc.dirty && doc.generation == generation, "captions are not an edit");
assert!(doc.signatures[0].verdict_shown);
let exported = exported_text(&s, id);
assert!(exported.contains("Signatur gültig"), "{exported}");
// Documents opened later draw with them too.
let (signer, _) = test_identity();
let mut cos = pdfcraft_cos::Document::open(Arc::new(named_layer_signable_fixture())).unwrap();
let later = s.open("later.pdf", None, Arc::new(sign_prepared(&mut cos, &signer, "D:20261002120000+01'00'")), None).unwrap();
let exported = exported_text(&s, later);
assert!(exported.contains("Signatur gültig"), "{exported}");
}
#[test]
fn non_win_ansi_captions_embed_inter_in_the_display_copy_only() {
// Words WinAnsi cannot encode are drawn in Inter, embedded into the display copy only.
let (mut s, id) = trusted_named_layer_session();
let russian = StatusCaptions {
valid: "Подпись действительна".into(),
unknown: "Действительность неизвестна".into(),
invalid: "Подпись недействительна".into(),
};
s.set_signature_captions(russian);
let exported = exported_text(&s, id);
assert!(exported.contains("Подпись действительна"), "{exported}");
let doc = s.get(id).unwrap();
assert!(bytes_has(&doc.display, b"/BaseFont /Inter") && !bytes_has(&doc.bytes, b"/BaseFont /Inter"), "Inter goes into the display copy only");
}
#[test]
#[ignore = "timing probe"]
fn probe_edit_latency_on_a_large_signed_document() {
@@ -5212,6 +5801,35 @@ fn backgrounds_and_watermarks_from_files() {
assert!(s.apply(id, Edit::AddBackground { pages: vec![0], settings: Background::default(), replace: true, file: Some(junk) }).is_err());
}
/// The web build reads a page at a time (prepared on the page, recognised in a worker, the
/// lines sent back as JSON); with the real ocrs engine that finds the same words as reading in
/// one go (needs the models: `cargo xtask models`; skipped without them).
#[test]
fn reading_a_page_at_a_time_finds_the_same_words() {
let settings = ocr::OcrSettings::default();
let Ok(engines) = ocr::recognizers(&settings) else {
eprintln!("skipped: OCR models not installed");
return;
};
let mut s = Session::new().with_clock(|| 1_700_000_000);
let text = s.create_from_text("t", "Invoice 42: pay by Friday").unwrap();
let typed = s.open("text.pdf", None, text, None).unwrap();
let png = export::Exporter::new(s.get(typed).unwrap()).png(0, 150.0).unwrap();
let scan = s.create_from_images(&[("scan.png".into(), png.clone()), ("scan2.png".into(), png)]).unwrap();
let id = s.open("scan.pdf", None, scan, None).unwrap();
let at_once = s.ocr_job(id, &[], settings.clone()).unwrap().run(&engines, |_, _| true);
let mut steps = s.ocr_job(id, &[], settings).unwrap().steps();
let mut one_by_one = Vec::new();
while let Some(next) = steps.next_page() {
let shot = next.unwrap();
let lines = engines.primary.recognize(shot.image().unwrap(), &pdfcraft_ocr::RecognizeOptions::default()).unwrap();
let sent = ocr::lines_from_json(&pdfcraft_ocr::lines_to_json(&lines), engines.primary.id()).map_err(|e| e.to_string());
one_by_one.push(shot.place(sent.map(|lines| ocr::StrategyOutcome { lines, ..Default::default() })));
}
assert_eq!(one_by_one, at_once);
assert!(at_once.len() == 2 && at_once.iter().all(|p| p.skipped.is_none() && !p.words.is_empty()), "{at_once:?}");
}
/// Scan & OCR ▸ Recognize text on a page that is only a picture of text (needs the models:
/// `cargo xtask models`; skipped without them).
#[test]
@@ -5916,6 +6534,35 @@ fn actions_fail_when_hidden_information_could_not_be_removed() {
assert!(e.starts_with("Sanitize document:"), "{e}");
}
/// A PDF whose catalog names its language (`/Lang`) exports to Word and HTML in that language, so
/// Word checks the spelling in it, not in English; one that names none exports as before.
#[test]
fn office_exports_take_the_documents_language() {
let pdf = |catalog: &str| {
pdfcraft_testkit::Pdf::new()
.obj(catalog)
.obj("<< /Type /Pages /Kids [3 0 R] /Count 1 >>")
.obj("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 << /Type /Font /Subtype /Type1 /BaseFont /Helvetica >> >> >> >>")
.stream("", b"BT /F1 12 Tf 20 200 Td (Hello) Tj ET")
.build()
};
let mut s = Session::new();
let bg = s.open("lang.pdf", None, Arc::new(pdf("<< /Type /Catalog /Pages 2 0 R /Lang (bg-BG) >>")), None).unwrap();
let plain = s.open("plain.pdf", None, Arc::new(pdf("<< /Type /Catalog /Pages 2 0 R >>")), None).unwrap();
let d = s.get(bg).unwrap();
let docx = d.export_office(compare::OfficeFormat::Docx);
assert!(docx == pdfcraft_export::docx_with_language(&d.export_pages(), "lang", Some("bg-BG")), "Word names bg-BG");
assert!(docx != pdfcraft_export::docx(&d.export_pages(), "lang"), "Word names a language");
for format in [compare::OfficeFormat::Html, compare::OfficeFormat::HtmlSimple] {
let html = String::from_utf8(d.export_office(format)).unwrap();
assert!(html.contains("<html lang=\"bg-BG\">"), "{html}");
}
let d = s.get(plain).unwrap();
assert!(d.export_office(compare::OfficeFormat::Docx) == pdfcraft_export::docx(&d.export_pages(), "plain"), "no language, as before");
assert!(String::from_utf8(d.export_office(compare::OfficeFormat::Html)).unwrap().contains("<html>\n"));
assert!(String::from_utf8(d.export_office(compare::OfficeFormat::HtmlSimple)).unwrap().contains("<html>\n"));
}
#[test]
fn exporting_office_files_keeps_images() {
let mut s = Session::new();
@@ -7653,6 +8300,37 @@ fn free_text_without_appearance() -> Arc<Vec<u8>> {
Arc::new(bytes)
}
/// The CLI renders what the app shows with no trusted certificates: a named-layer signature in
/// the state it has then (validity unknown), not the "not verified" its file stores. The source
/// is not changed.
#[test]
fn cli_display_draws_signature_states_as_the_app_does_without_trust() {
let (id, _) = test_identity();
let mut doc = pdfcraft_cos::Document::open(Arc::new(named_layer_signable_fixture())).unwrap();
let signed = Arc::new(sign_prepared(&mut doc, &id, "D:20261002120000+01'00'"));
let display = display_bytes_for(signed.clone(), None);
assert!(!Arc::ptr_eq(&display, &signed), "the state is drawn into a copy");
let mut renderer = pdfcraft_render::PageRenderer::new(display, pdfcraft_render::RenderConfig::default());
let page = renderer.render(pdfcraft_render::RenderRequest { page: 0, scale: 1.0, ..Default::default() });
assert!(pixels_near(&page.rgba, UNKNOWN_AMBER) > 500, "the unknown state is drawn");
assert!(pixels_near(&page.rgba, [255, 214, 51]) < 50, "the stored question mark is not");
}
/// A layer recognised but too small to draw an icon in (a side so small the icon's scale rounds to
/// zero) changes nothing, so the CLI renders the source, as the app does.
#[test]
fn cli_display_makes_no_copy_when_no_state_could_be_drawn() {
let (id, _) = test_identity();
let n1 = format!("[0 0 0.{}6 100]", "0".repeat(322));
let mut doc = pdfcraft_cos::Document::open(Arc::new(named_layer_signable_fixture_with(&n1))).unwrap();
let signed = Arc::new(sign_prepared(&mut doc, &id, "D:20261002120000+01'00'"));
let cos = pdfcraft_cos::Document::open(signed.clone()).unwrap();
let trust = pdfcraft_sign::TrustStore::default();
let signatures = crate::doc_state::signatures_of(&cos, &signed, &trust, &pdfcraft_sign::DigestCache::default());
assert!(pdfcraft_sign::has_status_layers(&cos, &signatures), "the layers are recognised");
assert!(Arc::ptr_eq(&display_bytes_for(signed.clone(), None), &signed), "nothing was drawn, so there is no copy");
}
/// #2371: a comment without an appearance stream is displayed by the app, so the CLI's page image
/// must show it too. The helper's copy is for rendering only; documents that need no copy, and
/// files it can't display, keep exactly the original source bytes.
@@ -8112,6 +8790,7 @@ fn content_edit_keeps_pages_annotations_fields_and_links() {
italic: false,
size: 12.0,
color: [0.0, 0.0, 0.0],
ink: None,
align: TextAlign::Left,
aligns: Vec::new(),
char_spacing: 0.0,
@@ -8153,6 +8832,7 @@ fn content_refresh_lists_match_the_storages_an_edit_touches() {
italic: false,
size: 12.0,
color: [0.0, 0.0, 0.0],
ink: None,
align: TextAlign::Left,
aligns: Vec::new(),
char_spacing: 0.0,
@@ -8511,6 +9191,141 @@ fn a_signed_document_rebuilt_from_object_headers_refuses_edits() {
assert!(!s.get(id).unwrap().dirty);
}
/// A one-page document whose only signature field is signed and named with `len` bytes. The
/// signature field walk reads names of up to 4096 bytes and passes longer ones over.
fn signed_field_named(len: usize) -> Vec<u8> {
pdf_of(&[
"<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [4 0 R] >> >>".to_string(),
"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".into(),
"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 300] /Annots [4 0 R] >>".into(),
format!("<< /Type /Annot /Subtype /Widget /FT /Sig /T ({}) /V 5 0 R /Rect [20 20 180 60] /P 3 0 R >>", "a".repeat(len)),
"<< /Type /Sig /Filter /Adobe.PPKLite /SubFilter /adbe.pkcs7.detached /ByteRange [0 1 2 1] /Contents <00> >>".into(),
])
}
/// A PDF of `objs`, numbered from 1, with a cross-reference table.
fn pdf_of(objs: &[String]) -> Vec<u8> {
let mut out = b"%PDF-1.7\n".to_vec();
let mut offsets = Vec::new();
for (i, o) in objs.iter().enumerate() {
offsets.push(out.len());
out.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes());
}
let xref = out.len();
out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes());
for o in offsets {
out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes());
}
out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes());
out
}
fn open_signed_field_named(len: usize) -> (Session, DocId) {
let mut s = Session::new();
let id = s.open("form.pdf", None, Arc::new(signed_field_named(len)), None).expect("opens");
(s, id)
}
/// The cap's boundary as the engine meets it: a 4096-byte name is read, and its signature blocks a
/// full rewrite.
#[test]
fn a_signature_named_at_the_cap_is_listed_and_blocks_a_full_rewrite() {
let (s, id) = open_signed_field_named(4096);
let doc = s.get(id).unwrap();
let listed: Vec<_> = doc.signatures.iter().map(|r| (r.field.len(), r.signed)).collect();
assert_eq!(listed, [(4096, true)], "the field is read, and no row stands for unread ones");
assert_eq!(s.save_full_bytes(id).unwrap_err(), EditError::Signed);
}
/// One byte past the cap the field goes unread: the list says so, and a full rewrite is still
/// refused.
#[test]
fn a_signature_named_one_byte_past_the_cap_still_blocks_a_full_rewrite() {
let (s, id) = open_signed_field_named(4097);
let doc = s.get(id).unwrap();
let unread = doc.signatures.iter().find(|r| r.field == "UnreadFields");
let said = unread.map(|r| (r.signed, r.status, r.details.first().is_some_and(|d| d.message.starts_with("This form has too many fields"))));
assert_eq!(said, Some((true, SignatureStatus::Unknown, true)), "{:?}", doc.signatures);
assert_eq!(s.save_full_bytes(id).unwrap_err(), EditError::Signed);
}
/// A signed field behind a kid with no /T: one merged with its widget under a named field that
/// isn't a signature field (the walk can't list it, so its row is the unread one), and one under a
/// node with no /T (listed by its full name). Either way the document is signed, and neither a full
/// save nor Optimize may rewrite it.
#[test]
fn signed_fields_behind_kids_without_a_name_block_a_full_rewrite() {
let page = |annot: u32| format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 300] /Annots [{annot} 0 R] >>");
let head = |annot: u32| {
vec![
"<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [4 0 R] >> >>".to_string(),
"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".into(),
page(annot),
]
};
let sig = "<< /Type /Sig /SubFilter /ETSI.CAdES.detached /ByteRange [0 1 2 1] /Contents <00> >>".to_string();
let mut merged = head(5);
merged.extend([
"<< /T (parent) /Kids [5 0 R] >>".to_string(),
"<< /Type /Annot /Subtype /Widget /FT /Sig /V 6 0 R /Parent 4 0 R /Rect [20 20 180 60] /P 3 0 R >>".into(),
sig.clone(),
]);
let mut nested = head(6);
nested.extend([
"<< /T (p) /Kids [5 0 R] >>".to_string(),
"<< /Kids [6 0 R] /Parent 4 0 R >>".into(),
"<< /Type /Annot /Subtype /Widget /FT /Sig /T (s) /V 7 0 R /Parent 5 0 R /Rect [20 20 180 60] /P 3 0 R >>".into(),
sig,
]);
for (shape, objs, row) in [("merged", merged, "UnreadFields"), ("nested", nested, "p.s")] {
let mut s = Session::new();
let id = s.open("form.pdf", None, Arc::new(pdf_of(&objs)), None).expect("opens");
let doc = s.get(id).unwrap();
assert!(doc.is_signed() && doc.signatures.iter().any(|r| r.field == row), "{shape}: {:?}", doc.signatures);
assert_eq!(s.save_full_bytes(id).unwrap_err(), EditError::Signed, "{shape}");
assert!(matches!(s.optimize_job(id, &optimize::Settings::default(), &[]), Err(EditError::Signed)), "{shape}");
}
}
/// The walk stops short of a signed field, so a signature may go unlisted; the document still
/// counts as signed, and each guard against rewriting a signed document holds.
#[test]
fn a_signature_the_field_walk_passes_over_still_counts_as_signed() {
let (s, id) = open_signed_field_named(64 << 10);
let doc = s.get(id).unwrap();
assert!(doc.is_signed(), "{:?}", doc.signatures);
}
#[test]
fn a_full_rewrite_is_refused_when_the_field_walk_passes_a_signature_over() {
let (s, id) = open_signed_field_named(64 << 10);
assert_eq!(s.save_full_bytes(id).unwrap_err(), EditError::Signed);
}
#[test]
fn optimizing_is_refused_when_the_field_walk_passes_a_signature_over() {
let (s, id) = open_signed_field_named(64 << 10);
assert!(matches!(s.optimize_job(id, &optimize::Settings::default(), &[]), Err(EditError::Signed)));
}
#[test]
fn a_rewriting_edit_is_refused_when_the_field_walk_passes_a_signature_over() {
let (mut s, id) = open_signed_field_named(64 << 10);
assert!(matches!(s.apply(id, Edit::Sanitize), Err(EditError::SignedRewrite(_))));
}
#[test]
fn an_action_appends_to_a_file_whose_field_walk_passes_a_signature_over() {
let bytes = Arc::new(signed_field_named(64 << 10));
let steps = vec![actions::Step::from_id("set_title", "Checked").unwrap()];
let title = actions::Action { name: "Title".into(), description: String::new(), steps, builtin: false };
let r = actions::run_on(&title, "form.pdf", bytes.clone(), |_, _| {}).unwrap();
assert!(r.bytes.starts_with(&bytes[..]), "the action rewrote the file instead of appending to it");
let mut s = Session::new();
let id = s.open("out.pdf", None, r.bytes, None).unwrap();
assert_eq!(s.get(id).unwrap().info.title.as_deref(), Some("Checked"), "the title is in what was appended");
}
/// #1324: marking a figure decorative rewrites that figure only. The header, added text and an
/// unknown entry on the original stream stay attached to the page. This lives in the engine tests
/// because edit and a11y are the same layer and cannot depend on each other.
crates/engine/src/word_count.rs
+1
View File
@@ -78,6 +78,7 @@ fn count_inner(source: ExportSource, pages: &[usize], options: Options, mut prog
tables: options.exclude_tables,
code: options.exclude_code,
images: options.exclude_images,
painted: false,
mcids: excluded.remove(&page).unwrap_or_default(),
};
let text = renderer.filtered_text(page, filter).map_err(|e| format!("page {}: {e}", page.saturating_add(1)))?.plain_text();
crates/engine/tests/it/ocr_edit.rs
+110
View File
@@ -160,6 +160,116 @@ fn a_rewritten_ocr_phrase_keeps_its_word_widths() {
}
}
/// Recognize text sizes each word from its box and a guess at how much of the em its letters fill,
/// so one phrase's words come out several points apart (#3313). Here the first word is the
/// largest and the middle one the smallest; the phrase is set in 11 pt.
fn uneven_phrase() -> (Session, pdfcraft_engine::DocId) {
scanned_with(vec![
scanned_word("John", 20.0, 45.0, 70.0, 15.0),
scanned_word("Smith", 49.0, 78.0, 70.0, 9.0),
scanned_word("Example", 82.0, 124.0, 70.0, 11.0),
])
}
/// An edited OCR phrase is redrawn in one size, the median of its words', and each word keeps
/// the width the scan shows it at (#3313).
#[test]
fn an_edited_ocr_phrase_is_drawn_at_one_size() {
let (mut s, id) = uneven_phrase();
let block = s.get(id).unwrap().text_blocks(0).remove(0);
assert!((block.size - 11.0).abs() < 0.05, "the phrase's size is its words' median: {}", block.size);
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "John Example".into(), style: Default::default() }).unwrap();
let lines = s.get(id).unwrap().text_lines(0);
assert_eq!(lines.iter().map(|l| l.text.trim()).collect::<Vec<_>>(), ["John", "Example"]);
for line in &lines {
assert!((line.size - 11.0).abs() < 0.05, "{:?} is drawn at {} pt, not the phrase's 11", line.text, line.size);
}
// Each word keeps its width; the rest of the phrase closes up where Smith was.
assert!((lines[0].rect[2] - lines[0].rect[0] - 25.0).abs() < 0.5, "John keeps its width: {:?}", lines[0].rect);
assert!((lines[1].rect[2] - lines[1].rect[0] - 42.0).abs() < 0.5, "Example keeps its width: {:?}", lines[1].rect);
assert!(lines[1].rect[0] >= lines[0].rect[2] - 0.5, "Example runs into John");
}
/// Only invisible OCR text is set in one phrase size: visible text keeps each run's own size, and
/// a phrase whose size is NaN, zero, negative or infinite falls back to its words' sizes (#3313).
#[test]
fn only_an_ocr_phrase_with_a_usable_size_is_set_in_one_size() {
let (s, id) = uneven_phrase();
let block = s.get(id).unwrap().text_blocks(0).remove(0);
assert!(block.invisible);
assert_eq!(block.phrase_size(), Some(block.size));
let visible = pdfcraft_engine::TextBlock { invisible: false, ..block.clone() };
assert_eq!(visible.phrase_size(), None, "visible text keeps its runs' sizes");
for size in [f64::NAN, 0.0, -3.0, f64::INFINITY] {
assert_eq!(pdfcraft_engine::TextBlock { size, ..block.clone() }.phrase_size(), None, "a {size} pt phrase falls back to its words");
}
}
/// Choosing a font sends the phrase through the single-style rewrite, which takes its size from
/// the first run: that is the phrase's size too, not the first word's guess.
#[test]
fn a_new_font_for_an_ocr_phrase_keeps_its_one_size() {
let (mut s, id) = uneven_phrase();
let style = pdfcraft_engine::BlockStyle { family: Some((pdfcraft_engine::FontFamily::Times, false, false)), ..Default::default() };
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "John Smith Example".into(), style }).unwrap();
let lines = s.get(id).unwrap().text_lines(0);
assert!(!lines.is_empty());
for line in &lines {
assert!((line.size - 11.0).abs() < 0.05, "{:?} is drawn at {} pt, not the phrase's 11", line.text, line.size);
}
}
/// Two neighbouring words OCR sized alike join into one run; the median still counts each word,
/// so 15, 9 and 9 pt make a 9 pt phrase, not the 12 pt of its two runs.
#[test]
fn words_joined_into_one_run_each_count_toward_the_phrase_size() {
let (s, id) = scanned_with(vec![
scanned_word("John", 20.0, 45.0, 70.0, 15.0),
scanned_word("SMITH", 49.0, 78.0, 70.0, 9.0),
scanned_word("JONES", 82.0, 112.0, 70.0, 9.0),
]);
let doc = s.get(id).unwrap();
assert_eq!(doc.text_lines(0).len(), 2, "the two 9 pt words share a run");
let size = doc.text_blocks(0)[0].size;
assert!((size - 9.0).abs() < 0.05, "the phrase is set in its words' median, 9 pt: {size}");
}
/// A word OCR sized at more than twice the phrase's size still keeps the width it has on the scan.
#[test]
fn a_word_far_larger_than_its_phrase_keeps_its_width() {
let (mut s, id) = scanned_with(vec![
scanned_word("John", 20.0, 45.0, 70.0, 25.0),
scanned_word("Smith", 49.0, 78.0, 70.0, 10.0),
scanned_word("Example", 82.0, 124.0, 70.0, 10.5),
]);
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "John Smith Exam".into(), style: Default::default() }).unwrap();
let lines = s.get(id).unwrap().text_lines(0);
assert_eq!(lines[0].text.trim(), "John");
assert!((lines[0].rect[2] - lines[0].rect[0] - 25.0).abs() < 0.5, "John keeps its width: {:?}", lines[0].rect);
}
/// With a new font the phrase is one run in its first word's text state. How wide it is must not
/// depend on how far OCR misjudged that word's size: the same boxes give the same line.
#[test]
fn a_new_font_for_an_ocr_phrase_is_as_wide_whatever_its_first_word_was_sized() {
let width = |first: f64| {
let (mut s, id) = scanned_with(vec![
scanned_word("John", 20.0, 45.0, 70.0, first),
scanned_word("Smith", 49.0, 78.0, 70.0, 11.0),
scanned_word("Example", 82.0, 124.0, 70.0, 11.0),
]);
let style = pdfcraft_engine::BlockStyle { family: Some((pdfcraft_engine::FontFamily::Helvetica, false, false)), ..Default::default() };
// Short enough not to wrap in the phrase's box either way.
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "John Smith".into(), style }).unwrap();
let lines = s.get(id).unwrap().text_lines(0);
assert_eq!(lines.len(), 1, "one line: {lines:?}");
lines.iter().map(|l| l.rect[2]).fold(f64::MIN, f64::max) - lines.iter().map(|l| l.rect[0]).fold(f64::MAX, f64::min)
};
let (guessed, right) = (width(15.0), width(11.0));
assert!((guessed - right).abs() < 0.5, "a 15 pt guess for John makes the line {guessed} wide, an 11 pt one {right}");
}
/// Visible text set with a horizontally stretched text matrix (3 × 1), as some generators write.
fn stretched_visible() -> std::sync::Arc<Vec<u8>> {
use pdfcraft_cos::{Dict, Document, Object, SaveOptions, Stream};
crates/export/src/as_docx.rs
+51 -7
View File
@@ -1,5 +1,5 @@
use super::Zip;
use super::as_html::esc;
use super::as_html::{esc, language_tag};
use super::layout::{Image, Item, Page, Table, body_size, items};
/// A colour's 8-bit sRGB components (NaN and out-of-range values clamp; NaN reads as 0).
@@ -31,15 +31,23 @@ fn run_xml(text: &str, size: f64, bold: bool, italic: bool, color: [f64; 3]) ->
format!("<w:r><w:rPr>{rpr}</w:rPr><w:t xml:space=\"preserve\">{}</w:t></w:r>", esc(text))
}
/// One Word table: explicit absent borders (alignment alone cannot prove PDF rules), a grid sized
/// from the detected column edges, `gridSpan` for spanning cells, empty cells for holes.
/// One Word table: when the page's rules draw its grid, single inside borders and single outer
/// borders on the edges they rule; explicit absent borders otherwise (alignment alone cannot
/// prove PDF rules), a grid sized from the detected column
/// edges, `gridSpan` for spanning cells, empty cells for holes.
pub(crate) fn docx_table(t: &Table) -> String {
let mut edges = t.cols.clone();
edges.push(t.rect[2]);
let widths: Vec<i64> = edges.windows(2).map(|w| ((w[1] - w[0]).max(20.0) * 20.0).round().clamp(60.0, 31680.0) as i64).collect();
let mut s = String::from("<w:tbl><w:tblPr><w:tblW w:w=\"0\" w:type=\"auto\"/><w:tblBorders>");
for edge in ["top", "left", "bottom", "right", "insideH", "insideV"] {
s.push_str(&format!("<w:{edge} w:val=\"nil\"/>"));
let [top, left, bottom, right] = t.ruled.unwrap_or_default();
let inside = t.ruled.is_some();
for (edge, drawn) in [("top", top), ("left", left), ("bottom", bottom), ("right", right), ("insideH", inside), ("insideV", inside)] {
if drawn {
s.push_str(&format!("<w:{edge} w:val=\"single\" w:sz=\"4\" w:space=\"0\" w:color=\"auto\"/>"));
} else {
s.push_str(&format!("<w:{edge} w:val=\"nil\"/>"));
}
}
s.push_str("</w:tblBorders></w:tblPr><w:tblGrid>");
for w in &widths {
@@ -228,9 +236,40 @@ fn word_background(pages: &[Page]) -> Option<String> {
(same && first != [255, 255, 255]).then(|| format!("{:02X}{:02X}{:02X}", first[0], first[1], first[2]))
}
/// The `w:lang` that makes `tag` the language of a Word document's text. Word keeps one language
/// per script: Chinese, Japanese and Korean are its East Asian text, Arabic, Hebrew, Persian and
/// Urdu its right-to-left text, and any other language its Latin, Greek and Cyrillic text. A
/// script the tag names (`pa-Arab`, `zh-Latn`) decides over its language.
fn word_lang(tag: &str) -> String {
let mut subtags = tag.split('-');
let primary = subtags.next().unwrap_or_default().to_ascii_lowercase();
// The script comes straight after the language and any three-letter extended languages; a
// singleton before it starts an extension or private use, which names no script.
let script = subtags
.find(|s| s.len() != 3 || !s.bytes().all(|b| b.is_ascii_alphabetic()))
.filter(|s| s.len() == 4 && s.bytes().all(|b| b.is_ascii_alphabetic()))
.map(str::to_ascii_lowercase);
let attribute = match (script.as_deref(), primary.as_str()) {
(Some("arab" | "hebr" | "syrc" | "thaa"), _) => "w:bidi",
(Some("hans" | "hant" | "hani" | "jpan" | "kore" | "hang" | "hira" | "kana"), _) => "w:eastAsia",
(Some("latn" | "cyrl" | "grek"), _) => "w:val",
(_, "zh" | "ja" | "ko") => "w:eastAsia",
(_, "ar" | "he" | "fa" | "ur") => "w:bidi",
_ => "w:val",
};
format!("<w:lang {attribute}=\"{}\"/>", esc(tag))
}
/// A Word document (.docx, Office Open XML). Ordinary pages flow as paragraphs and tables;
/// pages with rotated text, or text that overlaps artwork, use floating shapes to retain their layout.
pub fn docx(pages: &[Page], title: &str) -> Vec<u8> {
docx_with_language(pages, title, None)
}
/// [`docx`] for a document in `language` (a BCP 47 tag such as `bg-BG`, as a PDF's `/Lang` gives
/// it): the default language of its text, so Word checks the spelling in that language rather
/// than in its own. A language that is not a well formed tag is left out.
pub fn docx_with_language(pages: &[Page], title: &str, language: Option<&str>) -> Vec<u8> {
let mut body = String::new();
let mut media: Vec<(String, &Image)> = Vec::new();
// Word needs a paragraph between adjacent tables and after the last one in the body.
@@ -338,11 +377,16 @@ xmlns:wp=\"http://schemas.openxmlformats.org/drawingml/2006/wordprocessingDrawin
doc_rels.push_str("<Relationship Id=\"rIdSettings\" Type=\"http://schemas.openxmlformats.org/officeDocument/2006/relationships/settings\" Target=\"settings.xml\"/>");
}
doc_rels.push_str("</Relationships>");
let styles = "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<w:styles xmlns:w=\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\">\
let defaults = language_tag(language)
.map(|tag| format!("<w:docDefaults><w:rPrDefault><w:rPr>{}</w:rPr></w:rPrDefault></w:docDefaults>", word_lang(tag)))
.unwrap_or_default();
let styles = format!(
"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<w:styles xmlns:w=\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\">{defaults}\
<w:style w:type=\"paragraph\" w:default=\"1\" w:styleId=\"Normal\"><w:name w:val=\"Normal\"/><w:pPr><w:spacing w:after=\"120\"/></w:pPr></w:style>\
<w:style w:type=\"paragraph\" w:styleId=\"Heading1\"><w:name w:val=\"heading 1\"/><w:basedOn w:val=\"Normal\"/><w:next w:val=\"Normal\"/><w:pPr><w:keepNext/><w:spacing w:before=\"240\"/><w:outlineLvl w:val=\"0\"/></w:pPr><w:rPr><w:b/></w:rPr></w:style>\
<w:style w:type=\"paragraph\" w:styleId=\"Heading2\"><w:name w:val=\"heading 2\"/><w:basedOn w:val=\"Normal\"/><w:next w:val=\"Normal\"/><w:pPr><w:keepNext/><w:spacing w:before=\"200\"/><w:outlineLvl w:val=\"1\"/></w:pPr><w:rPr><w:b/></w:rPr></w:style>\
</w:styles>";
</w:styles>"
);
let core = format!(
"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<cp:coreProperties xmlns:cp=\"http://schemas.openxmlformats.org/package/2006/metadata/core-properties\" xmlns:dc=\"http://purl.org/dc/elements/1.1/\"><dc:title>{}</dc:title></cp:coreProperties>",
esc(title)
Loaded 100 of 234 files, more files were not shown because too many files have changed in this diff. Show more