Mirrors/pdfcraft
1
0 Fork 0
mirror of https://github.com/storytold/pdfcraft.git synced 2026-11-12 17:31:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge main into fix/link-uri-base-and-quadpoints

Browse Source 
crates/render/Cargo.toml and Cargo.lock: pdfcraft-render takes both url (the
/URI /Base resolution) and main's unicode-bidi.
This commit is contained in:
mtlcl410 committed 2026-11-09 14:22:00 +08:00
2 parents 8957eef644 25025fa271
commit 375398cfac
85 files changed
+11330 -554
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

Cargo.lock
Generated
+4 -14
View File
@@ -1289,7 +1289,7 @@ dependencies = [
[[package]]
name = "craft-update"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=787a7a0f972fd1d7a7618fdbeca3a5267a0407ff#787a7a0f972fd1d7a7618fdbeca3a5267a0407ff"
source = "git+https://github.com/storytold/craft-libs?rev=1b34eba41e62138fc2438d2734523b302840b45d#1b34eba41e62138fc2438d2734523b302840b45d"
dependencies = [
"chrono",
"dunce",
@@ -4703,6 +4703,7 @@ dependencies = [
"serde",
"skrifa 0.44.0",
"thiserror 2.0.21",
"unicode-bidi",
"url",
]
@@ -4837,10 +4838,12 @@ dependencies = [
name = "pdfcraft-xfdf"
version = "0.18.1"
dependencies = [
"base64 0.23.1",
"pdfcraft-annot",
"pdfcraft-cos",
"pdfcraft-forms",
"pdfcraft-model",
"proptest",
"roxmltree 0.21.2",
"thiserror 2.0.21",
]
@@ -8764,7 +8767,6 @@ dependencies = [
"nix",
"typed-path",
"windows-sys 0.61.2",
"zopfli",
]
[[package]]
@@ -8779,18 +8781,6 @@ version = "1.0.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b"
[[package]]
name = "zopfli"
version = "0.8.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aaf7fc5d30c28483d93805c4a5e12b05bbb52407fa67c5f8bd552374cd01fb11"
dependencies = [
"bumpalo",
"crc32fast",
"log",
"simd-adler32",
]
[[package]]
name = "zune-core"
version = "0.5.3"
Cargo.toml
+1 -1
View File
@@ -28,7 +28,7 @@ craft-print = { git = "https://github.com/storytold/craft-libs", version = "0.1.
# Whether a path stored in a document may be opened (GoToR link targets).
craft-docpath = { version = "0.1.0", git = "https://github.com/storytold/craft-libs", rev = "787a7a0f972fd1d7a7618fdbeca3a5267a0407ff" }
# craft-update: app-specific pretend versions apply to startup and manual checks.
craft-update = { git = "https://github.com/storytold/craft-libs", version = "=0.1.0", rev = "787a7a0f972fd1d7a7618fdbeca3a5267a0407ff", features = ["egui"] }
craft-update = { git = "https://github.com/storytold/craft-libs", version = "=0.1.0", rev = "1b34eba41e62138fc2438d2734523b302840b45d", features = ["egui"] }
# internal
pdfcraft-geom = { path = "crates/geom", version = "0.18.1" }
pdfcraft-filters = { path = "crates/filters", version = "0.18.1" }
ROADMAP.md
+3
View File
@@ -240,6 +240,8 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md
## Progress log
- **2026-10-13 (M8, redact by regex):** `redact_mark` takes `regex` (caller expression, linear-time engine, length/size/nesting caps, empty-matching refused) with optional `validate` (luhn, mod97, iban). Headless only; issue #1886.
- **2026-11-08 (M5, comment exchange):** Comments keep their formatting and appearance through XFDF and FDF. XFDF export and import carry rich text, default style, border width, style and dash pattern, rotation, justification, fringe, line leaders and captions, the caret symbol and redaction overlay text as XFDF 3.0 (ISO 19444-1) names them, and a stamp's appearance as `<appearance>` (the DICT/STREAM/DATA encoding, bounded); copy and paste carries every comment's appearance. FDF export keeps a stamp's /AP with its forms and images, as XFDF does, and FDF import keeps any /AP the file carries (ISO 32000-2 §12.7.8.3.4). The intent goes out as both `intent` and `IT`. An unreadable XFDF appearance is left out, said in the import report and the repair log, and the comment is drawn by PdfCraft; an FDF past the import budget is refused whole. A redaction mark is always drawn by PdfCraft.
- **2026-11-07 (M2, #6449):** A non-embedded Times New Roman set as a TrueType CID font (`TimesNewRoman,Bold`, `TimesNewRomanPSMT` and the other faces) draws the glyphs its codes name, as the non-embedded Arial does: its first 258 glyphs follow the same standard Macintosh order, which the embedded Times New Roman subsets of a public French form confirm in 941 of 942 `/ToUnicode` entries (vendored hayro-interpret patch 88). That form drew "Salarié(e) du régime général" as "Salari²*g+f w't²i lo g'ì²p²tcn" and "changement" as "ej angement". Times and Times New Roman MT Std are not read this way.
- **2026-11-05 (M2, #5889):** A non-embedded Arial set as a TrueType CID font (`CIDFontType2`, `Identity-H`, as Word and PDFMaker write it) draws the glyphs its codes name. Those codes are glyph indexes in the author's Arial, whose first 258 glyphs follow the standard Macintosh order, so the standard substitute now draws each glyph by name (vendored hayro-interpret patch 76). Before, it took each code as one of its own glyph indexes, and "Low Power" drew as "@ck 'Dck Yf". Arial Narrow, Black and Unicode MS, and `CIDFontType0` fonts, are not read this way. Copying or searching those runs still needs a `/ToUnicode`.
@@ -262,6 +264,7 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md
- **2026-11-02 (M10, #5743):** macOS print jobs name their title and owner (`com.apple.print.JobInfo.PMJobName`, `PMJobOwner`), as the system print dialog does, because a Kyocera filter crashes on a job without them. Checked by a test and through `doc_print` against a stand-in `lp`; not yet on a physical printer.
- **2026-11-02 (M3, #5708):** Pages shorter than the window sit in its middle, one page or row at a time and when scrolling continuously with every page in view.
- **2026-11-02 (M3, #5716):** A snapshot is drawn at 300 ppi, or at the zoom if that is finer, so one taken zoomed out stays readable; an area too large for one 4096-pixel tile comes out as fine as a tile allows (about 147 ppi for 2000 pt). A choice of resolution and a selection that stays active remain.
- **2026-11-02 (M14, #5737, #5460):** Preferences can turn off the update check at start. It stays on by default, and Help ▸ Check for updates still asks.
- **2026-11-01 (M7, #5657):** Header/footer, watermark and background additions and replacements stage the complete call before committing. Synthetic later-page failures preserve exact incremental bytes, object count, dirty state and allocation cursor; success controls retain other marks and shared/inherited resources. Estimates unchanged.
crates/annot/src/appearance.rs
+233 -9
View File
@@ -4,7 +4,9 @@
//! objects it points at, so the same code serves new comments and restyled ones. Drawings are in page space with
//! `/BBox = /Rect` (identity matrix), except note icons, which are drawn in a 20 × 20 box.
//! It returns `None` for anything it cannot draw faithfully (cloudy borders, unknown line
//! endings), so callers never replace an appearance with a worse one.
//! endings), so callers never replace an appearance with a worse one. It resolves `/QuadPoints`
//! itself; for the other values, callers that hold the document follow references first
//! (`draw::with_direct_values`).
//!
//! The note icons are PdfCraft's own drawings (AGENTS.md §1). Text boxes use the standard
//! Helvetica font with WinAnsi encoding, and draw Thai from the craft-fonts Thai face as Type 3
@@ -147,6 +149,29 @@ pub fn smooth_segments(pts: &[(f64, f64)]) -> Vec<[(f64, f64); 3]> {
.collect()
}
/// The points where the cubic Bézier segment from `p0` (control points `c1`, `c2`) to `p3` turns
/// back in x or in y, strictly between its ends: with its ends, they bound the segment.
fn cubic_extremes(p0: (f64, f64), c1: (f64, f64), c2: (f64, f64), p3: (f64, f64)) -> Vec<(f64, f64)> {
let at = |t: f64| {
let u = 1.0 - t;
let w = [u * u * u, 3.0 * u * u * t, 3.0 * u * t * t, t * t * t];
(w[0] * p0.0 + w[1] * c1.0 + w[2] * c2.0 + w[3] * p3.0, w[0] * p0.1 + w[1] * c1.1 + w[2] * c2.1 + w[3] * p3.1)
};
let mut out = Vec::new();
for (q0, q1, q2, q3) in [(p0.0, c1.0, c2.0, p3.0), (p0.1, c1.1, c2.1, p3.1)] {
// The derivative over 3: a t² + 2 b t + c.
let (a, b, c) = (-q0 + 3.0 * q1 - 3.0 * q2 + q3, q0 - 2.0 * q1 + q2, q1 - q0);
let roots = if a.abs() < 1e-12 {
if b.abs() < 1e-12 { vec![] } else { vec![-c / (2.0 * b)] }
} else {
let disc = b * b - a * c;
if disc < 0.0 { vec![] } else { vec![(-b + disc.sqrt()) / a, (-b - disc.sqrt()) / a] }
};
out.extend(roots.into_iter().filter(|t| *t > 0.0 && *t < 1.0).map(at));
}
out
}
/// `c` operators for [`smooth_segments`] (the path already starts at the first point).
fn smooth_curve(pts: &[(f64, f64)]) -> String {
smooth_segments(pts).iter().map(|[a, b, e]| format!("{} {} {} {} {} {} c\n", n(a.0), n(a.1), n(b.0), n(b.1), n(e.0), n(e.1))).collect()
@@ -545,7 +570,15 @@ impl AppearanceDrawing {
/// Text the standard fonts can't draw comes out as `?`: [`undrawable_text`] says which, and
/// [`crate::set_appearance`] marks the annotation with it (`pdfcraft_fonts::UNDRAWN_MARK`).
pub fn build(doc: &Document, d: &Dict) -> Option<Stream> {
let context = appearance_context(doc, d)?;
build_on_page(doc, d, None)
}
/// [`build`] for an annotation on a page whose visible region is `page` (`Page::crop` in
/// pdfcraft-model): a text markup, line, polyline, polygon or ink annotation whose points reach
/// past its `/Rect` is drawn through them as far as the page shows ([`point_reach`]), and its
/// appearance's `/BBox` says how far ([`widened_rect`]). Without a page it is drawn within `/Rect`.
pub(crate) fn build_on_page(doc: &Document, d: &Dict, page: Option<[f64; 4]>) -> Option<Stream> {
let context = appearance_context(doc, d, page)?;
let drawing = AppearanceDrawing::new(&context);
match context.subtype {
b"Text" => return draw_icon(&context, "Note", [1.0, 0.82, 0.0], drawing),
@@ -559,15 +592,15 @@ pub fn build(doc: &Document, d: &Dict) -> Option<Stream> {
finish_appearance(&context, drawing)
}
fn appearance_context<'a>(doc: &'a Document, d: &'a Dict) -> Option<AppearanceContext<'a>> {
fn appearance_context<'a>(doc: &'a Document, d: &'a Dict, page: Option<[f64; 4]>) -> Option<AppearanceContext<'a>> {
// Imported measurement appearances may include leaders, captions and formatting that
// this builder cannot reproduce. Preserve them rather than silently losing detail.
if d.contains(b"Measure") && !d.contains(b"PCMeasureValue") {
return None;
}
let subtype = d.name(b"Subtype")?;
let rect = nums(d, b"Rect").filter(|r| r.len() == 4)?;
let rect = [rect[0].min(rect[2]), rect[1].min(rect[3]), rect[0].max(rect[2]), rect[1].max(rect[3])];
let rect = normalised_rect(d)?;
let rect = page.and_then(|page| point_reach(doc, d, rect, page)).unwrap_or(rect);
// Cloudy borders (§12.5.4) are drawn for polygons only.
let cloudy = d.get(b"BE").and_then(|b| b.as_dict()).is_some_and(|b| b.name(b"S") == Some(b"C"));
if cloudy && subtype != b"Polygon" {
@@ -586,6 +619,170 @@ fn appearance_context<'a>(doc: &'a Document, d: &'a Dict) -> Option<AppearanceCo
})
}
/// `/Rect` as lower-left and upper-right corners (a rectangle may be written with either pair of
/// opposite corners, ISO 32000-2 §7.9.5); `None` unless it is four numbers.
fn normalised_rect(d: &Dict) -> Option<[f64; 4]> {
let [x0, y0, x1, y1] = <[f64; 4]>::try_from(nums(d, b"Rect")?).ok()?;
Some([x0.min(x1), y0.min(y1), x0.max(x1), y0.max(y1)])
}
/// The subtypes drawn through points of their own rather than within `/Rect`: text markup over
/// its `/QuadPoints` (ISO 32000-2 §12.5.6.10), lines along `/L` (§12.5.6.7), polygons and
/// polylines through `/Vertices` (§12.5.6.9) and ink through `/InkList` (§12.5.6.13).
pub(crate) fn is_drawn_from_points(d: &Dict) -> bool {
matches!(d.name(b"Subtype"), Some(b"Highlight" | b"Underline" | b"StrikeOut" | b"Squiggly" | b"Line" | b"PolyLine" | b"Polygon" | b"Ink"))
}
/// At most this many quadrilaterals widen a text markup's box ([`point_reach`]). Each takes in a
/// word or a group of contiguous words (ISO 32000-2 Table 182), usually a line, so even written
/// word by word a markup over a whole dense page stays well below it; a larger one is drawn
/// within its `/Rect`, as before the box was widened.
pub(crate) const MAX_MARKUP_QUADS: usize = 10_000;
/// At most this many points widen a line's, polyline's, polygon's or ink annotation's box
/// ([`point_reach`]): as many as [`MAX_MARKUP_QUADS`] quadrilaterals have. A larger one is drawn
/// within its `/Rect`.
pub(crate) const MAX_REACH_POINTS: usize = 4 * MAX_MARKUP_QUADS;
/// The points `d` is drawn through, as `x y` pairs, and how far its strokes reach past them:
/// half the border width, and for a line ending (`/LE`) or a cloudy border (§12.5.4) the size
/// they are drawn at. `None` for other subtypes, more points than the caps allow, and points that
/// aren't all finite numbers.
fn drawn_points(doc: &Document, d: &Dict) -> Option<(Vec<f64>, f64)> {
let w = border_width(d);
let has_ending = || d.get(b"LE").and_then(Object::as_array).is_some_and(|e| e.iter().any(|e| e.as_name().is_some_and(|e| e != b"None")));
let (points, margin) = match d.name(b"Subtype")? {
subtype @ (b"Highlight" | b"Underline" | b"StrikeOut" | b"Squiggly") => {
// The quads `draw_markup` draws (`quads`: resolved, malformed ones skipped), counted
// before the numbers are read.
let array = doc.resolve(d.get(b"QuadPoints")?);
let entries = array.as_array()?;
if entries.len() / 8 > MAX_MARKUP_QUADS {
return None;
}
let quads = pdfcraft_geom::quad_points::usable_quads(entries.iter().map(|x| doc.resolve(x).as_f64()));
if quads.is_empty() {
return None;
}
// How far each quad's stroke reaches past it: half the line's width, and a squiggle's
// peaks above a quad lower than they are (`draw_markup`).
let margin = quads.iter().map(|q| markup_overreach(subtype, (q[0] - q[4]).hypot(q[1] - q[5]))).fold(0.0, f64::max);
(quads.into_iter().flatten().collect(), margin)
}
b"Ink" => {
let strokes = d.get(b"InkList")?.as_array()?;
let count = strokes.iter().try_fold(0usize, |sum, s| sum.checked_add(s.as_array()?.len()))?;
if count / 2 > MAX_REACH_POINTS {
return None;
}
let mut points = Vec::with_capacity(count.saturating_mul(3));
for s in strokes {
let stroke: Vec<f64> = s.as_array()?.iter().map(Object::as_f64).collect::<Option<_>>()?;
// A stroke's odd last number is no point (`draw_ink` skips it too).
let pts: Vec<(f64, f64)> = stroke.as_chunks::<2>().0.iter().map(|p| (p[0], p[1])).collect();
points.extend(pts.iter().flat_map(|p| [p.0, p.1]));
// The smooth curve through three or more points (`smooth_segments`) can bulge
// past them: its turning points count too.
if pts.len() > 2 {
let mut start = pts.first().copied().unwrap_or_default();
for [c1, c2, end] in smooth_segments(&pts) {
points.extend(cubic_extremes(start, c1, c2, end).iter().flat_map(|p| [p.0, p.1]));
start = end;
}
}
}
(points, w / 2.0)
}
subtype @ (b"PolyLine" | b"Polygon") => {
let count = d.get(b"Vertices")?.as_array()?.len();
if count < 4 || count % 2 != 0 || count / 2 > MAX_REACH_POINTS {
return None;
}
let cloudy = d.get(b"BE").and_then(Object::as_dict).is_some_and(|b| b.name(b"S") == Some(b"C"));
// A cloud's bumps are drawn at up to twice their nominal radius (`draw_polygon`).
let reach = match subtype {
b"Polygon" if cloudy => 2.0 * cloud_radius(w),
b"PolyLine" if has_ending() => arrow_size(w),
_ => 0.0,
};
(nums(d, b"Vertices")?, w / 2.0 + reach)
}
b"Line" => {
let [x0, y0, x1, y1] = <[f64; 4]>::try_from(nums(d, b"L")?).ok()?;
if ![x0, y0, x1, y1].iter().all(|v| v.is_finite()) {
return None;
}
// Leader lines move the line off `/L` (§12.5.6.7, Table 178).
let layout = line_layout(d, [(x0, y0), (x1, y1)], w);
let mut points = vec![x0, y0, x1, y1];
for (x, y) in layout.line.iter().chain(layout.leaders.iter().flatten()).chain(layout.tick.iter().flatten()) {
points.extend([*x, *y]);
}
// The caption's box: its width along the text matrix, from the descender to the
// cap height and a little over (Helvetica, as `draw_line` sets it).
if let Some((text, m)) = &layout.caption {
let width = text_width(text, CAPTION_SIZE);
for (x, y) in [(0.0, -0.25), (width, -0.25), (0.0, 1.0), (width, 1.0)].map(|(x, y)| (x, y * CAPTION_SIZE)) {
points.extend([m[0] * x + m[2] * y + m[4], m[1] * x + m[3] * y + m[5]]);
}
}
(points, w / 2.0 + if has_ending() { arrow_size(w) } else { 0.0 })
}
_ => return None,
};
(points.iter().all(|v| v.is_finite()) && margin.is_finite()).then_some((points, margin))
}
/// An annotation's `/Rect` (normalised, `rect`) grown to take in the points it is drawn through
/// ([`drawn_points`]) where they reach past it, as far as the page's visible region `page` goes;
/// `None` when they don't, aren't usable or number more than the caps allow, and for subtypes
/// drawn within `/Rect` ([`is_drawn_from_points`]). The points are what the annotation draws (ISO
/// 32000-2 Table 182 `/QuadPoints`, Table 178 `/L`, Table 181 `/Vertices`, Table 185 `/InkList`),
/// and the appearance is mapped onto `/Rect` (§12.5.5), so a `/Rect` that leaves them out would
/// cut off what is drawn: a highlight or ink stroke whose `/Rect` lies beside it drew nothing.
/// Once a point is outside, the box takes in the strokes' width and line endings too. Nothing
/// outside the crop box is shown or printed (§14.11.2), so the box grows no further than it; it
/// never shrinks `rect`, and a `/Rect` that holds every point is kept as it is.
fn point_reach(doc: &Document, d: &Dict, rect: [f64; 4], page: [f64; 4]) -> Option<[f64; 4]> {
if !rect.iter().all(|v| v.is_finite()) {
return None;
}
let (points, margin) = drawn_points(doc, d)?;
let mut reach = [f64::INFINITY, f64::INFINITY, f64::NEG_INFINITY, f64::NEG_INFINITY];
for p in points.as_chunks::<2>().0 {
reach = [reach[0].min(p[0]), reach[1].min(p[1]), reach[2].max(p[0]), reach[3].max(p[1])];
}
let clip = |b: [f64; 4]| [b[0].max(page[0]), b[1].max(page[1]), b[2].min(page[2]), b[3].min(page[3])];
let shown = clip(reach);
// Strokes wholly off the page widen nothing. Points `/Rect` already holds don't either, and
// points off the page whose strokes reach onto it only as far as `/Rect` holds.
let stroked = clip([reach[0] - margin, reach[1] - margin, reach[2] + margin, reach[3] + margin]);
if stroked[0] > stroked[2] || stroked[1] > stroked[3] {
return None;
}
let inside = |b: [f64; 4]| rect[0] <= b[0] && rect[1] <= b[1] && b[2] <= rect[2] && b[3] <= rect[3];
let on_page = shown[0] <= shown[2] && shown[1] <= shown[3];
if inside(if on_page { shown } else { stroked }) {
return None;
}
let reach = stroked;
let out = [rect[0].min(reach[0]), rect[1].min(reach[1]), rect[2].max(reach[2]), rect[3].max(reach[3])];
// A `/Rect` of its own can still be too wide for a width or height (`-1e308` to `1e308`).
let usable = (out[2] - out[0]).is_finite() && (out[3] - out[1]).is_finite();
(usable && out != rect).then_some(out)
}
/// The `/Rect` an annotation takes with its built appearance `stream`: the appearance's `/BBox`
/// where [`point_reach`] widened it past `/Rect`, as the appearance is mapped onto `/Rect`
/// (§12.5.5); `None` (keep `/Rect`) otherwise and for subtypes drawn within `/Rect`.
pub(crate) fn widened_rect(d: &Dict, stream: &Stream) -> Option<[f64; 4]> {
if !is_drawn_from_points(d) {
return None;
}
let bbox = <[f64; 4]>::try_from(nums(&stream.dict, b"BBox")?).ok()?;
(bbox.iter().all(|v| v.is_finite()) && Some(bbox) != normalised_rect(d)).then_some(bbox)
}
fn draw_body(context: &AppearanceContext<'_>, drawing: &mut AppearanceDrawing) -> Option<()> {
match context.subtype {
b"Highlight" | b"Underline" | b"StrikeOut" | b"Squiggly" => draw_markup(context, drawing),
@@ -621,6 +818,30 @@ fn finish_appearance(context: &AppearanceContext<'_>, mut drawing: AppearanceDra
Some(form(context.rect, &out, drawing.resources))
}
/// The line width of an underline or strike-out over a quad `h` high.
fn markup_line_width(h: f64) -> f64 {
(h * 0.07).clamp(0.5, 3.0)
}
/// A squiggle's line width and amplitude over a quad `h` high: its peaks stand twice the
/// amplitude above the quad's bottom edge.
fn squiggle(h: f64) -> (f64, f64) {
((h * 0.05).clamp(0.5, 2.0), (h * 0.06).max(0.75))
}
/// How far a text markup of `subtype` drawn over a quad `h` high reaches past the quad: half its
/// line's width, and for a squiggle, how far its peaks stand above a quad lower than they are.
fn markup_overreach(subtype: &[u8], h: f64) -> f64 {
match subtype {
b"Underline" | b"StrikeOut" => markup_line_width(h) / 2.0,
b"Squiggly" => {
let (lw, amp) = squiggle(h);
lw / 2.0 + (amp * 2.0 - h).max(0.0)
}
_ => 0.0,
}
}
fn draw_markup(context: &AppearanceContext<'_>, drawing: &mut AppearanceDrawing) -> Option<()> {
let subtype = context.subtype;
let stroke = context.stroke;
@@ -639,15 +860,14 @@ fn draw_markup(context: &AppearanceContext<'_>, drawing: &mut AppearanceDrawing)
c.push_str(&format!("{} {} m {} {} l {} {} l {} {} l h f\n", n(p1.0), n(p1.1), n(p2.0), n(p2.1), n(p4.0), n(p4.1), n(p3.0), n(p3.1)));
}
b"Underline" | b"StrikeOut" => {
let lw = (h * 0.07).clamp(0.5, 3.0);
let lw = markup_line_width(h);
let t = if subtype == b"Underline" { 0.08 } else { 0.42 };
let (a, b) = (at(p3, p1, t), at(p4, p2, t));
c.push_str(&format!("{}{} w\n{} {} m {} {} l S\n", rg_stroke(col), n(lw), n(a.0), n(a.1), n(b.0), n(b.1)));
}
_ => {
// Squiggly: a zigzag along the bottom of the quad.
let lw = (h * 0.05).clamp(0.5, 2.0);
let amp = (h * 0.06).max(0.75);
let (lw, amp) = squiggle(h);
let len = (p4.0 - p3.0).hypot(p4.1 - p3.1);
let step = (h / 4.0).max(1.5);
let steps = ((len / step).ceil() as usize).clamp(1, 10_000);
@@ -1078,7 +1298,11 @@ fn draw_free_text(context: &AppearanceContext<'_>, mut drawing: AppearanceDrawin
let mut rect = rect;
if d.contains(b"CL") {
let cl = nums(d, b"CL").filter(|l| l.len() == 4 || l.len() == 6)?;
let rd = nums(d, b"RD").filter(|r| r.len() == 4 && r.iter().all(|x| *x >= 0.0))?;
// `/RD` is optional (Table 177; null is absent, §7.3.9): without it the text box is `/Rect`.
let rd = match d.get(b"RD") {
None | Some(Object::Null) => vec![0.0; 4],
Some(_) => nums(d, b"RD").filter(|r| r.len() == 4 && r.iter().all(|x| *x >= 0.0))?,
};
rect = [full[0] + rd[0], full[1] + rd[1], full[2] - rd[2], full[3] - rd[3]];
if rect[2] - rect[0] < 1.0 || rect[3] - rect[1] < 1.0 {
return None;
crates/annot/src/draw.rs
+252 -29
View File
@@ -33,14 +33,70 @@ pub(crate) fn redraw_or_restore(doc: &mut Document, r: ObjRef, before: Dict) ->
pub(crate) fn draw_appearance(doc: &mut Document, r: ObjRef) -> Result<(), AnnotError> {
let d = annot_dict(doc, r);
let subtype = String::from_utf8_lossy(d.name(b"Subtype").unwrap_or_default()).into_owned();
let Some(stream) = build_appearance(doc, &d) else { return Err(AnnotError::Unsupported(subtype)) };
apply_appearance(doc, r, &d, stream)
let page = appearance::is_drawn_from_points(&d).then(|| page_box_listing(doc, r)).flatten();
let direct = with_direct_values(doc, &d);
let Some(stream) = build_appearance(doc, &d, &direct, page) else { return Err(AnnotError::Unsupported(subtype)) };
let built = Built::new(stream, &direct);
apply_appearance(doc, r, &d, built)
}
/// Draw the appearance for an annotation dictionary: the expensive half of [`set_appearance`].
/// Pure from the dictionary and the objects it points at, so [`AppearanceCache`] can reuse it.
fn build_appearance(doc: &Document, d: &Dict) -> Option<Stream> {
let mut stream = appearance::build(doc, d)?;
/// Whether [`set_appearance`] can draw the annotation `d`: read as the display reads it, with
/// references followed and the same fallback. Edits check this before changing anything.
pub(crate) fn can_draw(doc: &Document, d: &Dict) -> bool {
build_appearance(doc, d, &with_direct_values(doc, d), None).is_some()
}
/// The visible region (`Page::crop`) of the page that lists `r` among its annotations: the page
/// its `/P` names when that page lists it, read up its own `/Parent` chain, else the first page
/// of the tree that lists it. An annotation is listed by one page only (ISO 32000-2 §12.5.2), so
/// both find the same page, and the first reads one page instead of walking the whole tree for
/// every comment redrawn.
fn page_box_listing(doc: &Document, r: ObjRef) -> Option<[f64; 4]> {
let named = annot_dict(doc, r).reference(b"P").filter(|p| annots(doc, *p).iter().any(|a| a.as_ref() == Some(r)));
if let Some(page) = named {
return Some(page_with_inherited_boxes(doc, page).crop(doc));
}
pdfcraft_model::pages(doc).into_iter().find(|p| annots(doc, p.obj).iter().any(|a| a.as_ref() == Some(r))).map(|p| p.crop(doc))
}
/// The page `page` with the `/MediaBox` and `/CropBox` it inherits from its ancestors filled in
/// (ISO 32000-2 §7.7.3.4), enough for [`pdfcraft_model::Page::crop`]. Follows at most 64
/// `/Parent` links and stops at a loop.
fn page_with_inherited_boxes(doc: &Document, page: ObjRef) -> pdfcraft_model::Page {
let mut dict = doc.get(page).as_dict().cloned().unwrap_or_default();
let mut seen = std::collections::HashSet::from([page]);
let mut up = dict.reference(b"Parent");
while let Some(node) = up {
if seen.len() > 64 || !seen.insert(node) {
break;
}
let obj = doc.get(node);
let Some(d) = obj.as_dict() else { break };
for key in [&b"MediaBox"[..], b"CropBox"] {
if !dict.contains(key)
&& let Some(v) = d.get(key)
{
dict.set(key.to_vec(), v.clone());
}
}
up = d.reference(b"Parent");
}
pdfcraft_model::Page { obj: page, dict }
}
/// Draw the appearance for an annotation dictionary on a page whose visible region is `page`
/// (used by annotations drawn through points of their own, [`appearance::build_on_page`]): the
/// expensive half of [`set_appearance`]. `direct` is `d` with references followed
/// ([`with_direct_values`]). Pure from the dictionary, the objects it points at and the page, so
/// [`AppearanceCache`] can reuse it.
fn build_appearance(doc: &Document, d: &Dict, direct: &Dict, page: Option<[f64; 4]>) -> Option<Stream> {
// References followed first. Where that turns up something the builder doesn't draw (a
// cloudy `/BE` on a square), the drawing made without them stays, rather than none.
let mut stream = match appearance::build_on_page(doc, direct, page) {
Some(stream) => stream,
None if !std::ptr::eq(direct, d) => appearance::build_on_page(doc, d, page)?,
None => return None,
};
// Picture-stamp restyling must retain the placement's page-axis correction.
if d.get(b"PCPicture").is_some() {
let matrix = d.get(b"AP").and_then(|ap| {
@@ -56,10 +112,170 @@ fn build_appearance(doc: &Document, d: &Dict) -> Option<Stream> {
Some(stream)
}
/// Entries that link the annotation to other objects rather than describe how it looks: left as
/// references by [`with_direct_values`].
const LINK_KEYS: [&[u8]; 5] = [b"P", b"Popup", b"IRT", b"Parent", b"AP"];
/// How deep inside an entry [`with_direct_values`] follows references: `/InkList` is an array of
/// arrays, `/Border` may end in a dash array.
const INLINE_DEPTH: usize = 4;
/// Values [`with_direct_values`] copies out of referenced arrays and dictionaries, or walks
/// before finding one too large to copy, for one annotation, so a file can't have a large object
/// copied or scanned for every entry that points at it.
pub(crate) const INLINE_BUDGET: usize = 1 << 16;
/// Values copied or walked for all the annotations one display pass draws
/// ([`with_missing_appearances_cached`]), so a large object that many annotations point at is
/// not copied, or scanned and found too large, for each of them without bound. Past it,
/// references are left as they are.
pub(crate) const PASS_BUDGET: usize = 1 << 22;
/// `d` with the references among the values [`appearance::build`] reads followed. Any value may be
/// given by reference (ISO 32000-2 §7.3.10): a border style dictionary (`/BS 5 0 R`, Table 168), a
/// colour, `/RD`, `/CL`, a `/W` inside `/BS`. The builder reads only direct values (apart from
/// `/QuadPoints`, which it resolves itself), so it draws from this copy. The entries in
/// [`LINK_KEYS`] and streams stay references; past [`INLINE_DEPTH`] levels or [`INLINE_BUDGET`]
/// copied values a reference is left as it is, which the builder treats as unreadable. `d` itself
/// when nothing is a reference.
pub(crate) fn with_direct_values<'d>(doc: &Document, d: &'d Dict) -> std::borrow::Cow<'d, Dict> {
with_direct_values_within(doc, d, &mut INLINE_BUDGET.clone())
}
/// [`with_direct_values`], copying at most [`INLINE_BUDGET`] values and no more than `pass` has
/// left, which is reduced by what was copied.
pub(crate) fn with_direct_values_within<'d>(doc: &Document, d: &'d Dict, pass: &mut usize) -> std::borrow::Cow<'d, Dict> {
let allowed = INLINE_BUDGET.min(*pass);
let mut budget = allowed;
let mut out: Option<Dict> = None;
for (key, value) in d.iter() {
if LINK_KEYS.contains(&key.as_slice()) {
continue;
}
if let Some(direct) = direct_value(doc, value, INLINE_DEPTH, &mut budget) {
out.get_or_insert_with(|| d.clone()).set(key.clone(), direct);
}
}
*pass = pass.saturating_sub(allowed.saturating_sub(budget));
out.map_or(std::borrow::Cow::Borrowed(d), std::borrow::Cow::Owned)
}
/// The number of values in `o`, counting into its arrays and dictionaries (not through
/// references, which are charged when followed), or, past `limit`, `Err` with the values it
/// visited or queued before finding that out (at least one), so a rejected walk is charged too.
/// Walks at most `limit` values and holds no more than `limit` waiting.
fn weight(o: &Object, limit: usize) -> Result<usize, usize> {
let mut count = 0usize;
let mut stack = vec![o];
while let Some(o) = stack.pop() {
let spent = count.saturating_add(stack.len()).max(1);
count = match count.checked_add(1).filter(|c| *c <= limit) {
Some(c) => c,
None => return Err(spent),
};
let inner = match o {
Object::Array(items) => items.len(),
Object::Dict(entries) => entries.len(),
_ => continue,
};
// Every value waiting is counted later, so more than `limit` in all is over it already.
if count.saturating_add(stack.len()).saturating_add(inner) > limit {
return Err(spent.saturating_add(1));
}
match o {
Object::Array(items) => stack.extend(items.iter()),
Object::Dict(entries) => stack.extend(entries.iter().map(|(_, v)| v)),
_ => {}
}
}
Ok(count)
}
/// `o` with references followed `depth` levels into arrays and dictionaries, or `None` when there
/// is nothing to follow (or it may not be: a stream, an over-budget object, a chain of references).
fn direct_value(doc: &Document, o: &Object, depth: usize, budget: &mut usize) -> Option<Object> {
let held;
let (base, followed) = match o {
Object::Ref(_) => {
// A spent budget follows nothing more, not even to find out what is there.
if *budget == 0 {
return None;
}
held = doc.resolve(o);
if matches!(&*held, Object::Stream(_) | Object::Ref(_)) {
return None;
}
// Everything copied counts, inner arrays and dictionaries too, and so does the walk
// that finds an object too large: many references to one would otherwise each walk
// it again for free.
match weight(&held, *budget) {
Ok(n) => *budget = budget.checked_sub(n)?,
Err(spent) => {
*budget = budget.saturating_sub(spent);
return None;
}
}
(&*held, true)
}
other => (other, false),
};
let Some(next) = depth.checked_sub(1) else { return followed.then(|| base.clone()) };
match base {
Object::Array(items) => {
let mut changed: Option<Vec<Object>> = None;
for (i, item) in items.iter().enumerate() {
if let Some(direct) = direct_value(doc, item, next, budget) {
let copy = changed.get_or_insert_with(|| items.clone());
if let Some(slot) = copy.get_mut(i) {
*slot = direct;
}
}
}
changed.map(Object::Array).or_else(|| followed.then(|| base.clone()))
}
Object::Dict(entries) => {
let mut changed: Option<Dict> = None;
for (key, value) in entries.iter() {
if let Some(direct) = direct_value(doc, value, next, budget) {
changed.get_or_insert_with(|| entries.clone()).set(key.clone(), direct);
}
}
changed.map(Object::Dict).or_else(|| followed.then(|| base.clone()))
}
_ => followed.then(|| base.clone()),
}
}
/// A built appearance and what storing it needs from the dictionary it was drawn from, read
/// with references followed, so a cached one is stored without following them again.
#[derive(Clone, Debug)]
struct Built {
stream: Stream,
/// The first character the standard fonts can't draw (it came out as `?`).
missing: Option<char>,
/// A note's or attachment's `/Rect`, as written.
rect: Option<Vec<f64>>,
/// Points (/QuadPoints, /L, /Vertices, /InkList) past /Rect: its appearance covers them, so
/// /Rect does too.
markup_rect: Option<[f64; 4]>,
}
impl Built {
fn new(stream: Stream, direct: &Dict) -> Self {
let missing = appearance::undrawable_text(direct).map(|(_, c)| c);
let rect = matches!(direct.name(b"Subtype"), Some(b"Text" | b"FileAttachment"))
.then(|| direct.get(b"Rect")?.as_array()?.iter().map(|o| o.as_f64()).collect::<Option<Vec<f64>>>())
.flatten();
let markup_rect = appearance::widened_rect(direct, &stream);
Built { stream, missing, rect, markup_rect }
}
}
/// Store a built appearance on the annotation: the cheap half of [`set_appearance`].
fn apply_appearance(doc: &mut Document, r: ObjRef, d: &Dict, mut stream: Stream) -> Result<(), AnnotError> {
// Characters the standard fonts can't draw came out as `?`: mark the annotation with the first.
let missing = appearance::undrawable_text(d).map(|(_, c)| c);
fn apply_appearance(doc: &mut Document, r: ObjRef, d: &Dict, built: Built) -> Result<(), AnnotError> {
// `missing`: characters the standard fonts can't draw came out as `?`; the annotation is
// marked with the first.
let Built { mut stream, missing, rect, markup_rect } = built;
// A text box's Thai glyphs are streams of their own.
pdfcraft_fonts::shaped_text::store_streams(doc, &mut stream.dict);
let ap = doc.add(Object::Stream(stream));
@@ -72,17 +288,13 @@ fn apply_appearance(doc: &mut Document, r: ObjRef, d: &Dict, mut stream: Stream)
// A note icon has a fixed size (`NOTE_SIZE`); viewers draw it at the upper-left corner of `/Rect`.
// The appearance's box is mapped onto `/Rect`, so any other `/Rect` (a large one, or one with
// no width, as LaTeX `pdfcomment` writes) would stretch the icon or shrink it to nothing.
let icon_rect = matches!(d.name(b"Subtype"), Some(b"Text" | b"FileAttachment"))
.then(|| d.get(b"Rect")?.as_array()?.iter().map(|o| o.as_f64()).collect::<Option<Vec<f64>>>())
.flatten()
.filter(|q| q.len() == 4 && finite(q))
.map(|q| {
let (x, top) = (q[0].min(q[2]), q[1].max(q[3]));
[x, top - NOTE_SIZE, x + NOTE_SIZE, top]
});
let icon_rect = rect.filter(|q| q.len() == 4 && finite(q)).map(|q| {
let (x, top) = (q[0].min(q[2]), q[1].max(q[3]));
[x, top - NOTE_SIZE, x + NOTE_SIZE, top]
});
doc.update_dict(r, |d| {
d.set(b"AP".to_vec(), Object::Dict(apd));
if let Some(rect) = icon_rect {
if let Some(rect) = icon_rect.or(markup_rect) {
d.set(b"Rect".to_vec(), num_array(&rect));
}
d.remove(b"AS");
@@ -103,7 +315,9 @@ fn apply_appearance(doc: &mut Document, r: ObjRef, d: &Dict, mut stream: Stream)
/// refreshes) and keep it for content-only ones.
#[derive(Clone, Debug, Default)]
pub struct AppearanceCache {
entries: std::collections::HashMap<ObjRef, Stream>,
/// Each appearance with the page region it was drawn for (see [`appearance::build_on_page`]):
/// a changed crop box draws it again.
entries: std::collections::HashMap<ObjRef, (Option<[f64; 4]>, Built)>,
bytes: usize,
hits: u64,
builds: u64,
@@ -118,20 +332,21 @@ impl AppearanceCache {
*self = Self::default();
}
fn get(&mut self, r: ObjRef) -> Option<Stream> {
let stream = self.entries.get(&r)?.clone();
fn get(&mut self, r: ObjRef, crop: Option<[f64; 4]>) -> Option<Built> {
let built = self.entries.get(&r).filter(|(drawn_for, _)| *drawn_for == crop)?.1.clone();
self.hits += 1;
Some(stream)
Some(built)
}
fn insert(&mut self, r: ObjRef, stream: &Stream) {
fn insert(&mut self, r: ObjRef, crop: Option<[f64; 4]>, built: &Built) {
let stream = &built.stream;
self.builds += 1;
if self.entries.len() >= 1_000_000 {
self.clear();
}
if self.bytes.saturating_add(stream.raw.len()) <= Self::MAX_BYTES {
self.bytes += stream.raw.len();
self.entries.insert(r, stream.clone());
self.entries.insert(r, (crop, built.clone()));
}
// Else over budget: keep serving the entries already cached instead of growing.
}
@@ -165,6 +380,9 @@ pub fn with_missing_appearances_cached(doc: &Document, cache: &mut AppearanceCac
// reads `doc`, which never gets the appearance, so every repeat used to add another stream to
// the copy, and the display bytes grew with the number of listings.
let mut seen = std::collections::HashSet::new();
// Pages' visible regions, read once and only when an annotation drawn through its points is.
let mut crops: Option<std::collections::HashMap<ObjRef, [f64; 4]>> = None;
let mut budget = PASS_BUDGET;
for page in page_refs(doc).ok()? {
for r in annots(doc, page).iter().filter_map(Object::as_ref) {
if !seen.insert(r) {
@@ -178,9 +396,12 @@ pub fn with_missing_appearances_cached(doc: &Document, cache: &mut AppearanceCac
if has_normal {
continue;
}
if let Some(stream) = cache.get(r) {
let crop = appearance::is_drawn_from_points(&d)
.then(|| crops.get_or_insert_with(|| pdfcraft_model::pages(doc).iter().map(|p| (p.obj, p.crop(doc))).collect()).get(&page).copied())
.flatten();
if let Some(built) = cache.get(r, crop) {
// Fails only when `r` isn't a dictionary, which the cached build ruled out.
apply_appearance(copy.get_or_insert_with(|| doc.clone()), r, &d, stream).ok();
apply_appearance(copy.get_or_insert_with(|| doc.clone()), r, &d, built).ok();
continue;
}
// Another program's stamp gets a drawing for display only (ISO 32000-2 Table 184). Its
@@ -190,10 +411,12 @@ pub fn with_missing_appearances_cached(doc: &Document, cache: &mut AppearanceCac
bare.remove(b"AP");
appearance::display_stamp(&bare)
};
let Some(stream) = build_appearance(doc, &d).or_else(other_stamp) else { continue };
cache.insert(r, &stream);
let direct = with_direct_values_within(doc, &d, &mut budget);
let Some(stream) = build_appearance(doc, &d, &direct, crop).or_else(other_stamp) else { continue };
let built = Built::new(stream, &direct);
cache.insert(r, crop, &built);
// Fails only when `r` isn't a dictionary, which the build just ruled out.
apply_appearance(copy.get_or_insert_with(|| doc.clone()), r, &d, stream).ok();
apply_appearance(copy.get_or_insert_with(|| doc.clone()), r, &d, built).ok();
}
}
copy
crates/annot/src/info.rs
+5 -2
View File
@@ -245,7 +245,10 @@ pub fn props(doc: &Document, page: usize, index: usize) -> Option<Props> {
let p = page_ref(doc, page).ok()?;
let entry = annots(doc, p).get(index).cloned()?;
let obj = doc.resolve(&entry);
let d = obj.as_dict()?;
// Values given by reference (`/C 7 0 R`, `/BS 5 0 R`, ISO 32000-2 §7.3.10) read as the display draws them.
let raw = obj.as_dict()?;
let direct = crate::draw::with_direct_values(doc, raw);
let d = direct.as_ref();
let subtype = String::from_utf8_lossy(d.name(b"Subtype")?).into_owned();
let color = if subtype == "FreeText" { Some(appearance::parse_da(d).0) } else { drawn_color(d, b"C") };
let width = matches!(subtype.as_str(), "Square" | "Circle" | "Line" | "Ink" | "Polygon" | "PolyLine" | "FreeText").then(|| {
@@ -263,7 +266,7 @@ pub fn props(doc: &Document, page: usize, index: usize) -> Option<Props> {
width,
icon: (subtype == "Text").then(|| d.name(b"Name").and_then(|n| NoteIcon::from_name(&String::from_utf8_lossy(n))).unwrap_or(NoteIcon::Note)),
modified: text_value(doc, d, b"M"),
restylable: appearance::build(doc, d).is_some(),
restylable: crate::draw::can_draw(doc, raw),
locked: annotation_flags(doc, d) & FLAG_LOCKED != 0,
fillable: fillable(&subtype),
fill: fill_of(d),
crates/annot/src/modify.rs
+10 -6
View File
@@ -785,7 +785,7 @@ fn style_one(doc: &mut Document, r: ObjRef, change: StyleChange, meta: &Meta) ->
let d = annot_dict(doc, r);
let subtype = String::from_utf8_lossy(d.name(b"Subtype").unwrap_or_default()).into_owned();
// Check before changing anything: a stale appearance would contradict the new style.
if appearance::build(doc, &d).is_none() {
if !crate::draw::can_draw(doc, &d) {
return Err(AnnotError::Unsupported(subtype.clone()));
}
if color.is_some_and(|c| !finite(&c)) || opacity.is_some_and(|o| !o.is_finite()) || width.is_some_and(|w| !w.is_finite()) {
@@ -805,8 +805,12 @@ fn style_one(doc: &mut Document, r: ObjRef, change: StyleChange, meta: &Meta) ->
// So does a text box inside an oval or a circle, which goes around it again.
let shape = if free_text { BorderShape::of(&d) } else { BorderShape::Rectangle };
let typewriter = free_text && (d.name(b"IT") == Some(b"FreeTextTypeWriter") || shape != BorderShape::Rectangle) && !d.contains(b"CL");
// A border style given by reference (`/BS 5 0 R`, ISO 32000-2 §7.3.10): its width and other
// entries, which a new width keeps.
let direct = crate::draw::with_direct_values(doc, &d);
let (old_width, old_bs) = (border_width_of(&direct), direct.get(b"BS").and_then(Object::as_dict).cloned());
let regrow = width.filter(|_| typewriter).and_then(|w| {
let by = w.max(0.0) - if d.contains(b"BS") || d.contains(b"Border") { border_width_of(&d) } else { 0.0 };
let by = w.max(0.0) - if d.contains(b"BS") || d.contains(b"Border") { old_width } else { 0.0 };
let rect = grow(text_box_of(&d)?, by);
(by != 0.0 && finite(&rect) && rect[2] - rect[0] >= 1.0 && rect[3] - rect[1] >= 1.0).then_some((rect, w.max(0.0)))
});
@@ -830,9 +834,9 @@ fn style_one(doc: &mut Document, r: ObjRef, change: StyleChange, meta: &Meta) ->
set_part_keys(d, appearance::PartOpacity { line: o, fill: o, text: o });
}
if let Some(w) = width {
let old_w = border_width_of(d);
let old_w = old_width;
let w = w.max(0.0);
let mut bs = d.get(b"BS").and_then(|b| b.as_dict()).cloned().unwrap_or_default();
let mut bs = old_bs.unwrap_or_default();
bs.set(b"W".to_vec(), Object::Real(w));
d.set(b"BS".to_vec(), Object::Dict(bs));
d.remove(b"Border");
@@ -946,7 +950,7 @@ pub fn set_fill(doc: &mut Document, page: usize, index: usize, fill: Option<Rgb>
return Err(AnnotError::Invalid("only rectangles, ovals, polygons and text boxes can be filled".into()));
}
// Check before changing anything: a stale appearance would contradict the new fill.
if appearance::build(doc, &d).is_none() {
if !crate::draw::can_draw(doc, &d) {
return Err(AnnotError::Unsupported(subtype));
}
if fill.is_some_and(|c| !finite(&c)) {
@@ -987,7 +991,7 @@ pub fn set_part_opacity(doc: &mut Document, page: usize, index: usize, fill: Opt
return Err(AnnotError::Invalid("invalid opacity".into()));
}
// Check before changing anything: a stale appearance would contradict the new opacity.
if appearance::build(doc, &d).is_none() {
if !crate::draw::can_draw(doc, &d) {
return Err(AnnotError::Unsupported(subtype));
}
refuse_foreign_text(doc, &d)?;
crates/annot/src/tests.rs
+555 -6
View File
@@ -1724,6 +1724,190 @@ fn malformed_optional_interior_color_does_not_hide_shape_strokes() {
assert!(list(&doc, 0)[0].get(b"AP").is_none(), "display generation leaves the source document unchanged");
}
/// Any value may be given by reference (ISO 32000-2 §7.3.10). A Square whose border style
/// dictionary (Table 168), colour and dash array are indirect objects is drawn with them, not
/// with the 1 pt solid default; a reference cycle inside `/BS` is not followed for ever.
#[test]
fn indirect_border_style_and_colour_are_drawn() {
let doc = document_of(&[
b"<< /Type /Catalog /Pages 2 0 R >>",
b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 200 200] >>",
b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R] >>",
b"<< /Type /Annot /Subtype /Square /P 3 0 R /Rect [20 20 180 180] /BS 5 0 R /C 6 0 R >>",
b"<< /Type /Border /W 7 0 R /S /D /D 8 0 R /Self 5 0 R >>",
b"[0 0 1]",
b"6",
b"[4 2]",
]);
let display = with_missing_appearances(&doc).expect("the square gets a display appearance");
let shown = list(&display, 0).remove(0);
let content = ap_content(&display, &shown);
assert!(content.contains("0 0 1 RG\n6 w\n[4 2] 0 d\n"), "the indirect /BS, /W, /D and /C are used: {content}");
assert!(content.contains("23 23 154 154 re"), "the 6 pt border is inset by half its width: {content}");
assert_eq!(shown.get(b"BS"), Some(&Object::Ref(ObjRef::new(5, 0))), "the display copy keeps the reference");
assert_eq!(shown.get(b"P"), Some(&Object::Ref(ObjRef::new(3, 0))));
let props = props(&doc, 0, 0).expect("props");
assert_eq!(props.width, Some(6.0), "the properties show the width drawn");
}
/// A followed reference the builder can't draw (a cloudy border effect on a square, which it
/// leaves alone) keeps the drawing made without it instead of none.
#[test]
fn an_undrawable_indirect_value_keeps_the_plain_drawing() {
let doc = document_of(&[
b"<< /Type /Catalog /Pages 2 0 R >>",
b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 200 200] >>",
b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R] >>",
b"<< /Type /Annot /Subtype /Square /Rect [20 20 180 180] /BE 5 0 R /C [1 0 0] >>",
b"<< /S /C /I 1 >>",
]);
let display = with_missing_appearances(&doc).expect("the square is still drawn");
let shown = list(&display, 0).remove(0);
assert!(ap_content(&display, &shown).contains("1 0 0 RG\n1 w\n"));
}
/// Following references copies a bounded number of values: an indirect array past the budget
/// stays a reference (and the comment is not drawn from it), a chain that never ends is not
/// followed, and links to other objects are never copied.
#[test]
fn direct_values_are_bounded() {
let mut doc = fixture();
let huge = doc.add(Object::Array(vec![Object::Int(0); (1 << 16) + 1]));
let chain = doc.add(Object::Null);
doc.set(chain, Object::Ref(chain));
let mut d = Dict::new();
d.set(b"Subtype".to_vec(), Object::name("Square"));
d.set(b"Rect".to_vec(), Object::Ref(doc.add(num_array(&[0.0, 0.0, 10.0, 10.0]))));
d.set(b"Vertices".to_vec(), Object::Ref(huge));
d.set(b"IC".to_vec(), Object::Ref(chain));
d.set(b"Popup".to_vec(), Object::Ref(huge));
let direct = crate::draw::with_direct_values(&doc, &d);
assert_eq!(direct.get(b"Rect").and_then(Object::as_array).map(Vec::len), Some(4), "a small array is followed");
assert_eq!(direct.get(b"Vertices"), Some(&Object::Ref(huge)), "over the budget: left as a reference");
assert_eq!(direct.get(b"IC"), Some(&Object::Ref(chain)), "a chain of references is left alone");
assert_eq!(direct.get(b"Popup"), Some(&Object::Ref(huge)), "a link is not copied");
let mut plain = Dict::new();
plain.set(b"W".to_vec(), Object::Int(2));
assert!(matches!(crate::draw::with_direct_values(&doc, &plain), std::borrow::Cow::Borrowed(_)), "nothing to follow: no copy");
}
/// A new width for a comment whose border style is an indirect object (§7.3.10) keeps that
/// dictionary's other entries (a dashed style stays dashed) and leaves the shared object alone.
#[test]
fn new_width_keeps_an_indirect_border_style() {
let mut doc = document_of(&[
b"<< /Type /Catalog /Pages 2 0 R >>",
b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 200 200] >>",
b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R] >>",
b"<< /Type /Annot /Subtype /Square /Rect [20 20 180 180] /BS 5 0 R /C [1 0 0] >>",
b"<< /W 6 /S /D /D [4 2] >>",
]);
set_style(&mut doc, 0, 0, StyleChange::new(None, None, Some(3.0), None), &meta("")).unwrap();
let doc = reopen(&doc);
let d = list(&doc, 0).remove(0);
let bs = d.get(b"BS").and_then(Object::as_dict).expect("a direct /BS");
assert_eq!(bs.get(b"W").and_then(Object::as_f64), Some(3.0));
assert_eq!(bs.name(b"S"), Some(b"D".as_slice()), "the dashed style is kept");
assert!(ap_content(&doc, &d).contains("3 w\n[4 2] 0 d\n"));
let shared = doc.get(ObjRef::new(5, 0));
assert_eq!(shared.as_dict().and_then(|b| b.get(b"W")).and_then(Object::as_f64), Some(6.0), "the shared object is unchanged");
}
/// A colour given by reference (`/C 6 0 R`, §7.3.10) is read by the properties and the edits as
/// the display draws it: the properties show it, the appearance can be restyled and filled.
#[test]
fn an_indirect_colour_reads_and_restyles_as_drawn() {
let mut doc = document_of(&[
b"<< /Type /Catalog /Pages 2 0 R >>",
b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 200 200] >>",
b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R] >>",
b"<< /Type /Annot /Subtype /Square /Rect [20 20 180 180] /C 5 0 R /IC 6 0 R >>",
b"[0 0 1]",
b"[1 1 0]",
]);
let p = props(&doc, 0, 0).expect("props");
assert_eq!((p.color, p.fill, p.restylable), (Some([0.0, 0.0, 1.0]), Some([1.0, 1.0, 0.0]), true));
set_style(&mut doc, 0, 0, StyleChange::new(None, None, Some(3.0), None), &meta("")).expect("restylable");
set_fill(&mut doc, 0, 0, Some([0.0, 1.0, 0.0]), &meta("")).expect("fillable");
let d = list(&doc, 0).remove(0);
let content = ap_content(&doc, &d);
assert!(content.contains("0 0 1 RG") && content.contains("0 1 0 rg") && content.contains("3 w"), "{content}");
}
/// References are followed into everything they point at, and all of it is counted: a referenced
/// array of two arrays of 40,000 numbers is over the per-annotation budget. One display pass
/// copies at most [`crate::draw::PASS_BUDGET`] values in all, so an array many annotations point
/// at is not copied again for each of them.
#[test]
fn following_references_is_bounded_per_value_and_per_pass() {
let mut doc = fixture();
let inner = Object::Array(vec![Object::Int(0); 40_000]);
let nested = doc.add(Object::Array(vec![inner.clone(), inner]));
let mut d = Dict::new();
d.set(b"Subtype".to_vec(), Object::name("Square"));
d.set(b"Junk".to_vec(), Object::Ref(nested));
let direct = crate::draw::with_direct_values(&doc, &d);
assert!(direct.get(b"Junk") == Some(&Object::Ref(nested)), "80,003 values: left as a reference");
let shared = doc.add(Object::Array(vec![Object::Int(0); 60_000]));
d.set(b"Junk".to_vec(), Object::Ref(shared));
let mut pass = crate::draw::PASS_BUDGET;
let copies = (0..100).filter(|_| matches!(crate::draw::with_direct_values_within(&doc, &d, &mut pass), std::borrow::Cow::Owned(_))).count();
assert_eq!(copies, crate::draw::PASS_BUDGET / 60_001, "copies stop when the pass budget is spent");
}
/// Finding an object too large to copy costs the walk that found it out: an array of two arrays
/// of 40,000 numbers that many annotations point at is walked about 40,000 values deep before it
/// is rejected, and that is charged to the annotation and to the pass, so after a bounded number
/// of such walks the pass has no room left to walk it again. A reference to it repeated in one annotation
/// is not walked again in full, and a small referenced colour after it is still read.
#[test]
fn a_rejected_large_object_is_charged_to_the_budgets() {
let mut doc = fixture();
let inner = Object::Array(vec![Object::Int(0); 40_000]);
let nested = doc.add(Object::Array(vec![inner.clone(), inner]));
let red = doc.add(Object::Array(vec![Object::Int(1), Object::Int(0), Object::Int(0)]));
let mut d = Dict::new();
d.set(b"Subtype".to_vec(), Object::name("Square"));
d.set(b"Junk".to_vec(), Object::Array(vec![Object::Ref(nested); 1_000]));
d.set(b"C".to_vec(), Object::Ref(red));
let mut pass = crate::draw::PASS_BUDGET;
let direct = crate::draw::with_direct_values_within(&doc, &d, &mut pass);
assert_eq!(direct.get(b"C"), Some(&Object::Array(vec![Object::Int(1), Object::Int(0), Object::Int(0)])));
let charged = crate::draw::PASS_BUDGET - pass;
assert!((40_000..=crate::draw::INLINE_BUDGET).contains(&charged), "the rejected walk is charged once: {charged}");
d.set(b"Junk".to_vec(), Object::Ref(nested));
for _ in 0..crate::draw::PASS_BUDGET / 40_000 {
crate::draw::with_direct_values_within(&doc, &d, &mut pass);
}
assert!(pass < 40_000, "after {} annotations the pass has no room for another full walk: {pass}", crate::draw::PASS_BUDGET / 40_000 + 1);
}
/// `/RD` is optional (ISO 32000-2 Table 177): a callout without it has its text box at `/Rect`
/// and is drawn, box, text and leader line. A callout line and `/RD` given by reference are read too.
#[test]
fn callout_without_rd_is_drawn() {
let doc = document_of(&[
b"<< /Type /Catalog /Pages 2 0 R >>",
b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 400 400] >>",
b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R 5 0 R 6 0 R] >>",
b"<< /Type /Annot /Subtype /FreeText /IT /FreeTextCallout /Rect [100 100 300 160] /CL [20 20 60 60 100 130] /LE /OpenArrow /C [1 1 0] /DA (0 g /Helv 12 Tf) /Contents (callout) >>",
b"<< /Type /Annot /Subtype /FreeText /IT /FreeTextCallout /Rect [10 200 300 300] /CL 7 0 R /RD 8 0 R /C [1 1 0] /DA (0 g /Helv 12 Tf) /Contents (indirect) >>",
b"<< /Type /Annot /Subtype /FreeText /IT /FreeTextCallout /Rect [10 300 300 390] /CL [5 305 10 350] /RD [1 2] /C [1 1 0] /DA (0 g /Helv 12 Tf) /Contents (bad) >>",
b"[15 205 40 250 100 250]",
b"[90 0 0 0]",
]);
let display = with_missing_appearances(&doc).expect("the callouts get a display appearance");
let shown = list(&display, 0);
let plain = ap_content(&display, &shown[0]);
assert!(plain.contains("1 1 0 rg\n100 100 200 60 re f"), "the yellow box fills /Rect: {plain}");
assert!(plain.contains("20 20 m\n60 60 l\n100 130 l\nS"), "the leader line: {plain}");
assert!(plain.contains("(callout) Tj"), "the text: {plain}");
let indirect = ap_content(&display, &shown[1]);
assert!(indirect.contains("100 200 200 100 re f") && indirect.contains("15 205 m"), "indirect /CL and /RD: {indirect}");
assert!(shown[2].get(b"AP").is_none(), "a malformed /RD is still not guessed at");
}
/// #3175: viewers draw a note without an appearance as a fixed-size icon at the upper-left corner
/// of its `/Rect`. The display copy must not stretch the icon over a large `/Rect` or shrink it
/// to nothing in one with no width.
@@ -3089,8 +3273,10 @@ fn a_referenced_colour_is_listed_like_a_direct_one() {
}
/// A reference among the four entries is not a non-number read as 0: `/C [1 0 0 6 0 R]` with
/// object 6 the number 1 is CMYK black (§7.3.10), so drawing it cyan would be wrong. The builder
/// doesn't follow it and draws no appearance, as for any colour it can't read.
/// object 6 the number 1 is CMYK black (§7.3.10), so drawing it cyan would be wrong. The display
/// and the properties follow the reference (`draw::with_direct_values`) and draw and
/// read it black; the comment list, which follows only a `/C` given whole by reference, reads no
/// colour rather than cyan.
#[test]
fn a_four_entry_colour_with_a_reference_is_not_read_as_zero() {
let doc = document_of(&[
@@ -3101,11 +3287,374 @@ fn a_four_entry_colour_with_a_reference_is_not_read_as_zero() {
b"<< /Type /Annot /Subtype /Square /Rect [55 5 95 35] /C [1 0 0] >>",
b"1",
]);
let display = with_missing_appearances(&doc).expect("the RGB square gets a fallback appearance");
let display = with_missing_appearances(&doc).expect("both squares get a fallback appearance");
let squares = list(&display, 0);
assert!(!squares[0].contains(b"AP"), "no cyan stand-in for CMYK black");
assert_eq!(props(&doc, 0, 0).and_then(|p| p.color), None);
assert_eq!(summaries(&doc)[0].color, None, "nor in the comment list");
let black = ap_content(&display, &squares[0]);
assert!(black.contains("0 0 0 RG") && !black.contains("0 1 1 RG"), "CMYK black, no cyan stand-in: {black}");
assert_eq!(props(&doc, 0, 0).and_then(|p| p.color), Some([0.0, 0.0, 0.0]));
assert_ne!(summaries(&doc)[0].color, Some([0.0, 1.0, 1.0]), "nor cyan in the comment list");
}
/// A text markup annotation is drawn over its `/QuadPoints` (ISO 32000-2 Table 182), and an
/// appearance is mapped onto `/Rect` (§12.5.5). A highlight whose `/Rect` lies beside its text
/// (here written with its corners swapped, as a real file has it) got an appearance cut to that
/// `/Rect`, so nothing showed. The display copy's appearance and `/Rect` now take in the
/// quadrilaterals; a markup whose `/Rect` already holds them keeps it.
#[test]
fn text_markup_whose_quads_reach_past_its_rect_is_drawn_over_them() {
let doc = document_of(&[
b"<< /Type /Catalog /Pages 2 0 R >>",
b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>",
b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R 5 0 R 6 0 R 7 0 R 8 0 R 9 0 R 10 0 R] >>",
b"<< /Type /Annot /Subtype /Highlight /Rect [335.1 370.83 324.6 471.78] /C [1 1 0] /QuadPoints [370 287 471 287 370 277 471 277] >>",
b"<< /Type /Annot /Subtype /Underline /Rect [10 10 20 20] /C [0 0 1] /QuadPoints [100 120 200 120 100 100 200 100] >>",
b"<< /Type /Annot /Subtype /Highlight /Rect [50 50 150 70] /C [1 1 0] /QuadPoints [60 68 140 68 60 52 140 52] >>",
b"<< /Type /Annot /Subtype /StrikeOut /Rect [10 10 20 20] /C [1 0 0] /QuadPoints [100 220 200 220 100 200 200 200] >>",
b"<< /Type /Annot /Subtype /Squiggly /Rect [10 10 20 20] /C [0 1 0] /QuadPoints [100 320 200 320 100 300 200 300] >>",
b"<< /Type /Annot /Subtype /Highlight /Rect [10 10 20 20] /C [1 1 0] /QuadPoints [-1e30 20 1e30 20 -1e30 10 1e30 10] >>",
b"<< /Type /Annot /Subtype /Highlight /Rect [-1e308 10 1e308 20] /C [1 1 0] /QuadPoints [100 120 200 120 100 100 200 100] >>",
]);
let display = with_missing_appearances(&doc).expect("the markups need an appearance");
let shown = list(&display, 0);
let rect = |d: &Dict| nums_of(d, b"Rect");
let bbox = |d: &Dict| {
let n = d.get(b"AP").map(|a| display.resolve(a)).and_then(|a| a.as_dict().and_then(|a| a.reference(b"N"))).unwrap();
let obj = display.get(n);
let Object::Stream(s) = &*obj else { panic!("AP is not a stream") };
nums_of(&s.dict, b"BBox")
};
assert_eq!(rect(&shown[0]), [324.6, 277.0, 471.0, 471.78], "the highlight's /Rect takes in its text");
assert_eq!(bbox(&shown[0]), rect(&shown[0]));
assert!(ap_content(&display, &shown[0]).contains("370 287 m 471 287 l 471 277 l 370 277 l h f"), "the quad is filled");
// Half the underline's width (0.07 × 20 / 2) past the quad.
assert_eq!(rect(&shown[1]), [10.0, 10.0, 200.7, 120.7], "an underline's too");
assert_eq!(bbox(&shown[1]), rect(&shown[1]));
assert_eq!(rect(&shown[2]), [50.0, 50.0, 150.0, 70.0], "a /Rect that holds the quads stays as it is");
assert_eq!(rect(&shown[3]), [10.0, 10.0, 200.7, 220.7], "a strikeout's too");
assert_eq!(bbox(&shown[3]), rect(&shown[3]));
assert_eq!(rect(&shown[4]), [10.0, 10.0, 200.5, 320.5], "a squiggly's too");
assert_eq!(bbox(&shown[4]), rect(&shown[4]));
assert_eq!(rect(&shown[5]), [0.0, 10.0, 612.0, 20.0], "quads far past the page reach only as far as it shows");
// A /Rect too wide for its width to be a number keeps its size: the on-page quad above it
// doesn't make it taller.
assert_eq!(rect(&shown[6]), [-1e308, 10.0, 1e308, 20.0], "an overflowing /Rect is not widened");
assert_eq!(rect(&list(&doc, 0)[0]), [335.1, 370.83, 324.6, 471.78], "the source document is unchanged");
}
/// The box widens over the quads the markup is drawn over, read as every `/QuadPoints` reader
/// reads them (`pdfcraft_geom::quad_points::usable_quads`): an indirect array or entry is
/// followed, and a malformed quad or trailing partial one is skipped while the others still widen
/// the box, so what is drawn is not cut to `/Rect`.
#[test]
fn text_markup_reach_reads_quads_like_the_drawing() {
let doc = document_of(&[
b"<< /Type /Catalog /Pages 2 0 R >>",
b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>",
b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R 5 0 R 6 0 R] >>",
b"<< /Type /Annot /Subtype /Highlight /Rect [10 10 20 20] /C [1 1 0] /QuadPoints 7 0 R >>",
b"<< /Type /Annot /Subtype /Highlight /Rect [10 10 20 20] /C [1 1 0] /QuadPoints [100 120 /oops 120 100 100 200 100 100 220 200 220 100 200 200 200 1 2 3] >>",
b"<< /Type /Annot /Subtype /Highlight /Rect [10 10 20 20] /C [1 1 0] /QuadPoints [100 120 200 120 100 100 8 0 R 100] >>",
b"[100 120 200 120 100 100 200 100]",
b"200",
]);
let display = with_missing_appearances(&doc).expect("the markups need an appearance");
let shown = list(&display, 0);
let rect = |d: &Dict| nums_of(d, b"Rect");
assert_eq!(rect(&shown[0]), [10.0, 10.0, 200.0, 120.0], "an indirect array widens the box");
assert_eq!(ap_bbox(&display, &shown[0]), rect(&shown[0]));
assert_eq!(rect(&shown[1]), [10.0, 10.0, 200.0, 220.0], "the usable quad still widens it");
assert_eq!(ap_bbox(&display, &shown[1]), rect(&shown[1]));
assert_eq!(rect(&shown[2]), [10.0, 10.0, 200.0, 120.0], "an indirect entry is followed");
}
/// `n` copies of the quadrilateral `[100 120 200 120 100 100 200 100]`, as `/QuadPoints`.
fn repeated_quads(n: usize) -> String {
format!("[{}]", vec!["100 120 200 120 100 100 200 100"; n].join(" "))
}
/// The `/BBox` of `d`'s normal appearance in `doc`.
fn ap_bbox(doc: &Document, d: &Dict) -> Vec<f64> {
let n = d.get(b"AP").map(|a| doc.resolve(a)).and_then(|a| a.as_dict().and_then(|a| a.reference(b"N"))).expect("has /AP /N");
let obj = doc.get(n);
let Object::Stream(s) = &*obj else { panic!("AP is not a stream") };
nums_of(&s.dict, b"BBox")
}
/// The widened box stops at the page's visible region (its crop box, ISO 32000-2 §14.11.2), and
/// quadrilaterals wholly outside it widen nothing. A markup with more than
/// `appearance::MAX_MARKUP_QUADS` quadrilaterals keeps its `/Rect`. An annotation with an
/// appearance of its own, and one that isn't text markup, keep theirs whatever `/QuadPoints` say.
#[test]
fn text_markup_reach_stops_at_the_page_and_the_quad_cap() {
let cap = appearance::MAX_MARKUP_QUADS;
let at_cap = format!("<< /Type /Annot /Subtype /Highlight /Rect [10 10 20 20] /C [1 1 0] /QuadPoints {} >>", repeated_quads(cap));
let past_cap = format!("<< /Type /Annot /Subtype /Highlight /Rect [10 10 20 20] /C [1 1 0] /QuadPoints {} >>", repeated_quads(cap + 1));
let doc = document_of(&[
b"<< /Type /Catalog /Pages 2 0 R >>",
b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>",
b"<< /Type /Page /Parent 2 0 R /CropBox [0 0 300 400] /Annots [4 0 R 5 0 R 6 0 R 7 0 R 8 0 R 9 0 R] >>",
b"<< /Type /Annot /Subtype /Highlight /Rect [10 10 20 20] /C [1 1 0] /QuadPoints [100 380 500 380 100 360 500 360] >>",
b"<< /Type /Annot /Subtype /Highlight /Rect [10 10 20 20] /C [1 1 0] /QuadPoints [400 500 450 500 400 480 450 480] >>",
at_cap.as_bytes(),
past_cap.as_bytes(),
b"<< /Type /Annot /Subtype /Highlight /Rect [10 10 20 20] /C [1 1 0] /QuadPoints [100 120 200 120 100 100 200 100] /AP << /N 10 0 R >> >>",
b"<< /Type /Annot /Subtype /Square /Rect [10 10 20 20] /C [0 0 1] /QuadPoints [100 120 200 120 100 100 200 100] >>",
b"<< /Type /XObject /Subtype /Form /BBox [0 0 10 10] /Length 0 >>\nstream\n\nendstream",
]);
let display = with_missing_appearances(&doc).expect("the markups need an appearance");
let shown = list(&display, 0);
let rect = |d: &Dict| nums_of(d, b"Rect");
assert_eq!(rect(&shown[0]), [10.0, 10.0, 300.0, 380.0], "the box stops at the crop box");
assert_eq!(ap_bbox(&display, &shown[0]), rect(&shown[0]));
assert_eq!(rect(&shown[1]), [10.0, 10.0, 20.0, 20.0], "quads off the page widen nothing");
assert_eq!(rect(&shown[2]), [10.0, 10.0, 200.0, 120.0], "as many quads as the cap widen the box");
assert_eq!(rect(&shown[3]), [10.0, 10.0, 20.0, 20.0], "more quads than the cap keep the /Rect");
assert_eq!(ap_bbox(&display, &shown[3]), rect(&shown[3]));
assert_eq!(rect(&shown[4]), [10.0, 10.0, 20.0, 20.0], "a markup with an appearance of its own keeps its /Rect");
assert_eq!(
shown[4].get(b"AP").map(|a| display.resolve(a)).and_then(|a| a.as_dict().and_then(|a| a.reference(b"N"))),
Some(ObjRef::new(10, 0)),
"and its appearance"
);
assert_eq!(rect(&shown[5]), [10.0, 10.0, 20.0, 20.0], "a square is not text markup");
assert_eq!(ap_bbox(&display, &shown[5]), rect(&shown[5]));
}
/// A restyle (the edit path, `set_appearance`) redraws a text markup over its quadrilaterals and
/// saves the `/Rect` its appearance covers, as far as the page shows, since the appearance is
/// mapped onto `/Rect` (ISO 32000-2 §12.5.5). A markup whose `/Rect` already holds its quads keeps
/// that `/Rect` as written, swapped corners and all.
#[test]
fn restyled_text_markup_saves_the_rect_its_appearance_covers() {
let mut doc = document_of(&[
b"<< /Type /Catalog /Pages 2 0 R >>",
b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>",
b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R 5 0 R 6 0 R] >>",
b"<< /Type /Annot /Subtype /Highlight /Rect [335.1 370.83 324.6 471.78] /C [1 1 0] /QuadPoints [370 287 471 287 370 277 471 277] >>",
b"<< /Type /Annot /Subtype /Highlight /Rect [150 70 50 50] /C [1 1 0] /QuadPoints [60 68 140 68 60 52 140 52] >>",
b"<< /Type /Annot /Subtype /Underline /Rect [10 10 20 20] /C [0 0 1] /QuadPoints [500 120 700 120 500 100 700 100] >>",
]);
for index in 0..3 {
set_style(&mut doc, 0, index, StyleChange::new(Some([1.0, 0.0, 0.0]), None, None, None), &meta("")).unwrap();
}
let saved = reopen(&doc);
let shown = list(&saved, 0);
let rect = |d: &Dict| nums_of(d, b"Rect");
assert_eq!(rect(&shown[0]), [324.6, 277.0, 471.0, 471.78], "the saved /Rect takes in the highlighted text");
assert_eq!(ap_bbox(&saved, &shown[0]), rect(&shown[0]));
assert!(ap_content(&saved, &shown[0]).contains("370 287 m 471 287 l 471 277 l 370 277 l h f"), "the quad is filled");
assert_eq!(rect(&shown[1]), [150.0, 70.0, 50.0, 50.0], "a /Rect that holds the quads is kept as written");
assert_eq!(rect(&shown[2]), [10.0, 10.0, 612.0, 120.7], "the saved /Rect stops at the page");
assert_eq!(ap_bbox(&saved, &shown[2]), rect(&shown[2]));
}
/// A one-page document whose page lists `annots` (objects 4 on) and the display copy
/// `with_missing_appearances` draws for it.
fn displayed(annots: &[&[u8]]) -> (Document, Document) {
let kids: Vec<String> = (0..annots.len()).map(|i| format!("{} 0 R", i + 4)).collect();
let page = format!("<< /Type /Page /Parent 2 0 R /Annots [{}] >>", kids.join(" "));
let mut objs: Vec<&[u8]> =
vec![b"<< /Type /Catalog /Pages 2 0 R >>", b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>", page.as_bytes()];
objs.extend_from_slice(annots);
let doc = document_of(&objs);
let display = with_missing_appearances(&doc).expect("the annotations need an appearance");
(doc, display)
}
/// Ink strokes (`/InkList`, ISO 32000-2 Table 185) outside an ink annotation's `/Rect` are drawn:
/// the display copy's `/Rect` and `/BBox` take in the strokes and half their width. Shaped like
/// the pencil marks of `issue13447.pdf`, whose `/Rect` has x and y swapped. An ink annotation
/// whose `/Rect` holds its points keeps it, even where its wide stroke reaches past it.
#[test]
fn ink_whose_strokes_reach_past_its_rect_is_drawn_through_them() {
let (doc, display) = displayed(&[
b"<< /Type /Annot /Subtype /Ink /Rect [109.25 260.01 21.25 313.51] /BS << /W 3 >> /C [1 0 0] /InkList [[260 502.75 300 537 330 520] [280 510]] >>",
b"<< /Type /Annot /Subtype /Ink /Rect [0 0 100 100] /BS << /W 30 >> /C [1 0 0] /InkList [[10 10 90 90]] >>",
]);
let shown = list(&display, 0);
let rect = |d: &Dict| nums_of(d, b"Rect");
// The smooth curve rises past its highest point (537) to 537.33; half the width (1.5) more.
assert_rect_near(&rect(&shown[0]), [21.25, 260.01, 331.5, 538.825], "the ink's /Rect takes in its strokes");
assert_eq!(ap_bbox(&display, &shown[0]), rect(&shown[0]));
assert!(ap_content(&display, &shown[0]).contains("260 502.75 m"), "the stroke is drawn");
assert_eq!(rect(&shown[1]), [0.0, 0.0, 100.0, 100.0], "a /Rect that holds the points stays as it is");
assert_eq!(rect(&list(&doc, 0)[0]), [109.25, 260.01, 21.25, 313.51], "the source document is unchanged");
}
/// A polyline's `/Vertices` (Table 181) outside its `/Rect` are drawn, with its line endings
/// (`/LE`): the box takes in the arrow at the last vertex.
#[test]
fn polyline_whose_vertices_reach_past_its_rect_is_drawn_through_them() {
let (_, display) = displayed(&[
b"<< /Type /Annot /Subtype /PolyLine /Rect [667.5 250.69 567.75 344.44] /BS << /W 2 >> /C [0 0 1] /Vertices [250 44 350 94 285 138] >>",
b"<< /Type /Annot /Subtype /PolyLine /Rect [10 10 20 20] /BS << /W 2 >> /C [0 0 1] /LE [/None /OpenArrow] /Vertices [100 100 200 100 200 200] >>",
]);
let shown = list(&display, 0);
let rect = |d: &Dict| nums_of(d, b"Rect");
assert_eq!(rect(&shown[0]), [249.0, 43.0, 667.5, 344.44], "the polyline's /Rect takes in its vertices");
assert_eq!(ap_bbox(&display, &shown[0]), rect(&shown[0]));
assert!(ap_content(&display, &shown[0]).contains("250 44 m"), "the polyline is drawn");
// Half the width (1) and the arrow (6 + 3 × 2) past the vertices.
assert_eq!(rect(&shown[1]), [10.0, 10.0, 213.0, 213.0], "and its arrow");
}
/// A polygon's `/Vertices` outside its `/Rect` are drawn, its cloudy border (§12.5.4) included.
#[test]
fn polygon_whose_vertices_reach_past_its_rect_is_drawn_through_them() {
let (_, display) = displayed(&[
b"<< /Type /Annot /Subtype /Polygon /Rect [10 10 20 20] /BS << /W 1 >> /C [0 0 1] /IC [1 1 0] /Vertices [100 100 200 100 150 180] >>",
b"<< /Type /Annot /Subtype /Polygon /Rect [10 10 20 20] /BS << /W 2 >> /C [0 0 1] /BE << /S /C >> /Vertices [100 100 200 100 150 180] >>",
]);
let shown = list(&display, 0);
let rect = |d: &Dict| nums_of(d, b"Rect");
assert_eq!(rect(&shown[0]), [10.0, 10.0, 200.5, 180.5], "the polygon's /Rect takes in its vertices");
assert_eq!(ap_bbox(&display, &shown[0]), rect(&shown[0]));
assert!(ap_content(&display, &shown[0]).contains("100 100 m"), "the polygon is drawn");
// Half the width (1) and twice the cloud radius (2 × (6 + 1.5 × 2)) past the vertices.
assert_eq!(rect(&shown[1]), [10.0, 10.0, 219.0, 199.0], "and its cloudy border");
}
/// A line's `/L` (Table 178) outside its `/Rect` is drawn, with its line ending and its leader
/// lines. Shaped like the arrows of `issue13447.pdf`.
#[test]
fn line_whose_points_reach_past_its_rect_is_drawn_through_them() {
let (_, display) = displayed(&[
b"<< /Type /Annot /Subtype /Line /Rect [598.31 146.63 537 316.13] /BS << /W 6 >> /C [0 0 1] /LE [/None /ClosedArrow] /L [176 45 178 154] >>",
b"<< /Type /Annot /Subtype /Line /Rect [10 10 20 20] /BS << /W 2 >> /C [0 0 1] /L [100 100 200 100] /LL 20 /LLE 5 >>",
b"<< /Type /Annot /Subtype /Line /Rect [90 90 210 110] /BS << /W 2 >> /C [0 0 1] /LE [/OpenArrow /OpenArrow] /L [100 100 200 100] >>",
]);
let shown = list(&display, 0);
let rect = |d: &Dict| nums_of(d, b"Rect");
// Half the width (3) and the arrow (6 + 3 × 6) past `/L`.
assert_eq!(rect(&shown[0]), [149.0, 18.0, 598.31, 316.13], "the line's /Rect takes in /L and its arrow");
assert_eq!(ap_bbox(&display, &shown[0]), rect(&shown[0]));
assert!(ap_content(&display, &shown[0]).contains("176 45 m"), "the line is drawn");
// The leaders rise 20 + 5 above `/L`; half the width (1) past them.
assert_eq!(rect(&shown[1]), [10.0, 10.0, 201.0, 126.0], "and its leader lines");
assert_eq!(rect(&shown[2]), [90.0, 90.0, 210.0, 110.0], "a /Rect that holds /L stays as it is");
}
/// Points past the caps (`appearance::MAX_REACH_POINTS`) leave `/Rect` as it is, as do points
/// that aren't numbers; a restyle (the edit path) saves the widened `/Rect` of an ink annotation.
#[test]
fn path_reach_stops_at_the_point_cap_and_is_saved_on_restyle() {
let cap = appearance::MAX_REACH_POINTS;
let stroke = |n: usize| format!("[{}]", vec!["100 120"; n].join(" "));
let at_cap = format!("<< /Type /Annot /Subtype /Ink /Rect [10 10 20 20] /BS << /W 0 >> /C [1 0 0] /InkList [{} [200 200]] >>", stroke(cap - 1));
let past_cap = format!("<< /Type /Annot /Subtype /Ink /Rect [10 10 20 20] /BS << /W 0 >> /C [1 0 0] /InkList [{} [200 200]] >>", stroke(cap));
let (_, display) = displayed(&[at_cap.as_bytes(), past_cap.as_bytes()]);
let shown = list(&display, 0);
let rect = |d: &Dict| nums_of(d, b"Rect");
assert_eq!(rect(&shown[0]), [10.0, 10.0, 200.0, 200.0], "as many points as the cap widen the box");
assert_eq!(rect(&shown[1]), [10.0, 10.0, 20.0, 20.0], "more points than the cap keep the /Rect");
let mut doc = document_of(&[
b"<< /Type /Catalog /Pages 2 0 R >>",
b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>",
b"<< /Type /Page /Parent 2 0 R /Annots [4 0 R] >>",
b"<< /Type /Annot /Subtype /Ink /Rect [109.25 260.01 21.25 313.51] /BS << /W 3 >> /C [1 0 0] /InkList [[260 502.75 300 537 330 520]] >>",
]);
set_style(&mut doc, 0, 0, StyleChange::new(Some([0.0, 0.0, 1.0]), None, None, None), &meta("")).unwrap();
let saved = reopen(&doc);
let shown = list(&saved, 0);
assert_rect_near(&rect(&shown[0]), [21.25, 260.01, 331.5, 538.825], "the saved /Rect takes in the strokes");
assert_eq!(ap_bbox(&saved, &shown[0]), rect(&shown[0]));
}
/// A squiggle's peaks stand 1.5 above a quad's bottom edge however low the quad is, so over a quad
/// 1 high they reach past it: the widened box takes them in, with half the line's width.
#[test]
fn squiggly_peaks_over_a_low_quad_are_inside_the_widened_box() {
let (_, display) =
displayed(&[b"<< /Type /Annot /Subtype /Squiggly /Rect [10 10 20 20] /C [0 1 0] /QuadPoints [100 301 200 301 100 300 200 300] >>"]);
let shown = list(&display, 0);
// Peak 300 + 1.5, half the 0.5 line width: 301.75; the sides reach 0.75 past the quad.
assert_eq!(nums_of(&shown[0], b"Rect"), [10.0, 10.0, 200.75, 301.75]);
assert_eq!(ap_bbox(&display, &shown[0]), nums_of(&shown[0], b"Rect"));
}
/// A line's caption (`/Cap`, ISO 32000-2 Table 178) is drawn beside `/L`: the widened box takes
/// in the caption's text as well as the line.
#[test]
fn a_line_caption_is_inside_the_widened_box() {
let (_, display) = displayed(&[
b"<< /Type /Annot /Subtype /Line /Rect [10 10 20 20] /BS << /W 2 >> /C [0 0 1] /L [100 100 200 100] /Cap true /CP /Top /Contents (Caption) >>",
]);
let shown = list(&display, 0);
let rect = nums_of(&shown[0], b"Rect");
// The caption sits on top of the line: its 10-point text rises well past the stroke's 101.
assert!(rect[3] > 108.0, "{rect:?}");
assert_eq!(ap_bbox(&display, &shown[0]), rect);
}
/// A line just above the crop box whose wide stroke reaches down onto the page is drawn as far as
/// the page shows, not cut to its `/Rect`.
#[test]
fn a_stroke_that_reaches_onto_the_page_from_points_off_it_widens_the_box() {
let doc = document_of(&[
b"<< /Type /Catalog /Pages 2 0 R >>",
b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>",
b"<< /Type /Page /Parent 2 0 R /CropBox [0 0 300 400] /Annots [4 0 R] >>",
b"<< /Type /Annot /Subtype /Line /Rect [10 10 20 20] /BS << /W 6 >> /C [0 0 1] /L [100 401 200 401] >>",
]);
let display = with_missing_appearances(&doc).expect("the line needs an appearance");
let shown = list(&display, 0);
// The stroke spans 398 to 404 across and 97 to 203 along; the page shows up to 400.
assert_eq!(nums_of(&shown[0], b"Rect"), [10.0, 10.0, 203.0, 400.0]);
}
/// A cached display appearance was drawn for its page's crop box: once the crop box grows, the
/// next refresh draws the annotation again for the larger region.
#[test]
fn a_cached_appearance_is_redrawn_for_a_changed_crop_box() {
let mut doc = document_of(&[
b"<< /Type /Catalog /Pages 2 0 R >>",
b"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 612 792] >>",
b"<< /Type /Page /Parent 2 0 R /CropBox [0 0 300 400] /Annots [4 0 R] >>",
b"<< /Type /Annot /Subtype /Highlight /Rect [10 10 20 20] /C [1 1 0] /QuadPoints [100 380 500 380 100 360 500 360] >>",
]);
let mut cache = AppearanceCache::default();
let display = with_missing_appearances_cached(&doc, &mut cache).expect("drawn");
assert_eq!(nums_of(&list(&display, 0)[0], b"Rect"), [10.0, 10.0, 300.0, 380.0]);
doc.update_dict(ObjRef::new(3, 0), |d| d.set(b"CropBox".to_vec(), Object::Array([0, 0, 612, 792].map(Object::Int).to_vec()))).unwrap();
let display = with_missing_appearances_cached(&doc, &mut cache).expect("drawn");
let shown = list(&display, 0);
assert_eq!(nums_of(&shown[0], b"Rect"), [10.0, 10.0, 500.0, 380.0], "drawn for the larger crop box");
assert_eq!(ap_bbox(&display, &shown[0]), nums_of(&shown[0], b"Rect"));
}
/// Redrawing a comment drawn through its points finds its page through `/P` when that page
/// lists it, reading that one page's `/Annots` rather than every page's: here the last of 300.
#[test]
fn restyling_a_comment_drawn_through_its_points_reads_only_its_page() {
let pages = 300;
let mut objs: Vec<String> = vec!["<< /Type /Catalog /Pages 2 0 R >>".into()];
let kids: Vec<String> = (0..pages).map(|i| format!("{} 0 R", 3 + 2 * i)).collect();
objs.push(format!("<< /Type /Pages /Kids [{}] /Count {pages} /MediaBox [0 0 612 792] >>", kids.join(" ")));
for i in 0..pages {
let (page, annot) = (3 + 2 * i, 4 + 2 * i);
objs.push(format!("<< /Type /Page /Parent 2 0 R /Annots [{annot} 0 R] >>"));
objs.push(format!(
"<< /Type /Annot /Subtype /Highlight /P {page} 0 R /Rect [10 10 20 20] /C [1 1 0] /QuadPoints [100 120 200 120 100 100 200 100] >>"
));
}
let bytes: Vec<&[u8]> = objs.iter().map(|o| o.as_bytes()).collect();
let mut doc = document_of(&bytes);
let last = pages - 1;
ANNOTS_ENTRIES_COPIED.with(|c| c.set(0));
set_style(&mut doc, last, 0, StyleChange::new(Some([1.0, 0.0, 0.0]), None, None, None), &meta("")).unwrap();
let copied = ANNOTS_ENTRIES_COPIED.with(std::cell::Cell::get);
assert!(copied < 20, "{copied} annotation entries copied for one restyle");
assert_eq!(nums_of(&list(&doc, last)[0], b"Rect"), [10.0, 10.0, 200.0, 120.0], "drawn for its page");
}
/// `got` is `want` to within 0.001 in every coordinate.
fn assert_rect_near(got: &[f64], want: [f64; 4], what: &str) {
assert!(got.len() == 4 && got.iter().zip(want).all(|(g, w)| (g - w).abs() < 1e-3), "{what}: {got:?}");
}
fn nums_of(d: &Dict, key: &[u8]) -> Vec<f64> {
d.get(key).and_then(|a| a.as_array()).map(|a| a.iter().filter_map(|o| o.as_f64()).collect()).unwrap_or_default()
}
/// The `y` of every `1 0 0 1 x y Tm` in `content`: where each piece of text sits.
crates/automation/src/json.rs
+1 -1
View File
@@ -296,7 +296,7 @@ pub(crate) fn info(d: &Document) -> Value {
// Custom document properties (Document Properties ▸ Custom): /Info entries beyond the
// ones the spec names, set and cleared with doc_set_info.
"custom": i.custom.iter().map(|(name, value)| json!({ "name": name, "value": value })).collect::<Vec<_>>(),
"creator": i.creator, "producer": i.producer, "created": i.created, "modified": i.modified,
"creator": i.creator, "producer": i.producer,
"tagged": i.tagged,
"has_javascript": i.has_javascript,
// Where the document opens (/OpenAction, named destinations resolved): 1-based page and
crates/automation/src/measure.rs
+1 -1
View File
@@ -147,7 +147,7 @@ impl Automation {
Ok(json!({"point":point,"kind":s.kind,"distance":measure::distance(at,point)}))
})
.transpose()?;
Ok(json!({"snap":snap,"truncated":geometry.truncated,"intersection_limited":intersection_limited,"segments":geometry.segments.len()}))
Ok(json!({"snap":snap,"truncated":geometry.truncated,"intersection_limited":intersection_limited,"segments":geometry.segment_count()}))
}
pub(crate) fn measurement_export(&self, a: &Args) -> Result<Value> {
let all = self.doc(a)?.measurements().map_err(failed)?;
crates/automation/src/tools.rs
+1 -1
View File
@@ -1361,7 +1361,7 @@ pub fn tools() -> Vec<ToolDef> {
t("print_readiness", "Print readiness", "Verification only. Reports effective image resolution where each image is drawn (not against the page size), page boxes and bleed past the trim, output intents, fonts that are not embedded, and the PDF/A-2b structural result. Does not modify the document, embed fonts, invent a press profile, or check PDF/X.")
.ro()
.with(schema(json!({ "doc": doc() }), &["doc"])),
t("pdfa_convert", "Save as PDF/A", "Standards ▸ Save as PDF/A: fix what can be fixed for PDF/A-2b or 3b (XMP identification and metadata, an sRGB output intent, forbidden actions and JavaScript, annotation print flags, image interpolation, encryption). Returns what was fixed and what remains (e.g. fonts that aren't embedded). Undoable; save the document to keep it.")
t("pdfa_convert", "Save as PDF/A", "Standards ▸ Save as PDF/A: fix what can be fixed for PDF/A-2b or 3b (XMP identification and metadata, an sRGB output intent, forbidden actions and JavaScript, annotation print flags, image interpolation, encryption, the file header, the file identifier). Returns what was fixed and what remains (e.g. fonts that aren't embedded). Undoable; save the document to keep it.")
.cmd("standards.pdfa")
.with(schema(json!({ "doc": doc(), "level": { "type": "string", "enum": ["2b", "3b"] } }), &["doc"])),
t("standards_remove_id", "Remove PDF/A or PDF/X identification", "Remove what identifies the document as PDF/A (its pdfaid: XMP properties) or PDF/X (GTS_PDFXVersion and GTS_PDFXConformance, in the document information and as pdfxid: XMP properties), e.g. after edits it no longer meets. Output intents and other metadata stay. Fails when the document declares nothing of the kind. Undoable.")
crates/cos/src/lib.rs
+1 -1
View File
@@ -30,7 +30,7 @@ pub use parser::{Lexer, is_whitespace, parse_indirect};
pub use pdfcraft_crypt::{
Algorithm, Auth, Method as CryptMethod, NewEncryption, Permissions, SecurityHandler, effective_password, pdfdoc_byte, pdfdoc_char,
};
pub use writer::{SaveOptions, civil_from_days, pdf_date, reachable_objects, serialize, write_full, write_incremental};
pub use writer::{SaveOptions, civil_from_days, new_file_id, pdf_date, reachable_objects, serialize, write_full, write_incremental};
#[derive(Debug, thiserror::Error, Clone, PartialEq)]
pub enum CosError {
crates/cos/src/writer.rs
+11 -3
View File
@@ -789,13 +789,21 @@ fn stamp_mod_date(doc: &mut Document, opts: &SaveOptions) {
}
}
/// Keep an existing `/ID`, or create one (two equal 16-byte strings, §14.4).
/// Keep any existing two-element `/ID` array as it is, even a malformed one such as `[<> <>]` or `[1 2]` (an ordinary
/// save preserves what it doesn't understand; Save as PDF/A replaces or completes it), or create one with
/// [`new_file_id`] when there is none or it isn't a two-element array.
fn ensure_id(trailer: &mut Dict, opts: &SaveOptions, content: &[u8]) {
if matches!(trailer.get(b"ID"), Some(Object::Array(a)) if a.len() == 2) {
return;
}
trailer.set(b"ID".to_vec(), new_file_id(content, opts.id_seed));
}
/// A new file identifier for a file whose bytes are `content` (ISO 32000-1 14.4): an array of
/// two equal 16-byte strings, the same for the same content and `seed`.
pub fn new_file_id(content: &[u8], seed: u64) -> Object {
// FNV-1a over a sample of the content and the seed: stable, dependency-free.
let mut h: u64 = 0xcbf2_9ce4_8422_2325 ^ opts.id_seed;
let mut h: u64 = 0xcbf2_9ce4_8422_2325 ^ seed;
for b in content.iter().step_by((content.len() / 4096).max(1)).chain(content.iter().rev().take(64)) {
h ^= *b as u64;
h = h.wrapping_mul(0x0100_0000_01b3);
@@ -804,7 +812,7 @@ fn ensure_id(trailer: &mut Dict, opts: &SaveOptions, content: &[u8]) {
id.extend_from_slice(&h.to_be_bytes());
id.extend_from_slice(&h.rotate_left(29).wrapping_mul(0x9E37_79B9_7F4A_7C15).to_be_bytes());
let s = Object::String(PdfString { bytes: id, hex: true });
trailer.set(b"ID".to_vec(), Object::Array(vec![s.clone(), s]));
Object::Array(vec![s.clone(), s])
}
/// Give a rewritten file a new identifier (§14.4), so that a redacted or sanitized copy does not
crates/edit/src/text.rs
+3 -3
View File
@@ -25,7 +25,7 @@ mod regions;
#[path = "text_session.rs"]
mod session;
pub use fallback::strip_subset_tag;
use regions::{alignment_frame, group_blocks, is_centred, rewrite_styled_block};
use regions::{alignment_frame, group_blocks, is_centred, rewrite_changed_words, rewrite_styled_block};
pub use session::TextSession;
mod blocks;
@@ -42,7 +42,7 @@ pub use blocks::{
BlockKey, BlockStyle, RangeStyle, ReadingBlocks, TextBlock, block_overlapping, clear_block, reading_blocks, reading_blocks_checked,
replace_block, text_blocks,
};
use blocks::{paragraph_separator, row_break};
use blocks::{kept_words, paragraph_separator, row_break};
pub use editor::{TextEditLayout, TextEditorFont, register_text_font, text_edit_layout, text_editor_fonts};
use editor::{builtin_thai_choice, layout_draft};
#[cfg(test)]
@@ -55,12 +55,12 @@ use fallback::{
use fallback::{add_type3_font, is_type3_fallback};
pub(crate) use fallback::{glyph_path, type3_font_dict, unique_font_name};
use interp::{Origin, Paint, union};
use lines::page_dict;
pub(crate) use lines::{
ContentStream, FormBudget, Read, content_streams, content_streams_within, dict_in_place, drawn_before_damage, first_op_after_damage, form_call,
joined, resource_dict, rewritten_contents, splice,
};
pub use lines::{MAX_FORM_DEPTH, MAX_FORM_VISITS, glyph_text_lines, text_lines};
use lines::{page_dict, word_lines};
pub use rewrite::{LineEdit, replace_line, rewrite_block};
use rewrite::{coincident_ops, rewrite_block_from};
pub(crate) use rewrite::{paragraph_ops, reveal_ops, selectable_blocks, show_starts, translate_blocks};
crates/edit/src/text/blocks.rs
+30
View File
@@ -44,6 +44,12 @@ impl TextBlock {
(self.invisible && self.size.is_finite() && self.size > 0.0).then_some(self.size)
}
/// Whether `draft` says what the paragraph says, give or take space around it. Spacing inside
/// it has to match: a typed second space is a change.
pub fn same_words(&self, draft: &str) -> bool {
draft.trim() == self.editor_text.trim()
}
/// Preserve styles outside the changed ranges; inserted characters inherit the surrounding
/// source run. The texts are aligned word by word, so words left untouched between two edits
/// keep their own runs (#2198); past [`MAX_CHANGED_WORDS`] the draft counts as one change.
@@ -163,6 +169,23 @@ fn edit_script<T: PartialEq>(a: &[T], b: &[T], max: usize) -> Option<Vec<Step>>
None
}
/// For each of `old`'s words, the index in `new` of the word it stays as in the shortest edit from
/// one to the other (`None` when it goes), or `None` overall past [`MAX_CHANGED_WORDS`] changes.
pub(crate) fn kept_words<T: PartialEq>(old: &[T], new: &[T]) -> Option<Vec<Option<usize>>> {
let mut kept = vec![None; old.len()];
let (mut o, mut n) = (0usize, 0usize);
for step in edit_script(old, new, MAX_CHANGED_WORDS)? {
if step == Step::Keep
&& let Some(slot) = kept.get_mut(o)
{
*slot = Some(n);
}
o = o.saturating_add(usize::from(step != Step::Insert));
n = n.saturating_add(usize::from(step != Step::Delete));
}
Some(kept)
}
/// Walk Myers' saved frontiers back from the end to the edit script.
fn backtrack(trace: &[Vec<isize>], n: isize, m: isize, max: isize) -> Vec<Step> {
let at = |v: &[isize], k: isize| usize::try_from(k + max + 1).ok().and_then(|i| v.get(i).copied()).unwrap_or(0);
@@ -444,6 +467,13 @@ pub struct BlockStyle {
/// For an invisible paragraph ([`TextBlock::invisible`]): fill its old area with this colour
/// (RGB 0–1) before the new text, hiding the scanned word beneath it.
pub cover: Option<[f64; 3]>,
/// For an invisible paragraph whose formatting nobody asked to change: cover and redraw only
/// the words the draft changes, and leave the rest as the scan shows them (#3313). The session
/// sets this; a caller that sets it should also give `cover`.
pub changed_words_only: bool,
/// How far across [`BlockStyle::cover`] may reach, whatever its padding: the edges of the words
/// on either side that stay (user space).
pub cover_between: Option<[f64; 2]>,
}
#[derive(Clone, Debug, PartialEq)]
crates/edit/src/text/lines.rs
+12 -5
View File
@@ -291,7 +291,13 @@ fn fill_color(fill: &[Op]) -> [f64; 3] {
/// The lines of text on a page (0-based), in content order.
pub fn text_lines(doc: &Document, page: usize) -> Result<Vec<TextLine>, EditError> {
text_lines_in(doc, &page_dict(doc, page)?, true)
text_lines_in(doc, &page_dict(doc, page)?, true, false)
}
/// [`text_lines`] with every show of invisible text a line of its own, however alike its
/// neighbours: an OCR layer's words, which an edit can keep or change one by one.
pub(crate) fn word_lines(doc: &Document, page: usize) -> Result<Vec<TextLine>, EditError> {
text_lines_in(doc, &page_dict(doc, page)?, true, true)
}
/// The lines `page`'s glyphs show, in content order, without `/ActualText` replacements: what the
@@ -299,11 +305,12 @@ pub fn text_lines(doc: &Document, page: usize) -> Result<Vec<TextLine>, EditErro
/// not trust that metadata (the redaction audit). Takes the page itself, so a caller going
/// through every page doesn't walk the page tree again for each.
pub fn glyph_text_lines(doc: &Document, page: &pdfcraft_model::Page) -> Result<Vec<TextLine>, EditError> {
text_lines_in(doc, page, false)
text_lines_in(doc, page, false, false)
}
/// [`text_lines`] of `p`; with `actual_text`, `/ActualText` replaces the glyphs it spans.
fn text_lines_in(doc: &Document, p: &pdfcraft_model::Page, actual_text: bool) -> Result<Vec<TextLine>, EditError> {
/// [`text_lines`] of `p`; with `actual_text`, `/ActualText` replaces the glyphs it spans, and with
/// `invisible_apart`, no show of invisible text joins the one before it.
fn text_lines_in(doc: &Document, p: &pdfcraft_model::Page, actual_text: bool, invisible_apart: bool) -> Result<Vec<TextLine>, EditError> {
let res = p.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default();
let mut cache = HashMap::new();
let mut lines: Vec<TextLine> = Vec::new();
@@ -345,7 +352,7 @@ fn text_lines_in(doc: &Document, p: &pdfcraft_model::Page, actual_text: bool) ->
&& near
// An OCR layer is one text object over the whole scan: its words continue a line
// only across a word space, not across a table cell or a tab stop.
&& (s.state.render_mode != 3 || gap < pen.size * 0.6)
&& (s.state.render_mode != 3 || (gap < pen.size * 0.6 && !invisible_apart))
});
if joins && let Some(l) = lines.last_mut() {
let gap = last.map_or(0.0, |(_, pen)| pen.continues(&s, 0.0).1);
crates/edit/src/text/rewrite.rs
+14 -6
View File
@@ -7,7 +7,7 @@ use super::interp::{Carry, Origin, TextState, interpret};
use super::lines::{content_streams, joined, page_dict, rewritten_contents, splice, text_lines};
use super::{
Dict, Document, EditError, HashMap, Matrix, Metrics, Object, Op, PdfString, TextLine, TextSession, alignment_frame, group_blocks, is_centred,
parse, rewrite_styled_block,
parse, rewrite_changed_words, rewrite_styled_block,
};
/// What replacing a line did.
@@ -851,6 +851,9 @@ pub(crate) fn rewrite_block_from(
if b.invisible {
crate::check_on_top(doc, &p)?;
}
if let Some(edit) = text.and_then(|draft| rewrite_changed_words(doc, page, &b, &lines, draft, style, max_glyphs)) {
return Ok(edit);
}
let mut res = p.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default();
let mut fonts_res = res.get(b"Font").map(|f| doc.resolve(f)).and_then(|f| f.as_dict().cloned()).unwrap_or_default();
let dispatch = choose_rewrite_path(doc, &fonts_res, &b, &members, first, text, style)?;
@@ -1018,6 +1021,14 @@ pub(crate) fn rewrite_block_from(
/// rewritten invisible paragraph, drawn in the space whose CTM is `ctm`. Nothing for visible text.
/// The paragraph's `q`/`Q` restores the rendering mode for the invisible text after it; one
/// painted over where it was is drawn on top of the page instead ([`crate::draw_on_top`]).
/// What covering the word in `rect` paints: scanned ink reaches a little past the recognised box,
/// but not past `between`, the words on either side that stay. It always covers the word itself.
pub(crate) fn cover_box(rect: [f64; 4], between: Option<[f64; 2]>) -> [f64; 4] {
let pad = ((rect[3] - rect[1]).abs() * 0.1).clamp(0.5, 2.0);
let [left, right] = between.unwrap_or([f64::NEG_INFINITY, f64::INFINITY]);
[(rect[0] - pad).max(left.min(rect[0])), rect[1] - pad, (rect[2] + pad).min(right.max(rect[2])), rect[3] + pad]
}
pub(crate) fn reveal_ops(block: &TextBlock, lines: &[TextLine], style: &BlockStyle, ctm: [f64; 6]) -> Result<Vec<Op>, EditError> {
if !block.invisible {
return Ok(Vec::new());
@@ -1031,11 +1042,8 @@ pub(crate) fn reveal_ops(block: &TextBlock, lines: &[TextLine], style: &BlockSty
let back = Matrix(ctm).invert().ok_or_else(|| EditError::Invalid("the paragraph is drawn in a space it can't be covered in".into()))?;
ops.push(Op::new("q", vec![]));
ops.push(Op::new("rg", colour.map(n).to_vec()));
for rect in block.lines.iter().filter_map(|i| lines.get(*i)).map(|l| l.rect) {
// Scanned ink reaches a little past the recognised box.
let pad = ((rect[3] - rect[1]).abs() * 0.1).clamp(0.5, 2.0);
let corners =
[[rect[0] - pad, rect[1] - pad], [rect[2] + pad, rect[1] - pad], [rect[2] + pad, rect[3] + pad], [rect[0] - pad, rect[3] + pad]];
for rect in block.lines.iter().filter_map(|i| lines.get(*i)).map(|l| cover_box(l.rect, style.cover_between)) {
let corners = [[rect[0], rect[1]], [rect[2], rect[1]], [rect[2], rect[3]], [rect[0], rect[3]]];
for (i, [x, y]) in corners.into_iter().enumerate() {
let (x, y) = back.apply(x, y);
if !(x.is_finite() && y.is_finite()) {
crates/edit/src/text_regions.rs
+6 -3
View File
@@ -3,15 +3,18 @@
use super::{
BlockStyle, CraftFont, Dict, Document, EditError, HashMap, LineEdit, MAX_TYPE3_GLYPHS, Matrix, Metrics, Object, Op, PdfString, Pieces,
RangeStyle, TextBlock, TextLine, Type3Glyph, builtin_thai_choice, coincident_ops, content_streams, fallback_face, japanese_glyph_from,
needs_type3, no_fallback_font, no_glyph_for, page_dict, paragraph_separator, reveal_ops, rewritten_contents, row_break, same_base_font,
same_font, source_encodes, source_family, source_program, source_show, source_space, source_width, splice, strip_subset_tag, type3_faces,
type3_glyph, union, write_type3_glyphs,
kept_words, needs_type3, no_fallback_font, no_glyph_for, page_dict, paragraph_separator, reveal_ops, rewritten_contents, row_break,
same_base_font, same_font, source_encodes, source_family, source_program, source_show, source_space, source_width, splice, strip_subset_tag,
type3_faces, type3_glyph, union, word_lines, write_type3_glyphs,
};
#[path = "text_regions/grouping.rs"]
mod grouping;
#[path = "text_regions/ocr_words.rs"]
mod ocr_words;
#[path = "text_regions/styled.rs"]
mod styled;
pub(super) use grouping::{alignment_frame, group_blocks, group_reading_blocks, is_centred};
pub(super) use ocr_words::rewrite_changed_words;
pub(super) use styled::rewrite_styled_block;
crates/edit/src/text_regions/ocr_words.rs
+436
View File
@@ -0,0 +1,436 @@
//! Edit text on a scanned page, word by word: a text edit of an OCR phrase covers and redraws only
//! the words it changes, and the rest stay as the scan shows them (#3313). Each changed stretch is
//! rewritten as a phrase of its own, which is how a phrase beside others in one OCR layer already
//! rewrites. A deleted word leaves its gap: the words after it are pixels, and don't move.
use std::collections::{BTreeMap, BTreeSet, HashMap};
use std::ops::Range;
use super::grouping::row_rect;
use super::styled::rewrite_styled_block;
use super::{BlockStyle, Document, EditError, LineEdit, TextBlock, TextLine, kept_words, word_lines};
/// How many times a replacement may be refitted to the room it has before the whole phrase is
/// redrawn instead. Each try rewrites a copy of the document.
const MAX_TRIES: usize = 16;
/// The narrowest a replacement is squeezed to stay in its word's place. Most replacements are
/// only a little wider than the word they replace, and taking over the next word pushes that one
/// along too, and so on down the line. Past this it would be hard to read, so it takes the word over.
const MIN_SQUEEZE: f64 = 0.75;
/// A stretch of the phrase's word runs that changes, the text that replaces it (empty when it is
/// deleted), and how much that text is squeezed across to fit.
#[derive(Debug, PartialEq)]
struct Change {
runs: Range<usize>,
/// Which of the draft's words replace them.
words: Range<usize>,
text: String,
squeeze: Option<f64>,
}
/// The stretches `new` changes, given how many words each run holds and which word of `new` each
/// old word stays as ([`kept_words`]). A run in `redrawn` counts as changed, and new words with no
/// run of their own to replace add the run after them to it (or the last one, at the end).
fn changes(runs: &[usize], kept_as: &[Option<usize>], new: &[&str], redrawn: &mut BTreeSet<usize>) -> Vec<Change> {
// Each pass either returns or adds a run to `redrawn`, so this ends.
'plan: loop {
let mut out = Vec::new();
let (mut first_run, mut first_new, mut token) = (0usize, 0usize, 0usize);
for (run, count) in runs.iter().enumerate() {
let words = kept_as.get(token..token.saturating_add(*count)).unwrap_or_default();
token = token.saturating_add(*count);
let at = words.first().copied().flatten();
let kept = !redrawn.contains(&run) && !words.is_empty() && words.iter().enumerate().all(|(i, w)| *w == at.map(|a| a.saturating_add(i)));
let (true, Some(at)) = (kept, at) else { continue };
if first_run == run && first_new < at {
redrawn.insert(run);
continue 'plan;
}
if first_run < run {
out.push(Change {
runs: first_run..run,
words: first_new..at,
text: new.get(first_new..at).unwrap_or_default().join(" "),
squeeze: None,
});
}
first_run = run.saturating_add(1);
first_new = at.saturating_add(*count);
}
if first_run < runs.len() {
out.push(Change {
runs: first_run..runs.len(),
words: first_new..new.len(),
text: new.get(first_new..).unwrap_or_default().join(" "),
squeeze: None,
});
} else if first_new < new.len()
&& let Some(last) = runs.len().checked_sub(1)
{
redrawn.insert(last);
continue 'plan;
}
return out;
}
}
/// Each word of `text` as a byte range.
fn word_ranges(text: &str) -> Vec<Range<usize>> {
let mut out = Vec::new();
let mut start = None;
for (at, c) in text.char_indices() {
match (c.is_whitespace(), start) {
(true, Some(from)) => {
out.push(from..at);
start = None;
}
(false, None) => start = Some(at),
_ => {}
}
}
if let Some(from) = start {
out.push(from..text.len());
}
out
}
/// Whether `plan` can stand for the spacing in `draft`, whose words are `words`. Spaces beside a
/// change are nothing to draw: Backspacing over a word leaves one either side of where it was, and
/// the words around it stay where the scan has them. Anything else that isn't one space has to be
/// drawn, inside the new words or between two words nothing else changed, which only the whole
/// rewrite can do.
fn spacing_fits(draft: &str, words: &[Range<usize>], plan: &[Change]) -> bool {
words.windows(2).enumerate().all(|(index, pair)| {
let [before, after] = pair else { return true };
let gap = draft.get(before.end..after.start).unwrap_or(" ");
let boundary = index.saturating_add(1);
gap == " " || (gap.chars().all(|c| c == ' ') && plan.iter().any(|change| change.words.start == boundary || change.words.end == boundary))
})
}
/// The phrase's stretch `indices` of `lines`, as a paragraph of its own. It keeps the whole
/// phrase's size, so what is redrawn is set alike wherever in the phrase it is.
fn stretch(phrase: &TextBlock, lines: &[TextLine], indices: &[usize]) -> Result<TextBlock, EditError> {
let first = indices.first().and_then(|i| lines.get(*i)).ok_or_else(|| EditError::Invalid("the phrase has no such words".into()))?;
let rect = row_rect(indices, lines).ok_or_else(|| EditError::Invalid("the phrase has no such words".into()))?;
let mut text = String::new();
let mut spans = Vec::with_capacity(indices.len());
for index in indices {
let Some(line) = lines.get(*index) else { continue };
if !text.is_empty() && !text.ends_with(char::is_whitespace) && !line.text.starts_with(char::is_whitespace) {
text.push(' ');
}
let start = text.len();
text.push_str(&line.text);
spans.push((start..text.len(), *index));
}
Ok(TextBlock {
reflow_text: text.clone(),
editor_text: text.clone(),
text,
spans,
rect,
base_font: first.base_font.clone(),
size: phrase.size,
bold: first.bold,
italic: first.italic,
color: first.color,
rotation: first.rotation(),
lines: indices.to_vec(),
rows: vec![indices.to_vec()],
invisible: true,
})
}
fn key(line: &TextLine) -> (&str, [i64; 4]) {
// Saturating casts: a non-finite edge only makes the comparison below fail, safely.
(line.text.as_str(), line.rect.map(|v| (v * 100.0).round() as i64))
}
/// How far right a rewrite of the runs `indices` drew (`None` when it drew nothing), or an error
/// when it moved or lost any other text on the page: text positioned after the end of the text
/// before it, say, depends on the words that went.
fn drawn_right(before: &[TextLine], indices: &[usize], after: &[TextLine]) -> Result<Option<f64>, EditError> {
let mut unmatched: HashMap<_, usize> = HashMap::new();
for line in before {
let count = unmatched.entry(key(line)).or_default();
*count = count.saturating_add(1);
}
let mut drawn = Vec::new();
for line in after {
match unmatched.get_mut(&key(line)).filter(|count| **count > 0) {
Some(count) => *count -= 1,
None => drawn.push(line),
}
}
// Visible text beside what was drawn reads as one line with it. It is still where it was.
let joined = |line: &TextLine| {
let [x0, y0, x1, y1] = line.rect;
drawn.iter().any(|d| {
d.text.contains(line.text.trim()) && d.rect[0] <= x0 + 0.01 && d.rect[1] <= y0 + 0.01 && d.rect[2] >= x1 - 0.01 && d.rect[3] >= y1 - 0.01
})
};
let moved = before
.iter()
.enumerate()
.any(|(index, line)| !indices.contains(&index) && unmatched.get(&key(line)).is_some_and(|c| *c > 0) && !joined(line));
if moved {
return Err(EditError::Invalid("rewriting these words alone would move the text around them".into()));
}
Ok(drawn.iter().map(|line| line.rect[2]).reduce(f64::max))
}
/// Rewrite each change of `plan` in `doc`, last first: a rewrite goes where its words were or at
/// the end of the page, so the words before it keep their indexes (an error if they don't). Gives
/// how far right each drew.
fn apply(
doc: &mut Document,
page: usize,
phrase: &TextBlock,
members: &[usize],
plan: &[Change],
style: &BlockStyle,
max_glyphs: usize,
) -> Result<(Vec<Option<f64>>, Option<String>), EditError> {
// The stretch starts where its words did. Its own box would centre a word that happens to
// sit in the middle of the page.
let mut style = BlockStyle { align: Some(crate::added::Align::Left), changed_words_only: false, ..style.clone() };
let mut rights = vec![None; plan.len()];
let mut substituted = None;
let original = word_lines(doc, page)?;
for (at, change) in plan.iter().enumerate().rev() {
let indices = members.get(change.runs.clone()).ok_or_else(|| EditError::Invalid("the phrase has no such words".into()))?;
let before = word_lines(doc, page)?;
// The words up to the next that stays must still be where they were. A rewrite also takes
// away copies of its words, and a copy earlier on the page moves these up the list.
let up_to = change.runs.end.saturating_add(1).min(members.len());
let same = |index: &usize| before.get(*index).map(key) == original.get(*index).map(key);
if !members.get(..up_to).unwrap_or_default().iter().all(same) {
return Err(EditError::Invalid("rewriting these words alone would lose track of the words before them".into()));
}
let part = stretch(phrase, &before, indices)?;
// Relative to the scale the words have, which a layer may have narrowed already.
let own_scale = indices.first().and_then(|i| before.get(*i)).map_or(1.0, |line| line.origin.state.scale);
style.scale = change.squeeze.map(|squeeze| squeeze * own_scale * 100.0);
// The words on either side stay, with all their ink.
let edge = |run: Option<usize>| run.and_then(|run| members.get(run)).and_then(|i| before.get(*i)).map(|line| line.rect);
style.cover_between = Some([
edge(change.runs.start.checked_sub(1)).map_or(f64::NEG_INFINITY, |rect| rect[2]),
edge(Some(change.runs.end)).map_or(f64::INFINITY, |rect| rect[0]),
]);
let edit = rewrite_styled_block(doc, page, &part, &before, Some(&change.text), &style, max_glyphs)?;
substituted = substituted.or(edit.substituted);
let right = drawn_right(&before, indices, &word_lines(doc, page)?)?;
if right.is_none() && !change.text.is_empty() {
return Err(EditError::Invalid("the new words were not drawn".into()));
}
if let Some(slot) = rights.get_mut(at) {
*slot = right;
}
}
Ok((rights, substituted))
}
/// Whether a change that drew as far as `right` needs another try. It should end by the middle of
/// the gap before the word after it, so the two don't touch: within [`MIN_SQUEEZE`] it is squeezed
/// to, and otherwise it takes that word over.
fn make_room(change: &Change, right: f64, boxes: &[[f64; 4]], squeezes: &mut BTreeMap<usize, f64>, redrawn: &mut BTreeSet<usize>) -> bool {
let (Some(first), Some(last), Some(next)) =
(boxes.get(change.runs.start), change.runs.end.checked_sub(1).and_then(|run| boxes.get(run)), boxes.get(change.runs.end))
else {
return false;
};
let limit = (last[2] + next[0]) / 2.0;
// Widths scale in proportion, so one squeeze lands within this; the rest is rounding.
if right <= limit + 0.25 {
return false;
}
let squeeze = change.squeeze.unwrap_or(1.0) * (limit - first[0]) / (right - first[0]);
if squeeze >= MIN_SQUEEZE {
squeezes.insert(change.runs.start, squeeze);
} else {
squeezes.remove(&change.runs.start);
redrawn.insert(change.runs.end);
}
true
}
/// Rewrite only the words `draft` changes in the invisible phrase `phrase`, or `None` when this
/// edit is not one that can (it changes the formatting or the lines, the phrase is turned or
/// right-to-left, or its words can't be told apart or kept in place): the caller then rewrites the
/// whole phrase. Nothing changes in `doc` unless the whole edit works.
pub(crate) fn rewrite_changed_words(
doc: &mut Document,
page: usize,
phrase: &TextBlock,
joined: &[TextLine],
draft: &str,
style: &BlockStyle,
max_glyphs: usize,
) -> Option<LineEdit> {
let right_to_left = |text: &str| text.chars().any(crate::bidi::is_rtl);
if !style.changed_words_only
|| !phrase.invisible
|| phrase.rows.len() != 1
|| draft.contains(['\n', '\r'])
|| right_to_left(draft)
|| right_to_left(&phrase.text)
{
return None;
}
let lines = word_lines(doc, page).ok()?;
let phrase_ops: BTreeSet<usize> = phrase.lines.iter().filter_map(|i| joined.get(*i)).flat_map(|line| line.ops.iter().copied()).collect();
let members: Vec<usize> =
lines.iter().enumerate().filter(|(_, line)| line.ops.first().is_some_and(|op| phrase_ops.contains(op))).map(|(index, _)| index).collect();
if members.iter().filter_map(|i| lines.get(*i)).any(|line| !line.upright() || line.visual_order) {
return None;
}
let runs: Vec<Vec<&str>> = members.iter().filter_map(|i| lines.get(*i)).map(|line| line.text.split_whitespace().collect()).collect();
let old: Vec<&str> = runs.iter().flatten().copied().collect();
let new: Vec<&str> = draft.split_whitespace().collect();
// Words the layer doesn't show apart (a run per letter, say) can't be kept apart.
if !phrase.editor_text.split_whitespace().eq(old.iter().copied()) {
return None;
}
// A single word that changes has nothing to keep. With the same words, the plan is empty.
if old.len() < 2 && old != new {
return None;
}
let kept_as = kept_words(&old, &new)?;
let counts: Vec<usize> = runs.iter().map(Vec::len).collect();
let boxes: Vec<[f64; 4]> = members.iter().filter_map(|i| lines.get(*i)).map(|line| line.rect).collect();
let mut redrawn = BTreeSet::new();
let mut squeezes = BTreeMap::new();
for _ in 0..MAX_TRIES {
let mut plan = changes(&counts, &kept_as, &new, &mut redrawn);
if !spacing_fits(draft, &word_ranges(draft), &plan) {
return None;
}
for change in &mut plan {
change.squeeze = squeezes.get(&change.runs.start).copied();
}
let mut trial = doc.clone();
let (rights, substituted) = apply(&mut trial, page, phrase, &members, &plan, style, max_glyphs).ok()?;
let mut again = false;
for (change, right) in plan.iter().zip(&rights) {
if let Some(right) = right {
again |= make_room(change, *right, &boxes, &mut squeezes, &mut redrawn);
}
}
if !again {
*doc = trial;
return Some(LineEdit { substituted });
}
}
None
}
#[cfg(test)]
mod tests {
use super::*;
fn plan(old: &[&[&str]], new: &str) -> Vec<(Range<usize>, String)> {
let flat: Vec<&str> = old.iter().flat_map(|run| run.iter().copied()).collect();
let new: Vec<&str> = new.split_whitespace().collect();
let counts: Vec<usize> = old.iter().map(|run| run.len()).collect();
changes(&counts, &kept_words(&flat, &new).unwrap(), &new, &mut BTreeSet::new()).into_iter().map(|c| (c.runs, c.text)).collect()
}
#[test]
fn only_the_runs_a_draft_changes_are_planned() {
let abc: &[&[&str]] = &[&["a"], &["b"], &["c"]];
assert_eq!(plan(abc, "a b c"), []);
assert_eq!(plan(abc, "a c"), [(1..2, String::new())], "a deletion");
assert_eq!(plan(abc, "a x c"), [(1..2, "x".to_owned())], "a replacement");
assert_eq!(plan(abc, "x b y"), [(0..1, "x".to_owned()), (2..3, "y".to_owned())], "two apart");
assert_eq!(plan(abc, "x y c"), [(0..2, "x y".to_owned())], "two together");
assert_eq!(plan(abc, ""), [(0..3, String::new())], "everything");
}
#[test]
fn new_words_with_no_run_to_replace_take_a_neighbour() {
let abc: &[&[&str]] = &[&["a"], &["b"], &["c"]];
assert_eq!(plan(abc, "x a b c"), [(0..1, "x a".to_owned())], "at the start");
assert_eq!(plan(abc, "a x b c"), [(1..2, "x b".to_owned())], "in the middle");
assert_eq!(plan(abc, "a b c x"), [(2..3, "c x".to_owned())], "at the end");
}
#[test]
fn a_run_of_several_words_is_kept_only_whole() {
let runs: &[&[&str]] = &[&["a", "b"], &["c"]];
assert_eq!(plan(runs, "a b x"), [(1..2, "x".to_owned())]);
assert_eq!(plan(runs, "a x c"), [(0..1, "a x".to_owned())], "one of its words changed");
assert_eq!(plan(runs, "a x b c"), [(0..1, "a x b".to_owned())], "a word went in between its words");
}
#[test]
fn a_drafts_words_are_found_by_their_bytes() {
assert!(word_ranges("").is_empty() && word_ranges(" ").is_empty());
assert_eq!(word_ranges("ab é c "), [0..2, 4..6, 7..8]);
}
/// The plan for `draft` against the words a, b and c, and whether it stands for its spacing.
fn fits(draft: &str) -> bool {
let (old, new): (Vec<&str>, Vec<&str>) = (vec!["a", "b", "c"], draft.split_whitespace().collect());
let plan = changes(&[1, 1, 1], &kept_words(&old, &new).unwrap(), &new, &mut BTreeSet::new());
spacing_fits(draft, &word_ranges(draft), &plan)
}
#[test]
fn spaces_beside_a_change_are_nothing_to_draw() {
assert!(fits("a b c"));
assert!(fits("a b c "), "after the words");
assert!(fits(" a b c"), "before them");
assert!(fits("a c"), "either side of a deleted word");
assert!(fits(" b c"), "where a first word went");
assert!(fits("a x c"), "either side of a replacement");
}
#[test]
fn spacing_that_has_to_be_drawn_doesnt_fit() {
assert!(!fits("a b c"), "between words nothing else changed");
assert!(!fits("x y c"), "inside the new words");
assert!(!fits("a\u{3000}x c"), "another kind of space, even beside a change");
assert!(!fits("a\u{a0}x c"));
}
/// Word boxes 10 wide, with a gap of 4 between them.
const BOXES: [[f64; 4]; 3] = [[0.0, 0.0, 10.0, 10.0], [14.0, 0.0, 24.0, 10.0], [28.0, 0.0, 38.0, 10.0]];
fn room_for(right: f64) -> (bool, BTreeMap<usize, f64>, BTreeSet<usize>) {
let (mut squeezes, mut redrawn) = (BTreeMap::new(), BTreeSet::new());
let change = Change { runs: 0..1, words: 0..1, text: "x".to_owned(), squeeze: None };
(make_room(&change, right, &BOXES, &mut squeezes, &mut redrawn), squeezes, redrawn)
}
#[test]
fn a_word_that_fits_is_left_alone() {
assert_eq!(room_for(12.0), (false, BTreeMap::new(), BTreeSet::new()), "it ends in the gap's first half");
let last = Change { runs: 2..3, words: 2..3, text: "x".to_owned(), squeeze: None };
assert!(!make_room(&last, 900.0, &BOXES, &mut BTreeMap::new(), &mut BTreeSet::new()), "the last word has the page to grow into");
}
#[test]
fn a_word_a_little_too_wide_is_squeezed_to_the_middle_of_the_gap() {
let (again, squeezes, redrawn) = room_for(15.0);
assert!(again && redrawn.is_empty());
assert_eq!(squeezes.get(&0).copied(), Some(12.0 / 15.0));
}
#[test]
fn a_word_far_too_wide_takes_over_the_next_one() {
let (again, squeezes, redrawn) = room_for(30.0);
assert!(again && squeezes.is_empty());
assert_eq!(redrawn, BTreeSet::from([1]));
}
#[test]
fn a_redrawn_run_joins_the_change_beside_it() {
let (counts, old, new) = ([1, 1, 1], ["a", "b", "c"], ["x", "b", "c"]);
let mut redrawn = BTreeSet::from([1]);
let planned = changes(&counts, &kept_words(&old, &new).unwrap(), &new, &mut redrawn);
assert_eq!(planned, [Change { runs: 0..2, words: 0..2, text: "x b".to_owned(), squeeze: None }]);
}
}
crates/engine/src/document.rs
+18 -5
View File
@@ -304,6 +304,22 @@ impl Document {
measure::snap::geometry(&e.cos, page).map_err(|e| e.to_string())
}
/// [`Self::measurement_paths`] within `limits`.
pub fn measurement_paths_within(&self, page: usize, limits: measure::snap::Limits) -> Result<measure::snap::Geometry, String> {
let e = self.editor.as_ref().ok_or("the document can't be read")?;
measure::snap::geometry_with_limits(&e.cos, page, limits).map_err(|e| e.to_string())
}
/// [`Self::measurement_paths`] as work to run away from the UI: a heavy drawing takes
/// seconds to read (millions of segments). It works on a copy of the document as it is now.
pub fn measurement_paths_job(&self, page: usize) -> impl FnOnce() -> Result<measure::snap::Geometry, String> + Send + 'static {
let cos = self.editor.as_ref().map(|e| e.cos.clone());
move || {
let cos = cos.ok_or("the document can't be read")?;
measure::snap::geometry(&cos, page).map_err(|e| e.to_string())
}
}
/// A counter that changes with every edit (for caches of derived data).
pub fn edit_generation(&self) -> u64 {
self.generation
@@ -763,9 +779,8 @@ pub(crate) fn same_pages(a: &[pdfcraft_render::PageInfo], b: &[pdfcraft_render::
}
/// Each document-information field of `info` with its value in `cos` (`/Info`, else XMP), read
/// the way the full inspection reads them: NULs and spaces trimmed, empty is none, dates shown as
/// [`pdfcraft_render::pretty_date`] shows them.
pub(crate) fn info_fields<'a>(info: &'a mut DocInfo, cos: &pdfcraft_cos::Document) -> [(&'a mut Option<String>, Option<String>); 8] {
/// the way the full inspection reads them: NULs and spaces trimmed, empty is none.
pub(crate) fn info_fields<'a>(info: &'a mut DocInfo, cos: &pdfcraft_cos::Document) -> [(&'a mut Option<String>, Option<String>); 6] {
let read = |key| pdfcraft_organize::info(cos, key).map(|v| v.trim_matches('\0').trim().to_string()).filter(|v| !v.is_empty());
[
(&mut info.title, read("Title")),
@@ -774,8 +789,6 @@ pub(crate) fn info_fields<'a>(info: &'a mut DocInfo, cos: &pdfcraft_cos::Documen
(&mut info.keywords, read("Keywords")),
(&mut info.creator, read("Creator")),
(&mut info.producer, read("Producer")),
(&mut info.created, read("CreationDate").map(|d| pdfcraft_render::pretty_date(&d))),
(&mut info.modified, read("ModDate").map(|d| pdfcraft_render::pretty_date(&d))),
]
}
crates/engine/src/edit/run.rs
+46 -1
View File
@@ -165,6 +165,47 @@ pub(crate) fn check_permission(edit: &Edit, p: &pdfcraft_cos::Permissions) -> Re
}
}
/// What a comment import left out of comments it did import (an appearance it could not read),
/// in the repair log as one line: said, not slipped in. The first five are named, each in at
/// most [`SKIPPED_CHARS`] characters with control characters escaped (a name from the file
/// holding a line break would otherwise read as a repair of its own).
fn note_skipped(doc: &mut pdfcraft_cos::Document, report: &pdfcraft_xfdf::Report) {
if report.skipped.is_empty() {
return;
}
let shown: Vec<String> = report.skipped.iter().take(5).map(|s| skipped_text(s)).collect();
let more = report.skipped.len().saturating_sub(shown.len());
let more = if more > 0 { format!("; and {more} more") } else { String::new() };
doc.note_repair(format!(
"imported {} comment{} without the appearance the file gave {} (PdfCraft draws it where it can; it may look different or not show): {}{more}",
report.skipped.len(),
if report.skipped.len() == 1 { "" } else { "s" },
if report.skipped.len() == 1 { "it" } else { "them" },
shown.join("; ")
));
}
/// At most this many characters of one skipped comment's description in the repair log.
const SKIPPED_CHARS: usize = 200;
/// `s` for one line of the repair log: control characters and line separators escaped, cut to
/// [`SKIPPED_CHARS`] characters.
fn skipped_text(s: &str) -> String {
let mut out = String::new();
for (n, c) in s.chars().enumerate() {
if n == SKIPPED_CHARS {
out.push('…');
break;
}
if c.is_control() || matches!(c, '\u{2028}' | '\u{2029}') {
out.extend(c.escape_default());
} else {
out.push(c);
}
}
out
}
/// Dates and unique ids stamped onto what an edit creates.
pub(crate) struct EditCtx {
/// Field scripts run (`None`: JavaScript is off) and what they produced.
@@ -514,6 +555,7 @@ pub(crate) fn run_edit(doc: &mut pdfcraft_cos::Document, edit: &Edit, cx: &mut E
if report.comments == 0 {
return Err(EditError::Invalid("the clipboard holds no comments".into()));
}
note_skipped(doc, &report);
}
Edit::AddCustomStamp { page, rect, name, file, author } => {
let src = mark_source(doc, file)?;
@@ -874,7 +916,10 @@ pub(crate) fn run_edit(doc: &mut pdfcraft_cos::Document, edit: &Edit, cx: &mut E
// the data importer's error.
let mut data = doc.clone();
match pdfcraft_xfdf::import(&mut data, bytes) {
Ok(_) => *doc = data,
Ok(report) => {
note_skipped(&mut data, &report);
*doc = data;
}
Err(e) if bytes[..bytes.len().min(1024)].windows(5).any(|w| w == b"%PDF-") => match comment_source(bytes) {
Ok(src) => import_pdf_comments(doc, &src)?,
Err(SourceProblem::Unreadable(_)) => return Err(e.into()),
crates/engine/src/session/editing.rs
+7
View File
@@ -59,6 +59,7 @@ impl Session {
// in the paper's colour and draw the new text in the ink's.
let edit = match edit {
Edit::EditTextBlock { page, block, text, mut style } => {
let plain = style == pdfcraft_edit::BlockStyle::default();
// Only invisible text is sampled; the blocks are usually cached from opening it.
if style.cover.is_none()
&& let Some(info) = doc.info.pages.get(page)
@@ -67,6 +68,12 @@ impl Session {
{
style.cover = Some(paper);
style.color = style.color.or(Some(ink));
// A text edit alone leaves the words it doesn't change as the scan shows them.
style.changed_words_only = plain;
// Nothing to redraw: leave undo history and the dirty flag alone.
if plain && b.same_words(&text) {
return Ok(None);
}
}
Edit::EditTextBlock { page, block, text, style }
}
crates/engine/src/session/files.rs
+2 -1
View File
@@ -305,7 +305,8 @@ impl Session {
/// [`Self::combine_unlocked`] where sources with the same `groups` key (by position) are one
/// file split into several runs placed apart: each file is opened and copied once, so it keeps
/// one bookmark, its links between its own pages, its fields and its attachments whole (see
/// one bookmark, and its links between its own pages and its fields whole across its runs; its
/// attachments come along when its runs use all of its pages (see
/// `pdfcraft_organize::combine_grouped`). A group's later sources use its first source's bytes
/// and password; a source without a key is a file of its own.
pub fn combine_grouped(&self, sources: &[CombineSource], groups: &[u64], passwords: &[Option<&str>]) -> Result<Arc<Vec<u8>>, EditError> {
crates/engine/src/tests.rs
+65
View File
@@ -11870,6 +11870,71 @@ fn imported_bookmarks_undo_in_one_step() {
assert!(s.get(id).unwrap().info.outline.is_empty());
}
/// A comment imported or pasted without the appearance its file carried (unreadable there) is
/// said in the repair log, which the interface shows and the tools return.
#[test]
fn a_comment_imported_without_its_appearance_is_said_in_the_repair_log() {
let xfdf = r#"<?xml version="1.0"?><xfdf xmlns="http://ns.adobe.com/xfdf/"><annots><stamp page="0" name="st" rect="10,10,110,60" icon="Draft"><appearance>@@@not base 64@@@</appearance></stamp></annots></xfdf>"#;
let said = |s: &Session, id: DocId| {
s.get(id)
.unwrap()
.repair_log()
.iter()
.filter(|l| l.contains("imported 1 comment without the appearance") && l.contains("st: the appearance is not base 64"))
.count()
};
let (mut s, id) = session_with(1);
s.apply(id, Edit::ImportData { name: "notes.xfdf".into(), bytes: Arc::new(xfdf.as_bytes().to_vec()) }).unwrap();
assert_eq!(said(&s, id), 1, "{:?}", s.get(id).unwrap().repair_log());
let (mut s, id) = session_with(1);
s.apply(id, Edit::PasteComments { xfdf: xfdf.into(), page: 0, offset: None }).unwrap();
assert_eq!(said(&s, id), 1, "{:?}", s.get(id).unwrap().repair_log());
}
/// Six comments without their appearances: one line naming five and counting the sixth, a name
/// holding a line break stays on that line, and a long name is cut short.
#[test]
fn many_comments_imported_without_their_appearances_are_summed_up_in_one_line() {
let long = "L".repeat(1000);
let names = ["a&#10;imported 9 comments", "b", "c", "d", long.as_str(), "f"];
let stamps: String = names
.iter()
.map(|nm| format!(r#"<stamp page="0" name="{nm}" rect="10,10,110,60" icon="Draft"><appearance>@@@</appearance></stamp>"#))
.collect();
let xfdf = format!(r#"<?xml version="1.0"?><xfdf xmlns="http://ns.adobe.com/xfdf/"><annots>{stamps}</annots></xfdf>"#);
let (mut s, id) = session_with(1);
let before = s.get(id).unwrap().repair_log().len();
s.apply(id, Edit::PasteComments { xfdf, page: 0, offset: None }).unwrap();
let log = s.get(id).unwrap().repair_log();
let added = log.get(before..).unwrap_or_default();
assert_eq!(added.len(), 1, "{added:?}");
let line = &added[0];
assert!(line.starts_with("imported 6 comments without the appearance"), "{line}");
assert!(line.ends_with("; and 1 more"), "{line}");
assert!(!line.contains('\n') && line.contains(r"a\nimported 9 comments: "), "{line}");
assert!(line.contains(&format!("{}…", "L".repeat(200))) && !line.contains(&"L".repeat(201)), "{line}");
assert_eq!(line.matches(": the appearance").count(), 4, "five named, the long one cut before its reason: {line}");
}
/// PDF/A-2 6.1.2 through the ordinary save: Save as PDF/A on a file without a binary comment, or
/// with a 2.0 header, saves a file whose header verifies.
#[test]
fn save_as_pdfa_saves_a_compliant_file_header() {
let plain = fixture(1);
assert!(plain.starts_with(b"%PDF-1.7\n1 0 obj"), "the fixture has no binary comment");
let mut newer = plain.clone();
newer.splice(5..8, *b"2.0");
for bytes in [plain, newer] {
let mut s = Session::new();
let id = s.open("x.pdf", None, Arc::new(bytes), None).unwrap();
s.apply(id, Edit::ConvertPdfA { level: pdfcraft_preflight::Level::A2b }).unwrap();
let saved = s.save_bytes(id).unwrap();
assert!(saved.starts_with(b"%PDF-1.7\n%\xE2\xE3\xCF\xD3\n"), "{:?}", String::from_utf8_lossy(&saved[..20]));
let reopened = pdfcraft_cos::Document::open(saved).unwrap();
assert!(pdfcraft_preflight::verify(&reopened, pdfcraft_preflight::Level::A2b).iter().all(|i| i.clause != "6.1.2"));
}
}
/// ISO 32000-2 §7.6.6, Table 25: a document whose only encryption is for its embedded files,
/// authorised when one is opened (`/AuthEvent /EFOpen`), opens without a password, lists its
/// attachment, and saves with the attachment still encrypted.
crates/engine/tests/it/ocr_edit.rs
+310 -17
View File
@@ -128,7 +128,7 @@ fn ocr_words_group_into_phrases_that_edit_in_place() {
assert!((blocks[1].rect[0] - john[0]).abs() < 1.0, "the new word starts where the old one did: {:?} vs {john:?}", blocks[1].rect);
}
/// Several words of one OCR phrase rewrite as one line, without wrapping onto the line below.
/// Words added to an OCR phrase grow rightwards as one line, without wrapping onto the line below.
#[test]
fn a_longer_ocr_phrase_grows_rightwards() {
let (mut s, id) = scanned_with(vec![
@@ -138,8 +138,8 @@ fn a_longer_ocr_phrase_grows_rightwards() {
]);
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "Project Coordinator Lead".into(), style: Default::default() }).unwrap();
let blocks = s.get(id).unwrap().text_blocks(0);
assert_eq!(blocks.iter().map(|b| b.text.as_str()).collect::<Vec<_>>(), ["Project Coordinator Lead", "Testing"]);
assert_eq!(blocks[0].rows.len(), 1, "one line");
assert_eq!(blocks.iter().map(|b| b.text.as_str()).collect::<Vec<_>>(), ["Project", "Coordinator Lead", "Testing"]);
assert_eq!(blocks[1].rows.len(), 1, "one line");
}
/// OCR words differ in size and are stretched to fit the scan: a rewritten phrase keeps each
@@ -151,12 +151,15 @@ fn a_rewritten_ocr_phrase_keeps_its_word_widths() {
scanned_word("Smith", 49.0, 78.0, 70.0, 11.0),
scanned_word("Example", 82.0, 124.0, 70.0, 11.5),
]);
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "John Smith Exam".into(), style: Default::default() }).unwrap();
// Every word changes, so every word is redrawn. "Jahn" is as wide as "John" in Helvetica.
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "Jahn Smyth Exam".into(), style: Default::default() }).unwrap();
let lines = s.get(id).unwrap().text_lines(0);
assert_eq!(lines.iter().map(|l| l.text.trim()).collect::<Vec<_>>(), ["John", "Smith", "Exam"]);
assert!((lines[0].rect[2] - 45.0).abs() < 0.5, "John keeps its width: {:?}", lines[0].rect);
for pair in lines.windows(2) {
assert!(pair[1].rect[0] >= pair[0].rect[2] - 0.5, "{:?} runs into {:?}", pair[1].text, pair[0].text);
assert_eq!(lines.iter().map(|l| l.text.trim()).collect::<Vec<_>>(), ["Jahn Smyth Exam"]);
let ink = &lines[0].ink;
assert!(ink.len() == 3, "three words: {ink:?}");
assert!((ink[0][2] - 45.0).abs() < 0.5, "Jahn keeps John's width: {ink:?}");
for pair in ink.windows(2) {
assert!(pair[1][0] >= pair[0][2] - 0.5, "{pair:?} run into each other");
}
}
@@ -179,16 +182,15 @@ fn an_edited_ocr_phrase_is_drawn_at_one_size() {
let block = s.get(id).unwrap().text_blocks(0).remove(0);
assert!((block.size - 11.0).abs() < 0.05, "the phrase's size is its words' median: {}", block.size);
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "John Example".into(), style: Default::default() }).unwrap();
// Every word changes, so every word is redrawn. "Jahn" is as wide as "John" in Helvetica.
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "Jahn Smyth Exbmple".into(), style: Default::default() }).unwrap();
let lines = s.get(id).unwrap().text_lines(0);
assert_eq!(lines.iter().map(|l| l.text.trim()).collect::<Vec<_>>(), ["John", "Example"]);
assert_eq!(lines.iter().map(|l| l.text.as_str()).collect::<String>().replace(' ', ""), "JahnSmythExbmple");
for line in &lines {
assert!((line.size - 11.0).abs() < 0.05, "{:?} is drawn at {} pt, not the phrase's 11", line.text, line.size);
}
// Each word keeps its width; the rest of the phrase closes up where Smith was.
assert!((lines[0].rect[2] - lines[0].rect[0] - 25.0).abs() < 0.5, "John keeps its width: {:?}", lines[0].rect);
assert!((lines[1].rect[2] - lines[1].rect[0] - 42.0).abs() < 0.5, "Example keeps its width: {:?}", lines[1].rect);
assert!(lines[1].rect[0] >= lines[0].rect[2] - 0.5, "Example runs into John");
let ink = &lines[0].ink;
assert!((ink[0][2] - ink[0][0] - 25.0).abs() < 0.5, "Jahn keeps John's width: {ink:?}");
}
/// Only invisible OCR text is set in one phrase size: visible text keeps each run's own size, and
@@ -243,10 +245,12 @@ fn a_word_far_larger_than_its_phrase_keeps_its_width() {
scanned_word("Smith", 49.0, 78.0, 70.0, 10.0),
scanned_word("Example", 82.0, 124.0, 70.0, 10.5),
]);
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "John Smith Exam".into(), style: Default::default() }).unwrap();
// Every word changes, so every word is redrawn. "Jahn" is as wide as "John" in Helvetica.
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "Jahn Smyth Exam".into(), style: Default::default() }).unwrap();
let lines = s.get(id).unwrap().text_lines(0);
assert_eq!(lines[0].text.trim(), "John");
assert!((lines[0].rect[2] - lines[0].rect[0] - 25.0).abs() < 0.5, "John keeps its width: {:?}", lines[0].rect);
let ink = &lines[0].ink;
assert!(ink.len() == 3, "three words: {ink:?}");
assert!((ink[0][2] - ink[0][0] - 25.0).abs() < 0.5, "Jahn keeps John's width: {ink:?}");
}
/// With a new font the phrase is one run in its first word's text state. How wide it is must not
@@ -270,6 +274,295 @@ fn a_new_font_for_an_ocr_phrase_is_as_wide_whatever_its_first_word_was_sized() {
assert!((guessed - right).abs() < 0.5, "a 15 pt guess for John makes the line {guessed} wide, an 11 pt one {right}");
}
/// The lines drawn invisibly (still the scan's) and visibly (redrawn), by their text.
fn scan_and_drawn(s: &Session, id: pdfcraft_engine::DocId) -> (Vec<String>, Vec<String>) {
let blocks = s.get(id).unwrap().text_blocks(0);
let of = |invisible: bool| blocks.iter().filter(|b| b.invisible == invisible).map(|b| b.text.clone()).collect::<Vec<_>>();
(of(true), of(false))
}
/// A phrase over the scan's block of ink ([`WORD`]): two words side by side.
fn two_words_on_the_ink() -> (Session, pdfcraft_engine::DocId) {
scanned_with(vec![scanned_word("John", 42.0, 66.0, 42.0, 18.0), scanned_word("Smith", 70.0, 98.0, 42.0, 18.0)])
}
/// The share of paper among the page's pixels from x0 to x1 inside the scanned word's height.
fn paper_between(s: &Session, id: pdfcraft_engine::DocId, x0: u32, x1: u32) -> f64 {
let doc = s.get(id).unwrap();
let out = PageRenderer::new(doc.bytes.clone(), Default::default()).render(RenderRequest { page: 0, scale: 1.0, ..Default::default() });
assert!(out.error.is_none(), "{:?}", out.error);
let (mut paper, mut all) = (0, 0);
for y in WORD[1] + 4..WORD[3] - 4 {
for x in x0..x1 {
let i = ((y * out.width + x) * 4) as usize;
paper += usize::from(near([out.rgba[i], out.rgba[i + 1], out.rgba[i + 2]], PAPER));
all += 1;
}
}
paper as f64 / all as f64
}
/// Deleting a word covers that word and nothing else: the rest of the line is still the scan, and
/// the words after it stay where the scan shows them (#3313).
#[test]
fn deleting_an_ocr_word_leaves_the_rest_of_the_line_as_the_scan() {
let (mut s, id) = uneven_phrase();
let before = s.get(id).unwrap().text_lines(0);
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "John Example".into(), style: Default::default() }).unwrap();
assert_eq!(scan_and_drawn(&s, id), (vec!["John".to_owned(), "Example".to_owned()], Vec::new()), "nothing is redrawn");
let after = s.get(id).unwrap().text_lines(0);
assert_eq!(after.len(), 2);
assert_eq!(after[0].rect, before[0].rect, "John hasn't moved");
assert_eq!(after[1].rect, before[2].rect, "Example hasn't closed up the gap");
assert_eq!(after[0].size, before[0].size, "and kept words keep the layer's own sizes");
}
/// On the page: the deleted word's pixels are covered, its neighbour's aren't.
#[test]
fn a_deleted_ocr_word_is_covered_and_its_neighbour_is_not() {
let (mut s, id) = two_words_on_the_ink();
assert!(paper_between(&s, id, 44, 64) < 0.02 && paper_between(&s, id, 72, 96) < 0.02, "both words are ink to begin with");
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "John".into(), style: Default::default() }).unwrap();
assert!(paper_between(&s, id, 44, 64) < 0.02, "John is still the scan");
assert!(paper_between(&s, id, 72, 96) > 0.98, "Smith is covered");
assert_eq!(s.get(id).unwrap().text_blocks(0).iter().map(|b| b.text.as_str()).collect::<Vec<_>>(), ["John"], "and gone from the text");
}
/// Replacing a word redraws that word, at the phrase's size and in its place.
#[test]
fn replacing_an_ocr_word_redraws_only_that_word() {
let (mut s, id) = uneven_phrase();
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "John Smyth Example".into(), style: Default::default() }).unwrap();
assert_eq!(scan_and_drawn(&s, id), (vec!["John".to_owned(), "Example".to_owned()], vec!["Smyth".to_owned()]));
let lines = s.get(id).unwrap().text_lines(0);
let smyth = lines.iter().find(|l| l.text == "Smyth").unwrap();
assert!((smyth.size - 11.0).abs() < 0.05, "at the phrase's size: {}", smyth.size);
assert!((smyth.rect[0] - 49.0).abs() < 0.5, "where Smith was: {:?}", smyth.rect);
}
/// Changes apart from each other are each their own, with the word between them kept, as one step.
#[test]
fn two_changed_ocr_words_keep_the_word_between_them() {
let (mut s, id) = uneven_phrase();
let original = s.get(id).unwrap().bytes.clone();
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "Jon Smith Exam".into(), style: Default::default() }).unwrap();
assert_eq!(scan_and_drawn(&s, id), (vec!["Smith".to_owned()], vec!["Jon".to_owned(), "Exam".to_owned()]));
s.undo(id).unwrap();
assert_eq!(s.get(id).unwrap().bytes, original, "one undo brings the phrase back");
s.redo(id).unwrap();
assert_eq!(scan_and_drawn(&s, id), (vec!["Smith".to_owned()], vec!["Jon".to_owned(), "Exam".to_owned()]));
}
/// A replacement a little wider than its word is squeezed into its place, and the next word stays.
#[test]
fn a_slightly_longer_ocr_word_is_squeezed_into_its_place() {
let (mut s, id) = uneven_phrase();
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "John Smithy Example".into(), style: Default::default() }).unwrap();
assert_eq!(scan_and_drawn(&s, id), (vec!["John".to_owned(), "Example".to_owned()], vec!["Smithy".to_owned()]));
let lines = s.get(id).unwrap().text_lines(0);
let smithy = lines.iter().find(|l| l.text == "Smithy").unwrap();
let example = lines.iter().find(|l| l.text == "Example").unwrap();
assert!(smithy.rect[2] <= 80.5, "it ends before the middle of the gap: {:?}", smithy.rect);
assert_eq!(example.rect[0].round(), 82.0, "and Example hasn't moved");
assert!((smithy.size - 11.0).abs() < 0.05, "squeezed across, not smaller: {}", smithy.size);
}
/// A replacement too wide to squeeze takes over the word it would run into, and no more.
#[test]
fn a_longer_ocr_word_takes_over_the_word_it_would_run_into() {
let (mut s, id) = scanned_with(vec![
scanned_word("John", 20.0, 45.0, 70.0, 11.0),
scanned_word("Smith", 49.0, 78.0, 70.0, 11.5),
scanned_word("Jones", 82.0, 112.0, 70.0, 12.0),
scanned_word("Brown", 118.5, 150.0, 70.0, 12.5),
]);
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "John Smithsons Jones Brown".into(), style: Default::default() }).unwrap();
let (scan, drawn) = scan_and_drawn(&s, id);
assert_eq!(scan, ["John", "Brown"], "the words it doesn't reach stay: {drawn:?}");
assert_eq!(drawn, ["Smithsons Jones"]);
let lines = s.get(id).unwrap().text_lines(0);
let brown = lines.iter().find(|l| l.text == "Brown").unwrap();
let jones = lines.iter().find(|l| l.text.ends_with("Jones")).unwrap();
assert!(jones.rect[2] <= brown.rect[0], "{:?} runs into {:?}", jones.rect, brown.rect);
}
/// A word added with nothing to replace redraws a neighbour; the rest stay.
#[test]
fn an_added_ocr_word_redraws_only_a_neighbour() {
let (mut s, id) = uneven_phrase();
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "John Smith Example again".into(), style: Default::default() }).unwrap();
assert_eq!(scan_and_drawn(&s, id), (vec!["John Smith".to_owned()], vec!["Example again".to_owned()]));
}
/// Words OCR sized alike share a run in the text lines; they are still kept one by one.
#[test]
fn ocr_words_of_one_size_are_kept_one_by_one() {
let (mut s, id) = scanned_with(vec![
scanned_word("JOHN", 20.0, 45.0, 70.0, 11.0),
scanned_word("SMITH", 49.0, 78.0, 70.0, 11.0),
scanned_word("JONES", 82.0, 112.0, 70.0, 11.0),
]);
assert_eq!(s.get(id).unwrap().text_lines(0).len(), 1, "one run to begin with");
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "JOHN SMYTH JONES".into(), style: Default::default() }).unwrap();
let (scan, drawn) = scan_and_drawn(&s, id);
assert_eq!((scan, drawn), (vec!["JOHN".to_owned(), "JONES".to_owned()], vec!["SMYTH".to_owned()]));
}
/// Changing the formatting is about the whole phrase, so all of it is redrawn.
#[test]
fn a_formatting_change_redraws_the_whole_ocr_phrase() {
let (mut s, id) = uneven_phrase();
let style = pdfcraft_engine::BlockStyle { color: Some([1.0, 0.0, 0.0]), ..Default::default() };
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "John Smyth Example".into(), style }).unwrap();
let (scan, drawn) = scan_and_drawn(&s, id);
assert!(scan.is_empty(), "nothing is left as the scan: {scan:?}");
assert_eq!(drawn.join(" "), "John Smyth Example");
}
/// A layer whose words follow one another with no position of their own: a word that went would
/// move those after it, so keeping them isn't possible and the whole phrase is redrawn instead.
#[test]
fn words_that_depend_on_the_one_before_are_redrawn_rather_than_moved() {
let bytes = under_scan("BT /F1 10 Tf 3 Tr 20 44 Td (John) Tj ( ) Tj (Smith) Tj ( ) Tj (Example) Tj ET");
let mut s = Session::new();
let id = s.open("relative.pdf", None, bytes, None).unwrap();
assert_eq!(texts(&s, id), ["John Smith Example"]);
edit_block(&mut s, id, 0, "John Example");
let (scan, drawn) = scan_and_drawn(&s, id);
assert!(scan.is_empty(), "no word is left to shift under the scan: {scan:?}");
assert_eq!(drawn, ["John Example"]);
}
/// The edit survives a save and reopen, and each piece can be edited again.
#[test]
fn a_kept_ocr_word_can_be_edited_after_a_save() {
let (mut s, id) = uneven_phrase();
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "John Smyth Example".into(), style: Default::default() }).unwrap();
let saved = s.get(id).unwrap().bytes.clone();
let id = s.open("saved.pdf", None, saved, None).unwrap();
assert_eq!(scan_and_drawn(&s, id), (vec!["John".to_owned(), "Example".to_owned()], vec!["Smyth".to_owned()]));
let john = s.get(id).unwrap().text_blocks(0).iter().position(|b| b.text == "John").unwrap();
s.apply(id, Edit::EditTextBlock { page: 0, block: john, text: "Jon".into(), style: Default::default() }).unwrap();
assert_eq!(scan_and_drawn(&s, id), (vec!["Example".to_owned()], vec!["Jon".to_owned(), "Smyth".to_owned()]));
}
/// A draft with the same words, a space typed after the last of them, changes nothing: not the
/// page, and not the undo history. It used to redraw the whole line.
#[test]
fn a_draft_with_the_same_words_changes_nothing() {
let (mut s, id) = uneven_phrase();
let (bytes, generation) = (s.get(id).unwrap().bytes.clone(), s.get(id).unwrap().edit_generation());
for draft in ["John Smith Example ", " John Smith Example"] {
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: draft.into(), style: Default::default() }).unwrap();
assert_eq!(s.get(id).unwrap().bytes, bytes, "{draft:?} wrote nothing");
assert_eq!(s.get(id).unwrap().edit_generation(), generation, "{draft:?} is no undo step");
assert_eq!(scan_and_drawn(&s, id).1, Vec::<String>::new(), "{draft:?} drew nothing");
}
}
/// Visible text positioned after the end of an OCR word depends on that word. An edit may not
/// leave the word gone and the visible text somewhere else while calling the rest kept.
#[test]
fn visible_text_that_follows_an_ocr_word_stops_it_being_kept_alone() {
let bytes = under_scan("BT /F1 10 Tf 3 Tr 20 44 Td (John) Tj ( ) Tj (Smith) Tj 0 Tr /F1 14 Tf ( Label) Tj ET");
let mut s = Session::new();
let id = s.open("label.pdf", None, bytes, None).unwrap();
let label = |s: &Session| s.get(id).unwrap().text_lines(0).iter().find(|l| l.text.contains("Label")).map(|l| l.rect[0]);
assert_eq!(scan_and_drawn(&s, id).0, ["John Smith"]);
let before = label(&s).unwrap();
edit_block(&mut s, id, 0, "John");
let (scan, _) = scan_and_drawn(&s, id);
assert!(scan.is_empty(), "John can't stay alone when Label would move with it: {scan:?}, Label at {:?} from {before}", label(&s));
}
/// A large word right before a small one, as OCR's sizes can make them: the large word's cover,
/// padded in proportion to its height, stops at the small word, or that word would lose ink while
/// staying in the text.
#[test]
fn a_cover_stops_at_the_word_beside_it() {
let (mut s, id) = scanned_with(vec![scanned_word("John", 42.0, 66.0, 42.0, 30.0), scanned_word("Smith", 67.2, 98.0, 42.0, 8.0)]);
assert_eq!(s.get(id).unwrap().text_blocks(0)[0].text, "John Smith");
assert!(paper_between(&s, id, 67, 70) < 0.02, "Smith's start is ink to begin with");
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "Smith".into(), style: Default::default() }).unwrap();
assert_eq!(scan_and_drawn(&s, id), (vec!["Smith".to_owned()], Vec::new()), "only John goes");
assert!(paper_between(&s, id, 44, 64) > 0.98, "John is covered");
// John's padding would reach to x = 68, over this column of Smith's ink.
let smith = paper_between(&s, id, 67, 68);
assert!(smith < 0.02, "Smith keeps its ink: {smith}");
}
/// Deleting a word with Backspace leaves the spaces either side of it. That isn't spacing anyone
/// asked for: the words around it stay where the scan has them, with the gap already between them.
#[test]
fn the_spaces_a_deleted_word_leaves_keep_the_scan() {
for (draft, scan, drawn) in [
("John Example", vec!["John", "Example"], vec![]),
(" John Example", vec!["John", "Example"], vec![]),
("John Smyth Example", vec!["John", "Example"], vec!["Smyth"]),
] {
let (mut s, id) = uneven_phrase();
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: draft.into(), style: Default::default() }).unwrap();
let (kept, redrawn) = scan_and_drawn(&s, id);
assert_eq!(
(kept.iter().map(String::as_str).collect::<Vec<_>>(), redrawn.iter().map(String::as_str).collect::<Vec<_>>()),
(scan, drawn),
"{draft:?}"
);
}
}
/// Spacing that has to be drawn goes to the rewrite that can draw it: inside new words, or typed
/// between words nothing else changed. It isn't quietly dropped.
#[test]
fn spacing_that_has_to_be_drawn_redraws_the_phrase() {
for draft in ["Jon Smyth Example", "John Smith Example"] {
let (mut s, id) = uneven_phrase();
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: draft.into(), style: Default::default() }).unwrap();
let (scan, _) = scan_and_drawn(&s, id);
assert!(scan.is_empty(), "{draft:?} keeps nothing as the scan: {scan:?}");
}
let (mut s, id) = uneven_phrase();
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "Jon Smyth Example".into(), style: Default::default() }).unwrap();
let (_, drawn) = scan_and_drawn(&s, id);
assert!(drawn.join("").contains("Jon Smy"), "the typed spaces are there: {drawn:?}");
}
/// A word drawn twice, its copy earlier in the layer: rewriting the word takes the copy with it, and
/// the words after the copy move up the list. The next change must not land on the wrong word.
#[test]
fn a_change_lands_on_its_own_word_when_a_rewrite_removes_an_earlier_copy() {
let bytes = under_scan(
"BT /F1 12 Tf 3 Tr 1 0 0 1 84 44 Tm (Jones) Tj 1 0 0 1 20 44 Tm (John) Tj 1 0 0 1 50 44 Tm (Smith) Tj 1 0 0 1 84 44 Tm (Jones) Tj ET",
);
let mut s = Session::new();
let id = s.open("copy.pdf", None, bytes, None).unwrap();
let block = texts(&s, id).iter().position(|t| t == "John Smith Jones").expect("the phrase, after its copy");
// John goes. A deletion draws nothing, so nothing else would notice it landing on Smith.
edit_block(&mut s, id, block, "Smith Joness");
let text = s.get(id).unwrap().text_lines(0).iter().map(|l| l.text.as_str()).collect::<Vec<_>>().join(" ");
let words: Vec<&str> = text.split_whitespace().collect();
assert!(words.contains(&"Smith"), "Smith wasn't changed: {words:?}");
assert!(!words.contains(&"John"), "John is the word that goes: {words:?}");
assert!(words.contains(&"Joness"), "{words:?}");
}
/// A layer whose words are already narrowed (50 Tz): squeezing a replacement narrows it further,
/// rather than setting a scale wider than the one it had and running it into the next word.
#[test]
fn a_squeeze_is_relative_to_the_words_own_scale() {
let bytes = under_scan("BT /F1 12 Tf 3 Tr 50 Tz 1 0 0 1 20 44 Tm (John) Tj 1 0 0 1 36 44 Tm (Smith) Tj 1 0 0 1 56 44 Tm (Jones) Tj ET");
let mut s = Session::new();
let id = s.open("narrow.pdf", None, bytes, None).unwrap();
assert_eq!(scan_and_drawn(&s, id).0, ["John Smith Jones"]);
edit_block(&mut s, id, 0, "John Smithy Jones");
let (scan, drawn) = scan_and_drawn(&s, id);
assert_eq!(drawn, ["Smithy"], "Jones isn't taken over; the scan has {scan:?}");
let lines = s.get(id).unwrap().text_lines(0);
let smithy = lines.iter().find(|l| l.text == "Smithy").unwrap();
let jones = lines.iter().find(|l| l.text == "Jones").unwrap();
assert!(smithy.rect[2] <= jones.rect[0], "{:?} runs into {:?}", smithy.rect, jones.rect);
}
/// Visible text set with a horizontally stretched text matrix (3 × 1), as some generators write.
fn stretched_visible() -> std::sync::Arc<Vec<u8>> {
use pdfcraft_cos::{Dict, Document, Object, SaveOptions, Stream};
crates/forms/src/appearances.rs
+25 -11
View File
@@ -10,12 +10,12 @@ pub fn need_appearances(doc: &Document) -> bool {
/// A copy of `doc` in which form fields are drawn the way the file leaves to the viewer, for
/// displaying the document (`None` when no widget needs it). Text and choice fields are drawn from
/// their values: every one when the form sets `/NeedAppearances` (ISO 32000-2 Table 224: the
/// writer left the appearances to the viewer), and otherwise each widget without a normal
/// appearance (`/AP /N`), as typing the value would draw it (#3470). Check boxes and radio buttons
/// with no normal appearance for their state (`/AS`) are drawn from their `/MK` background, border
/// and caption, and with `/NeedAppearances` every one that has an `/MK` is, over its own
/// appearance; push buttons with none are drawn from their `/MK` and `/BS` (Table 192). `doc` is
/// not changed: saving writes the file as it was.
/// writer left the appearances to the viewer), and otherwise each widget whose normal appearance
/// (`/AP /N`) gives no stream to draw, as typing the value would draw it (#3470). Check boxes and
/// radio buttons with no normal appearance stream for their state (`/AS`; an Off state may have
/// none) are drawn from their `/MK` background, border and caption, and with `/NeedAppearances`
/// every one that has an `/MK` is, over its own appearance; push buttons with none are drawn from
/// their `/MK` and `/BS` (Table 192). `doc` is not changed: saving writes the file as it was.
pub fn with_field_appearances(doc: &Document) -> Option<Document> {
with_field_appearances_cached(doc, &mut FieldAppearanceCache::default())
}
@@ -48,13 +48,14 @@ pub fn with_field_appearances_cached(doc: &Document, cache: &mut FieldAppearance
// drawn from it (Table 192), its own appearance or not; one without /MK keeps its own.
let described = || widget.as_dict().and_then(|d| d.get(b"MK")).is_some_and(|mk| doc.resolve(mk).as_dict().is_some());
let missing = match f.kind {
FieldKind::Text | FieldKind::Combo | FieldKind::List => normal.is_none() || all,
FieldKind::PushButton => normal.is_none(),
FieldKind::Text | FieldKind::Combo | FieldKind::List => all || !shows_stream(doc, normal.as_deref(), w.state.as_deref()),
FieldKind::PushButton => !shows_stream(doc, normal.as_deref(), w.state.as_deref()),
FieldKind::CheckBox | FieldKind::Radio => match (normal.as_deref(), state.as_deref()) {
_ if all && described() => true,
(None, _) => true,
(Some(Object::Dict(states)), Some(s)) => s != "Off" && !states.contains(&name_bytes(s)),
_ => false,
(Some(Object::Stream(_)), _) => false,
// An Off state may go without an appearance; an on state needs a stream.
(Some(Object::Dict(_)), Some("Off")) => false,
(n, s) => !shows_stream(doc, n, s),
},
FieldKind::Signature => false,
};
@@ -104,6 +105,19 @@ pub fn with_field_appearances_cached(doc: &Document, cache: &mut FieldAppearance
copy
}
/// Whether a widget's normal appearance `normal` (`/AP /N`, resolved) gives a stream to draw: the
/// entry itself when it is a stream, or the stream it names for the appearance state `state`
/// (`/AS`) when it is a subdictionary of states (ISO 32000-2 12.5.5, Table 170), as the renderer
/// picks it. Anything else (no entry, null, a subdictionary without `/AS` or without a stream for
/// that state, a dictionary that only looks like a form XObject) draws nothing.
fn shows_stream(doc: &Document, normal: Option<&Object>, state: Option<&str>) -> bool {
match (normal, state) {
(Some(Object::Stream(_)), _) => true,
(Some(Object::Dict(states)), Some(s)) => states.get(&name_bytes(s)).is_some_and(|a| matches!(doc.resolve(a).as_ref(), Object::Stream(_))),
_ => false,
}
}
/// The objects a widget's drawn appearance is built from, as the document holds them: the widget
/// and the fields above it (value, `/DA`, `/Q`, flags, options, `/MK`, formatting actions), the
/// form dictionary (`/DA`) and the fonts of its `/DR`, each with the objects its entries refer to
crates/forms/src/read.rs
+79 -6
View File
@@ -93,7 +93,8 @@ fn qualified_name(doc: &Document, r: ObjRef) -> Option<String> {
if seen.len() > 64 || !seen.insert(c) {
break;
}
let obj = doc.get(c);
// An object that is only a reference to another is followed, as `Document::resolve` does.
let obj = doc.resolve(&Object::Ref(c));
let d = obj.as_dict()?;
if let Some(t) = d.get(b"T").and_then(|o| text_of(&doc.resolve(o))) {
parts.push(t);
@@ -111,6 +112,38 @@ fn parent_of(doc: &Document, d: &Dict) -> Option<ObjRef> {
d.reference(b"Parent").filter(|p| doc.get(*p).as_dict().is_some())
}
/// The fields of the form's tree whose `/Kids` list one of `kids` and whose own fully qualified
/// name is `name`: a field listing kids that don't name it as their `/Parent`, and so share its
/// name rather than extend it. Each node is visited once, at most 64 levels down.
pub(crate) fn listing_fields_named(doc: &Document, name: &str, kids: &[ObjRef]) -> Vec<ObjRef> {
let mut out = Vec::new();
let Some(af) = acroform(doc) else { return out };
let top = af.get(b"Fields").map(|f| doc.resolve(f)).and_then(|f| f.as_array().cloned()).unwrap_or_default();
let mut stack: Vec<(ObjRef, usize)> = top.iter().filter_map(Object::as_ref).map(|r| (r, 0)).collect();
let mut seen = std::collections::HashSet::new();
while let Some((r, depth)) = stack.pop() {
if depth > 64 || !seen.insert(r) {
continue;
}
let obj = doc.get(r);
let Some(d) = obj.as_dict() else { continue };
let list: Vec<ObjRef> = d
.get(b"Kids")
.map(|k| doc.resolve(k))
.and_then(|k| k.as_array().cloned())
.unwrap_or_default()
.iter()
.filter_map(Object::as_ref)
.collect();
if list.iter().any(|k| kids.contains(k)) && qualified_name(doc, r).as_deref() == Some(name) {
out.push(r);
}
let next = depth.saturating_add(1);
stack.extend(list.into_iter().filter(|k| doc.get(*k).as_dict().is_some_and(|kd| kd.contains(b"T"))).map(|k| (k, next)));
}
out
}
/// JavaScript streams one reading of the form (its fields, or its document scripts) may decode
/// in all, however many actions name the same stream.
pub(crate) const MAX_SCRIPT_BYTES: usize = 64 << 20;
@@ -373,6 +406,19 @@ pub(crate) fn enumerate(doc: &Document) -> (Vec<Field>, usize) {
values: DecodeBudget::new(MAX_VALUE_BYTES, 0),
};
for r in listed_roots {
// A `/Fields` entry that is a nameless kid of a field `/Fields` doesn't list is read
// through that field, which names it (ISO 32000-2 §12.7.4.2). A kid a field already read
// has claimed as its widget is not read again as a nameless field; one its parent's
// `/Kids` omits is still read on its own, as before.
let nameless_kid = doc.get(r).as_dict().is_some_and(|d| !d.contains(b"T") && d.contains(b"Parent"));
if nameless_kid {
if let Some(root) = unlisted_root(doc, r, &seen.fields) {
walk(doc, root, &base, &page_of, &mut seen, &mut out, 0);
}
if seen.widgets.contains(&r) {
continue;
}
}
walk(doc, r, &base, &page_of, &mut seen, &mut out, 0);
}
let listed = out.len();
@@ -520,15 +566,34 @@ fn walk(
let kids: Vec<ObjRef> =
d.get(b"Kids").map(|k| doc.resolve(k)).and_then(|k| k.as_array().cloned()).unwrap_or_default().iter().filter_map(|k| k.as_ref()).collect();
// Kids with /T are fields; kids without are this field's widgets.
let (field_kids, kids): (Vec<ObjRef>, Vec<ObjRef>) = kids.into_iter().partition(|k| doc.get(*k).as_dict().is_some_and(|kd| kd.contains(b"T")));
let (field_kids, mut kids): (Vec<ObjRef>, Vec<ObjRef>) =
kids.into_iter().partition(|k| doc.get(*k).as_dict().is_some_and(|kd| kd.contains(b"T")));
let field_kids: Vec<ObjRef> = field_kids.into_iter().filter(|k| !not_a_child(doc, seen, r, *k)).collect();
if !field_kids.is_empty() {
seen.path.push(r);
for k in field_kids {
walk(doc, k, &inh, page_of, seen, out, depth + 1);
// A fully qualified name is built up the field's own `/Parent` chain (ISO 32000-2
// §12.7.4.2). A kid that doesn't name this field as its `/Parent` is named from its
// own chain, as other readers name it; it still inherits this field's attributes.
// The `/Parent` as written: `qualified_name` follows an object that only refers on,
// and names a missing one as no parent.
let parent = doc.get(k).as_dict().and_then(|kd| kd.reference(b"Parent"));
if parent == Some(r) {
walk(doc, k, &inh, page_of, seen, out, depth + 1);
} else {
let mut own = inh.clone();
own.name = parent.and_then(|p| qualified_name(doc, p)).unwrap_or_default();
walk(doc, k, &own, page_of, seen, out, depth + 1);
}
}
seen.path.pop();
return;
// "A field's children in the hierarchy may also include widget annotations" (ISO 32000-2
// §12.7.4): widget annotations listed beside named kids are still this field's widgets,
// read as a field of this name next to the named kids.
kids.retain(|k| doc.get(*k).as_dict().is_some_and(|kd| kd.name(b"Subtype") == Some(b"Widget")));
if kids.is_empty() {
return;
}
}
// A field of no kind this reads is skipped before it claims widgets or scripts, which a later
// field listing them then reads.
@@ -545,8 +610,16 @@ fn walk(
};
// A child names its parent (ISO 32000-2 §12.7.4.1, Table 226): a page widget naming this field
// is one of its widgets even when its `/Kids` leave it out, as MuPDF and PDFium read it.
let widget_refs: Vec<ObjRef> =
if d.contains(b"Rect") { vec![r] } else { kids.into_iter().chain(seen.by_parent.get(&r).into_iter().flatten().copied()).collect() };
// A field's widgets are its widget kids. The field dictionary is its own widget only when it
// has none (no kid whose `/Subtype` is `/Widget`): field and widget are merged "when a field has only a single associated widget
// annotation" (ISO 32000-2 §12.7.4), so a merged dictionary that also lists widget kids is a
// field whose buttons are those kids, not one more (often zero-area) button.
let has_widget_kids = kids.iter().any(|k| doc.get(*k).as_dict().is_some_and(|kd| kd.name(b"Subtype") == Some(b"Widget")));
let widget_refs: Vec<ObjRef> = if d.contains(b"Rect") && !has_widget_kids {
vec![r]
} else {
kids.into_iter().chain(seen.by_parent.get(&r).into_iter().flatten().copied()).collect()
};
let widget_refs: Vec<ObjRef> = widget_refs.into_iter().filter(|w| seen.widgets.insert(*w)).collect();
let actions = actions_of(doc, d, &widget_refs, &mut seen.scripts);
let button = (inh.ft.as_deref() == Some(b"Btn") && inh.ff.unwrap_or(0) & flags::PUSH_BUTTON != 0)
crates/forms/src/tests.rs
+375
View File
@@ -2493,6 +2493,332 @@ fn fields_sharing_a_name_take_the_same_value() {
}
}
/// A radio group `Group1` whose two kids are fields of their own: each has `/T` (`kid_t`) and,
/// with `parent`, a `/Parent` naming the group.
fn radio_kids_with_names(kid_t: [&str; 2], parent: bool) -> Document {
let parent = if parent { " /Parent 4 0 R" } else { "" };
let kid = |t: &str, state: &str, x: u32| {
format!(
"<< /Type /Annot /Subtype /Widget /FT /Btn /Ff 49152 /T ({t}){parent} /V /Off /AS /Off /Rect [{x} 700 {} 715] /P 3 0 R /AP << /N << /Off 7 0 R /{state} 7 0 R >> >> >>",
x + 15
)
};
let objs: Vec<String> = vec![
"<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [4 0 R] >> >>".into(), // 1
"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 800] >>".into(), // 2
"<< /Type /Page /Parent 2 0 R /Annots [5 0 R 6 0 R] >>".into(), // 3
"<< /FT /Btn /Ff 49152 /T (Group1) /V /Choice2 /Kids [6 0 R 5 0 R] >>".into(), // 4
kid(kid_t[0], "Choice1", 50), // 5
kid(kid_t[1], "Choice2", 100), // 6
"<< /Length 0 >>\nstream\n\nendstream".into(), // 7
];
document(&objs)
}
/// A field's fully qualified name is built from its own `/Parent` chain (ISO 32000-2 §12.7.4.2,
/// Table 226): a kid with `/T` that a field's `/Kids` lists but that has no `/Parent` naming
/// that field is named on its own, not under the field that lists it.
#[test]
fn a_named_kid_without_a_parent_keeps_its_own_name() {
let names = |doc: &Document| fields(doc).iter().map(|f| (f.name.clone(), f.obj.num)).collect::<Vec<_>>();
let same = radio_kids_with_names(["Group1", "Group1"], false);
assert_eq!(names(&same), [("Group1".to_string(), 6), ("Group1".to_string(), 5)]);
let other = radio_kids_with_names(["A", "B"], false);
assert_eq!(names(&other), [("B".to_string(), 6), ("A".to_string(), 5)]);
// With /Parent the kids are the group's children, as before.
let nested = radio_kids_with_names(["A", "B"], true);
assert_eq!(names(&nested), [("Group1.B".to_string(), 6), ("Group1.A".to_string(), 5)]);
}
/// Fields with the same fully qualified name are one field (ISO 32000-2 §12.7.4.2): a radio
/// option that only a later one of them draws can be chosen by the name, and the others turn off.
#[test]
fn a_radio_option_of_any_field_sharing_the_name_can_be_chosen() {
for parent in [false, true] {
let name = if parent { "Group1.Group1" } else { "Group1" };
let mut doc = radio_kids_with_names(["Group1", "Group1"], parent);
for (choice, on, off) in [("Choice1", 5, 6), ("Choice2", 6, 5), ("Choice1", 5, 6)] {
set_value(&mut doc, name, &FieldValue::Radio(Some(choice.into()))).unwrap_or_else(|e| panic!("{name} = {choice}: {e}"));
doc = reopen(&doc);
let st = |n: u32| doc.get(ObjRef { num: n, generation: 0 }).as_dict().and_then(|d| d.name(b"AS").map(<[u8]>::to_vec));
assert_eq!((st(on), st(off)), (Some(choice.as_bytes().to_vec()), Some(b"Off".to_vec())), "{name} = {choice}");
}
let err = set_value(&mut doc, name, &FieldValue::Radio(Some("Choice3".into()))).unwrap_err();
assert!(err.to_string().contains("Choice3"), "{err}");
}
}
/// The same group as a broken save leaves it: the kids carry no `/P` (their page is found through
/// the page's `/Annots`), and both are drawn on with values that disagree (`/V /Off` with
/// `/AS /Choice1`, `/V /Choice2` with `/AS /Choice2`). Choosing by the name leaves exactly one
/// button on, across a save, whichever option it is, and gives every field of the name the
/// chosen value: both kids and the group listing them.
#[test]
fn a_radio_choice_by_the_name_turns_one_button_on_in_kids_without_a_page_that_disagree() {
let kid = |state: &str, v: &str, x: u32| {
format!(
"<< /Type /Annot /Subtype /Widget /FT /Btn /Ff 49152 /T (Group1) /V /{v} /AS /{state} /Rect [{x} 700 {} 715] /AP << /N << /Off 7 0 R /{state} 7 0 R >> >> >>",
x + 15
)
};
let objs: Vec<String> = vec![
"<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [4 0 R] >> >>".into(),
"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 800] >>".into(),
"<< /Type /Page /Parent 2 0 R /Annots [5 0 R 6 0 R] >>".into(),
"<< /FT /Btn /Ff 49152 /T (Group1) /V /Choice2 /Kids [6 0 R 5 0 R] >>".into(),
kid("Choice1", "Off", 50),
kid("Choice2", "Choice2", 100),
"<< /Length 0 >>\nstream\n\nendstream".into(),
];
let mut doc = document(&objs);
let st = |doc: &Document, n: u32| doc.get(ObjRef { num: n, generation: 0 }).as_dict().and_then(|d| d.name(b"AS").map(<[u8]>::to_vec));
let all = fields(&doc);
assert_eq!(all.iter().map(|f| f.name.as_str()).collect::<Vec<_>>(), ["Group1", "Group1"]);
assert!(all.iter().flat_map(|f| &f.widgets).all(|w| w.page == Some(0)), "found on the page without /P");
assert_eq!((st(&doc, 5), st(&doc, 6)), (Some(b"Choice1".to_vec()), Some(b"Choice2".to_vec())), "both drawn on as read");
for (choice, on, off) in [("Choice1", 5, 6), ("Choice2", 6, 5), ("Choice1", 5, 6)] {
set_value(&mut doc, "Group1", &FieldValue::Radio(Some(choice.into()))).unwrap_or_else(|e| panic!("Group1 = {choice}: {e}"));
doc = reopen(&doc);
assert_eq!((st(&doc, on), st(&doc, off)), (Some(choice.as_bytes().to_vec()), Some(b"Off".to_vec())), "Group1 = {choice}");
// Every field named Group1 has the same value (ISO 32000-2 §12.7.4.2): both kids and the
// field listing them, whose /V was a stale Choice2.
let v = |n: u32| doc.get(ObjRef { num: n, generation: 0 }).as_dict().and_then(|d| d.name(b"V").map(<[u8]>::to_vec));
assert_eq!(
[v(4), v(5), v(6)],
[Some(choice.as_bytes().to_vec()), Some(choice.as_bytes().to_vec()), Some(choice.as_bytes().to_vec())],
"Group1 = {choice}"
);
let values: Vec<Vec<String>> = fields(&doc).into_iter().map(|f| f.value).collect();
assert_eq!(values, [vec![choice.to_string()], vec![choice.to_string()]], "Group1 = {choice}");
}
}
/// A radio group whose field dictionary is merged with a zero-area widget and also lists four
/// widget kids, as some form exporters write it: field and widget may be merged only "when a field
/// has only a single associated widget annotation" (ISO 32000-2 §12.7.4), so the kids are the
/// buttons. Two such groups: each lists its four options, and choosing one by the name turns on
/// exactly that button, across a save, and leaves the other group alone.
#[test]
fn a_field_merged_with_a_widget_that_lists_widget_kids_has_the_kids_as_buttons() {
const OPTIONS: [&str; 4] = ["Not#20Satisfied", "Somewhat", "Satisfied", "Very#20Satisfied"];
let group = |t: &str, first: u32| {
let kids: Vec<String> = (first..first + 4).map(|n| format!("{n} 0 R")).collect();
format!("<< /Type /Annot /Subtype /Widget /Rect [0 0 0 0] /FT /Btn /Ff 49152 /Kids [{}] /V /Off /T ({t}) >>", kids.join(" "))
};
let kid = |parent: u32, i: usize, y: u32| {
let state = OPTIONS.get(i).copied().unwrap_or("X");
let x = 100 + 100 * u32::try_from(i).unwrap_or(0);
format!(
"<< /Type /Annot /Subtype /Widget /F 4 /FT /Btn /Ff 49152 /Parent {parent} 0 R /Rect [{x} {y} {} {}] /P 3 0 R /AS /Off /AP << /N << /{state} 14 0 R /Off 14 0 R >> >> >>",
x + 12,
y + 12
)
};
let mut objs: Vec<String> = vec![
"<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [4 0 R 5 0 R] >> >>".into(), // 1
"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 800] >>".into(), // 2
"<< /Type /Page /Parent 2 0 R /Annots [6 0 R 7 0 R 8 0 R 9 0 R 10 0 R 11 0 R 12 0 R 13 0 R] >>".into(), // 3
group("typeA15[0]", 6), // 4
group("typeA15[1]", 10), // 5
];
objs.extend((0..4).map(|i| kid(4, i, 600))); // 6-9
objs.extend((0..4).map(|i| kid(5, i, 560))); // 10-13
objs.push("<< /Length 0 >>\nstream\n\nendstream".into()); // 14
let mut doc = document(&objs);
let all = fields(&doc);
assert_eq!(all.iter().map(|f| f.name.as_str()).collect::<Vec<_>>(), ["typeA15[0]", "typeA15[1]"]);
let want: Vec<String> = ["Not Satisfied", "Somewhat", "Satisfied", "Very Satisfied"].map(String::from).to_vec();
for f in &all {
assert_eq!(f.widgets.iter().filter_map(|w| w.on_state.clone()).collect::<Vec<_>>(), want, "{}", f.name);
assert!(f.widgets.iter().all(|w| w.page == Some(0) && w.rect[2] > w.rect[0]), "{}: the kids, on the page", f.name);
}
let st = |doc: &Document, n: u32| doc.get(ObjRef { num: n, generation: 0 }).as_dict().and_then(|d| d.name(b"AS").map(<[u8]>::to_vec));
for (name, choice, on) in [("typeA15[0]", "Satisfied", 8), ("typeA15[1]", "Very Satisfied", 13), ("typeA15[0]", "Not Satisfied", 6)] {
set_value(&mut doc, name, &FieldValue::Radio(Some(choice.into()))).unwrap_or_else(|e| panic!("{name} = {choice}: {e}"));
doc = reopen(&doc);
let first = if name == "typeA15[0]" { 6 } else { 10 };
for n in first..first + 4 {
let want = if n == on { choice.as_bytes().to_vec() } else { b"Off".to_vec() };
assert_eq!(st(&doc, n), Some(want), "{name} = {choice}: widget {n}");
}
}
assert_eq!(st(&doc, 13), Some(b"Very Satisfied".to_vec()), "the other group keeps its choice");
let values: Vec<Vec<String>> = fields(&doc).into_iter().map(|f| f.value).collect();
assert_eq!(values, [vec!["Not Satisfied".to_string()], vec!["Very Satisfied".to_string()]]);
}
/// Radio buttons named `2` listed straight in `/Fields`, some fields of their own (`/T (2)`, no
/// `/Parent`) and some nameless widgets of an unlisted field `2` whose `/Kids` mix both: "A
/// field's children in the hierarchy may also include widget annotations" (ISO 32000-2 §12.7.4).
/// Every option is listed under the name `2`, and each, including the nameless widgets', can be
/// chosen by the name, leaving exactly that button on across a save.
#[test]
fn nameless_widgets_of_an_unlisted_field_are_options_of_its_name() {
let named = |state: &str, x: u32| {
format!(
"<< /Type /Annot /Subtype /Widget /FT /Btn /Ff 49152 /T (2) /V /Off /AS /Off /Rect [{x} 700 {} 712] /P 3 0 R /AP << /N << /{state} 9 0 R /Off 9 0 R >> >> >>",
x + 12
)
};
let nameless = |state: &str, x: u32| {
format!(
"<< /Type /Annot /Subtype /Widget /Parent 4 0 R /AS /Off /Rect [{x} 700 {} 712] /P 3 0 R /AP << /N << /{state} 9 0 R /Off 9 0 R >> >> >>",
x + 12
)
};
let objs: Vec<String> = vec![
"<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [5 0 R 6 0 R 7 0 R 8 0 R] >> >>".into(), // 1
"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 800] >>".into(), // 2
"<< /Type /Page /Parent 2 0 R /Annots [5 0 R 6 0 R 7 0 R 8 0 R] >>".into(), // 3
"<< /FT /Btn /Ff 49152 /T (2) /Kids [5 0 R 6 0 R 7 0 R 8 0 R] >>".into(), // 4 unlisted
named("0", 50), // 5
named("1", 100), // 6
nameless("2", 150), // 7
nameless("6", 200), // 8
"<< /Length 0 >>\nstream\n\nendstream".into(), // 9
];
let mut doc = document(&objs);
let all = fields(&doc);
assert!(all.iter().all(|f| f.name == "2"), "{:?}", all.iter().map(|f| &f.name).collect::<Vec<_>>());
let mut options: Vec<String> = all.iter().flat_map(|f| &f.widgets).filter_map(|w| w.on_state.clone()).collect();
options.sort();
assert_eq!(options, ["0", "1", "2", "6"]);
let st = |doc: &Document, n: u32| doc.get(ObjRef { num: n, generation: 0 }).as_dict().and_then(|d| d.name(b"AS").map(<[u8]>::to_vec));
for (choice, on) in [("6", 8), ("1", 6), ("2", 7), ("0", 5)] {
set_value(&mut doc, "2", &FieldValue::Radio(Some(choice.into()))).unwrap_or_else(|e| panic!("2 = {choice}: {e}"));
doc = reopen(&doc);
for n in 5..=8 {
let want = if n == on { choice.as_bytes().to_vec() } else { b"Off".to_vec() };
assert_eq!(st(&doc, n), Some(want), "2 = {choice}: widget {n}");
}
}
}
/// A kid with `/T` that a field's `/Kids` lists but whose `/Parent` names a different field is
/// named up that field's chain (ISO 32000-2 §12.7.4.2), not under the field listing it.
#[test]
fn a_named_kid_whose_parent_is_another_field_is_named_under_it() {
let objs: Vec<String> = vec![
"<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [4 0 R] >> >>".into(), // 1
"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 800] >>".into(), // 2
"<< /Type /Page /Parent 2 0 R /Annots [5 0 R] >>".into(), // 3
"<< /FT /Tx /T (Listing) /Kids [5 0 R] >>".into(), // 4
"<< /Type /Annot /Subtype /Widget /T (k) /Parent 6 0 R /Rect [50 700 150 715] /P 3 0 R >>".into(), // 5
"<< /T (Other) /Kids [5 0 R] >>".into(), // 6 unlisted
];
let names: Vec<String> = fields(&document(&objs)).into_iter().map(|f| f.name).collect();
assert_eq!(names, ["Other.k"]);
}
/// A `/Parent` naming an object that is only a reference to the parent field is followed to it,
/// as document inspection follows it, so both name the kid the same.
#[test]
fn a_parent_given_through_an_object_that_is_a_reference_names_the_kid() {
let objs: Vec<String> = vec![
"<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [4 0 R] >> >>".into(), // 1
"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 800] >>".into(), // 2
"<< /Type /Page /Parent 2 0 R /Annots [5 0 R] >>".into(), // 3
"<< /FT /Tx /T (Listing) /Kids [5 0 R] >>".into(), // 4
"<< /Type /Annot /Subtype /Widget /T (k) /Parent 6 0 R /Rect [50 700 150 715] /P 3 0 R >>".into(), // 5
"7 0 R".into(), // 6
"<< /T (Other) /Kids [5 0 R] >>".into(), // 7
];
let names: Vec<String> = fields(&document(&objs)).into_iter().map(|f| f.name).collect();
assert_eq!(names, ["Other.k"]);
}
/// The two kids of [`radio_kids_with_names`] (`Group1`, no `/Parent`), with the one drawing
/// `Choice2` (6, read first) or the one drawing `Choice1` (5) read-only.
fn radio_kids_one_read_only(read_only: u32) -> Document {
let kid = |n: u32, state: &str, x: u32| {
let ff = if n == read_only { 49153 } else { 49152 };
format!(
"<< /Type /Annot /Subtype /Widget /FT /Btn /Ff {ff} /T (Group1) /V /Off /AS /Off /Rect [{x} 700 {} 715] /P 3 0 R /AP << /N << /Off 7 0 R /{state} 7 0 R >> >> >>",
x + 15
)
};
let objs: Vec<String> = vec![
"<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [4 0 R] >> >>".into(),
"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 800] >>".into(),
"<< /Type /Page /Parent 2 0 R /Annots [5 0 R 6 0 R] >>".into(),
"<< /FT /Btn /Ff 49152 /T (Group1) /Kids [6 0 R 5 0 R] >>".into(),
kid(5, "Choice1", 50),
kid(6, "Choice2", 100),
"<< /Length 0 >>
stream
endstream"
.into(),
];
document(&objs)
}
/// A radio choice by the name reaches every radio field of that name (ISO 32000-2 §12.7.4.2), so
/// it is refused when any of them is read-only: the first one, even when a later one draws the
/// choice, or a later one, even when the first draws it. Nothing is written.
#[test]
fn a_radio_choice_by_the_name_is_refused_when_a_field_of_the_name_is_read_only() {
for (read_only, choice) in [(6, "Choice1"), (5, "Choice2")] {
let mut doc = radio_kids_one_read_only(read_only);
let state = |doc: &Document| {
[4, 5, 6].map(|n| {
doc.get(ObjRef { num: n, generation: 0 }).as_dict().map(|d| (d.name(b"V").map(<[u8]>::to_vec), d.name(b"AS").map(<[u8]>::to_vec)))
})
};
let before = state(&doc);
let err = set_value(&mut doc, "Group1", &FieldValue::Radio(Some(choice.into()))).unwrap_err();
assert!(matches!(err, FormError::ReadOnly(_)), "{read_only} read-only, {choice}: {err}");
assert_eq!(state(&doc), before, "{read_only} read-only, {choice}: nothing written");
}
}
/// A nameless widget listed in `/Fields` that the field before it already claimed is not read a
/// second time as a nameless field with no buttons.
#[test]
fn a_listed_widget_its_parent_claimed_is_read_once() {
let objs: Vec<String> = vec![
"<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [4 0 R 5 0 R] >> >>".into(), // 1
"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 800] >>".into(), // 2
"<< /Type /Page /Parent 2 0 R /Annots [5 0 R] >>".into(), // 3
"<< /FT /Tx /T (p) /Kids [5 0 R] >>".into(), // 4
"<< /Type /Annot /Subtype /Widget /FT /Tx /Parent 4 0 R /Rect [50 700 150 715] /P 3 0 R >>".into(), // 5
];
let all = fields(&document(&objs));
assert_eq!(all.iter().map(|f| (f.name.as_str(), f.widgets.len())).collect::<Vec<_>>(), [("p", 1)]);
}
/// A nameless widget listed in `/Fields` whose `/Parent` doesn't list it in its `/Kids` is still
/// read, as it was before such widgets were read through their parent.
#[test]
fn a_listed_widget_its_parent_omits_is_still_read() {
let objs: Vec<String> = vec![
"<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [5 0 R] >> >>".into(), // 1
"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 800] >>".into(), // 2
"<< /Type /Page /Parent 2 0 R /Annots [5 0 R] >>".into(), // 3
"<< /FT /Tx /T (p) /Kids [] >>".into(), // 4 unlisted
"<< /Type /Annot /Subtype /Widget /FT /Tx /Parent 4 0 R /Rect [50 700 150 715] /P 3 0 R >>".into(), // 5
];
let all = fields(&document(&objs));
let widgets: Vec<u32> = all.iter().flat_map(|f| &f.widgets).map(|w| w.obj.num).collect();
assert_eq!(widgets, [5]);
}
/// A field merged with its widget whose `/Kids` holds a dictionary that isn't a widget keeps
/// itself as its one widget: only widget kids replace it (ISO 32000-2 §12.7.4).
#[test]
fn a_merged_field_with_a_kid_that_is_not_a_widget_keeps_itself_as_widget() {
let objs: Vec<String> = vec![
"<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [4 0 R] >> >>".into(), // 1
"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 800] >>".into(), // 2
"<< /Type /Page /Parent 2 0 R /Annots [4 0 R] >>".into(), // 3
"<< /Type /Annot /Subtype /Widget /FT /Tx /T (t) /Rect [50 700 150 715] /P 3 0 R /Kids [5 0 R] >>".into(), // 4
"<< /Foo 1 >>".into(), // 5
];
let all = fields(&document(&objs));
let widgets: Vec<(u32, [f64; 4])> = all.iter().flat_map(|f| &f.widgets).map(|w| (w.obj.num, w.rect)).collect();
assert_eq!(widgets, [(4, [50.0, 700.0, 150.0, 715.0])]);
}
#[test]
fn locked_fields_only_take_unlocking() {
let mut doc = one_page();
@@ -3761,6 +4087,55 @@ fn without_need_appearances_only_missing_appearances_are_drawn() {
assert!(with_field_appearances(&copy).is_none(), "every widget has one now");
}
/// ISO 32000-2 12.5.5 and Table 170: a normal appearance is a stream or a subdictionary of
/// streams chosen by `/AS`. A form XObject's dictionary without its stream (`x`: object 6 says
/// it is one, but 7.5.7 keeps streams out of object streams, so a writer that packed it there
/// lost the data), a subdictionary with no `/AS` (`s`), or a state whose entry is no stream
/// (`u`, the push button `p`, the check box `k`) draws nothing, so these are drawn like widgets
/// without one: `x` with its `/MK /BC` border at its `/BS /W` of 3, given by reference. A
/// subdictionary whose `/AS` names a stream (`t`) is the file's own.
fn appearances_without_a_stream() -> Document {
let widget = |n: &str, rest: &str| format!("<< /Type /Annot /Subtype /Widget /T ({n}) /F 4 /P 3 0 R {rest} >>");
let tx = "/FT /Tx /V (v) /DA (/Helv 12 Tf 0 g)";
document(&[
"<< /Type /Catalog /Pages 2 0 R /AcroForm 4 0 R >>".into(),
"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 800] >>".into(),
"<< /Type /Page /Parent 2 0 R /Annots [9 0 R 10 0 R 11 0 R 12 0 R 13 0 R 14 0 R] >>".into(),
"<< /Fields [9 0 R 10 0 R 11 0 R 12 0 R 13 0 R 14 0 R] /DA (/Helv 0 Tf 0 g) /DR << /Font << /Helv 5 0 R >> >> >>".into(),
"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>".into(),
"<< /Type /XObject /Subtype /Form /FormType 1 /BBox [0 0 106 14] >>".into(), // 6
"<< /W 3 >>".into(),
"<< /Length 18 >>\nstream\nBT (old T) Tj ET\nendstream".into(), // 8
widget("x", &format!("{tx} /Rect [54 705 160 719] /MK << /BC [0 0 0] /BG [0.88 0.88 0.88] >> /BS 7 0 R /AP << /N 6 0 R >>")),
widget("s", &format!("{tx} /Rect [54 680 160 694] /AP << /N << /On 8 0 R >> >>")), // 10
widget("t", &format!("{tx} /Rect [54 655 160 669] /AS /On /AP << /N << /On 8 0 R >> >>")),
widget("u", &format!("{tx} /Rect [54 630 160 644] /AS /On /AP << /N << /On 6 0 R >> >>")),
widget("p", "/FT /Btn /Ff 65536 /DA (/Helv 12 Tf 0 g) /MK << /CA (Go) /BC [0 0 0] >> /Rect [54 600 160 620] /AP << /N 6 0 R >>"),
widget(
"k",
"/FT /Btn /V /Yes /AS /Yes /DA (/ZaDb 0 Tf 0 g) /MK << /CA (4) /BC [0 0 0] >> /Rect [200 700 224 724] /AP << /N << /Yes 6 0 R /Off 8 0 R >> >>",
),
])
}
#[test]
fn normal_appearances_that_give_no_stream_are_drawn() {
let doc = appearances_without_a_stream();
let copy = with_field_appearances(&doc).expect("x, s, u, p and k draw nothing of their own");
let all = fields(&copy);
let x = ap(&copy, &field(&all, "x").widgets[0]);
assert!(x.contains("0.88 0.88 0.88 rg\n0 0 106 14 re f") && x.contains("0 0 0 RG\n3 w\n1.5 1.5 103 11 re S"), "{x}");
assert!(x.contains("(v) Tj"), "{x}");
for name in ["s", "u"] {
assert!(ap(&copy, &field(&all, name).widgets[0]).contains("(v) Tj"), "{name}");
}
assert!(ap(&copy, &field(&all, "p").widgets[0]).contains("(Go) Tj"));
assert!(state_ap(&copy, &field(&all, "k").widgets[0], "Yes").contains("(4) Tj"));
let t = |d: &Document| d.get(field(&fields(d), "t").widgets[0].obj).as_dict().and_then(|w| w.get(b"AP").cloned());
assert_eq!(t(&copy), t(&doc), "a state with a stream is the file's own");
assert!(with_field_appearances(&copy).is_none(), "every widget has a stream now");
}
/// #3470: refreshing the display copy redraws only the widgets whose inputs changed. Typing in
/// one field of a form that leaves every appearance to the viewer otherwise redrew all of them.
#[test]
crates/forms/src/values.rs
+35 -4
View File
@@ -1,5 +1,5 @@
use super::appearances::{declares_pdfa, need_appearances};
use super::read::{acroform, field_flags, fields, page_refs};
use super::read::{acroform, field_flags, fields, listing_fields_named, page_refs};
use super::{
Dict, Document, Field, FieldEvent, FieldKind, FieldValue, FormError, NoScripts, ObjRef, Object, PdfString, Scripts, Widget, af, appearance,
author, calc_order, flags, invalid, name_obj, scripting,
@@ -89,8 +89,23 @@ impl NamedFields {
/// Write `value` into the field `name`, without recalculating: `named` are the positions in
/// `all` (the form as read before this write) of the fields with that name, in field order.
fn fill(doc: &mut Document, all: &[Field], named: &[usize], name: &str, value: &FieldValue, scripts: &mut dyn Scripts) -> Result<(), FormError> {
let f = named.first().and_then(|&i| all.get(i)).ok_or_else(|| FormError::NoSuchField(name.into()))?;
if f.read_only() {
let mut same = named.iter().filter_map(|&i| all.get(i));
let first = same.clone().next().ok_or_else(|| FormError::NoSuchField(name.into()))?;
// Fields with the same fully qualified name are one field (ISO 32000-2 §12.7.4.2), so a radio
// option any of them draws is an option of the name: it is written through the first one
// that draws it, and the others turn off below.
let choice = match value {
FieldValue::Radio(Some(c)) | FieldValue::Text(c) => Some(c.as_str()),
_ => None,
};
let f = match choice {
Some(c) if first.kind == FieldKind::Radio => same.find(|t| t.kind == FieldKind::Radio && t.state_for(c).is_some()).unwrap_or(first),
_ => first,
};
// The name is read-only when the first field of it is, whichever field draws the choice; and
// a radio choice reaches every radio field of the name, so none of them may be read-only.
let radio_locked = f.kind == FieldKind::Radio && named.iter().filter_map(|&i| all.get(i)).any(|t| t.kind == FieldKind::Radio && t.read_only());
if f.read_only() || first.read_only() || radio_locked {
return Err(FormError::ReadOnly(name.into()));
}
write_value(doc, f, value, scripts)?;
@@ -105,6 +120,16 @@ fn fill(doc: &mut Document, all: &[Field], named: &[usize], name: &str, value: &
share_value(doc, f.obj, t, &taken, scripts)?;
}
}
// A field whose `/Kids` lists radio fields that don't name it as their `/Parent` has their
// name too, so it takes the same value (§12.7.4.2) rather than keep a stale choice.
if f.kind == FieldKind::Radio {
let objs: Vec<ObjRef> = named.iter().filter_map(|&i| all.get(i)).map(|t| t.obj).collect();
if let Some(v) = doc.get(f.obj).as_dict().and_then(|d| d.get(b"V").cloned()) {
for p in listing_fields_named(doc, name, &objs) {
doc.update_dict(p, |d| d.set(b"V".to_vec(), v.clone()))?;
}
}
}
Ok(())
}
@@ -118,7 +143,13 @@ fn share_value(doc: &mut Document, source: ObjRef, twin: &Field, value: &[String
}
FieldKind::Radio => {
let choice = value.first().filter(|c| twin.widgets.iter().any(|w| w.on_state.as_deref() == Some(c.as_str())));
set_states(doc, twin, choice.map(String::as_str))
set_states(doc, twin, choice.map(String::as_str))?;
// The same value (§12.7.4.2), even when none of the twin's buttons draws it: they all
// stay off, and only the field that draws the choice shows it.
match value.first() {
Some(v) if choice.is_none() => Ok(doc.update_dict(twin.obj, |d| d.set(b"V".to_vec(), name_obj(v)))?),
_ => Ok(()),
}
}
FieldKind::Text | FieldKind::Combo | FieldKind::List => {
let src = doc.get(source).as_dict().cloned().unwrap_or_default();
crates/measure/README.md
+1 -1
View File
@@ -8,7 +8,7 @@ Scale::calibrate derives a conversion from two points and a known real-world dis
The desktop Measure objects panel exposes all three tools, live values and coordinate deltas, 45-degree constraints, vertex cancellation, snapping, calibration and CSV export. Automation provides measure_distance, measure_perimeter, measure_area, measure_info, measure_list, measure_scale, measure_snap and measure_export. The panel's drawing scale and measure_scale's `page_distance`/`page_unit` give the page side in a unit, by default the real-world one. Tool coordinates and explicit units-per-point scales use physical display points.
Snapping interprets painted paths and nested Form XObjects, including graphics-state transforms. Cubic paths are flattened to a maximum deviation of 0.2 user units. Raster edges, glyph outlines, clipping and optional-content visibility are not interpreted. Extraction stops at 20,000 segments, 40,000 endpoint and midpoint targets, 16 MiB of decoded content, 4,096 streams, 32 nested forms or 256 nested `q`; intersection searches use at most 256 nearby segments. Extraction is lenient: a content stream that can't be decoded is skipped and counted, and the rest of the page still snaps. The UI and automation report these limits.
Snapping interprets painted paths and nested Form XObjects, including graphics-state transforms. Cubic paths are flattened to a maximum deviation of 0.2 user units. Raster edges, glyph outlines, clipping and optional-content visibility are not interpreted. Extraction stops at 8,000,000 segments, 24,000,000 endpoint and midpoint targets, 512 MiB of decoded content, 4,096 streams, 32 nested forms or 256 nested `q`; intersection searches use at most 256 nearby segments. Extraction is lenient: a content stream that can't be decoded is skipped and counted, and the rest of the page still snaps. The UI and automation report these limits.
Imported single-unit decimal rectilinear scales support differing X and Y conversion factors when CYX is present. Unit labels are text strings and may be non-ASCII ("m²"). Compound units (such as feet and inches), fraction formats and non-rectilinear scales are not supported: `list` reports such measurements, and ones with invalid geometry, under `unsupported` with a reason instead of failing, and leaves their PDF data untouched. A polygon whose last vertex repeats the first is treated as closed. Interoperability with Acrobat-authored measurements is untested. Geospatial and 3D measuring are outside this component.
crates/measure/examples/snap_timing.rs
+32
View File
@@ -0,0 +1,32 @@
//! Times extracting a page's snapping geometry and snapping at points on it.
//!
//! Usage: `snap_timing FILE PAGE` (PAGE is 1-based).
use std::time::Instant;
fn main() -> Result<(), Box<dyn std::error::Error>> {
let args: Vec<String> = std::env::args().collect();
let (Some(path), Some(page)) = (args.get(1), args.get(2)) else {
return Err("usage: snap_timing FILE PAGE".into());
};
let page: usize = page.parse::<usize>()?.checked_sub(1).ok_or("pages are numbered from 1")?;
let bytes = std::sync::Arc::new(std::fs::read(path)?);
let doc = pdfcraft_cos::Document::open(bytes).map_err(|e| format!("{e:?}"))?;
let start = Instant::now();
let g = pdfcraft_measure::snap::geometry(&doc, page)?;
let extracted = start.elapsed();
println!("extracted {} segments in {} ms: truncated {} unreadable {}", g.segment_count(), extracted.as_millis(), g.truncated, g.unreadable);
// Snaps at points spread over the drawing, with a 6-unit tolerance (a few pixels on screen).
let points: Vec<[f64; 2]> =
g.segments().step_by((g.segment_count() / 2000).max(1)).map(|[a, b]| [(a[0] + b[0]) / 2.0 + 0.7, (a[1] + b[1]) / 2.0 - 0.4]).collect();
let start = Instant::now();
let mut hits = 0;
for &p in &points {
if g.snap(p, 6.0, pdfcraft_measure::snap::SnapOptions::default())?.is_some() {
hits += 1;
}
}
let per = start.elapsed().as_secs_f64() * 1e6 / points.len().max(1) as f64;
println!("{} snaps, {hits} hits, {per:.0} us per snap", points.len());
Ok(())
}
crates/measure/src/snap.rs
+389 -36
View File
@@ -5,19 +5,50 @@ use pdfcraft_cos::{DecodeBudget, Dict, Document, ObjRef, Object};
use serde::Serialize;
use std::collections::HashSet;
const MAX_SEGMENTS: usize = 20_000;
/// Endpoints and midpoints together, across the whole page.
const MAX_TARGETS: usize = 2 * MAX_SEGMENTS;
const MAX_BYTES: usize = 16 * 1024 * 1024;
/// Open subpaths remembered for a fill, which closes each of them.
const MAX_OPEN_SUBPATHS: usize = Limits::DEFAULT.segments;
/// How much of a page extraction reads and keeps.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Limits {
/// Straight segments (curves are split into several).
pub segments: usize,
/// Endpoints and midpoints together, across the whole page.
pub targets: usize,
/// Decoded content bytes, across every stream read.
pub bytes: usize,
}
impl Limits {
/// Enough for heavy CAD sheets (several million segments), and
/// the renderer's own content limit. At about 40 bytes a segment, the cap holds ~320 MB, plus
/// up to [`MAX_INDEX_ENTRIES`] in the index.
pub const DEFAULT: Self = Self { segments: 8_000_000, targets: 24_000_000, bytes: 512 << 20 };
/// What a window can read without missing frames that matter.
pub const WITHIN_A_FRAME: Self = Self { segments: 20_000, targets: 40_000, bytes: 16 << 20 };
/// Within what the index can number.
fn indexable(self) -> Self {
let most = u32::MAX as usize / 2;
Self { segments: self.segments.min(most), targets: self.targets.min(most), ..self }
}
}
impl Default for Limits {
fn default() -> Self {
Self::DEFAULT
}
}
/// The page's drawn geometry in user space, compactly (`f32`: about 0.0003 units of precision at
/// A0 sizes), with a grid index so that a snap looks only near the pointer.
#[derive(Clone, Debug, Default)]
pub struct Geometry {
pub segments: Vec<[Point; 2]>,
pub endpoints: Vec<Point>,
pub midpoints: Vec<Point>,
/// Segments as `[ax, ay, bx, by]`.
segs: Vec<[f32; 4]>,
/// Snap targets: a straight edge's or curve's two ends, then its midpoint.
tgts: Vec<[f32; 6]>,
/// A limit stopped extraction early (segments, targets, bytes, streams, nesting).
pub truncated: bool,
/// Content streams that couldn't be decoded and were skipped.
pub unreadable: usize,
limits: Limits,
grid: Grid,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize)]
#[serde(rename_all = "lowercase")]
@@ -80,7 +111,7 @@ fn remember_close(closures: &mut Vec<[Point; 2]>, current: Option<Point>, start:
if let (Some(a), Some(b)) = (current, start)
&& distance(a, b) > 1e-9
{
if closures.len() >= MAX_SEGMENTS {
if closures.len() >= MAX_OPEN_SUBPATHS {
return false;
}
closures.push([a, b]);
@@ -88,10 +119,45 @@ fn remember_close(closures: &mut Vec<[Point; 2]>, current: Option<Point>, start:
true
}
fn seg(s: &[f32; 4]) -> [Point; 2] {
[[f64::from(s[0]), f64::from(s[1])], [f64::from(s[2]), f64::from(s[3])]]
}
fn pt(x: f32, y: f32) -> Point {
[f64::from(x), f64::from(y)]
}
impl Geometry {
fn with_limits(limits: Limits) -> Self {
Self { limits, ..Self::default() }
}
/// How many segments were kept.
pub fn segment_count(&self) -> usize {
self.segs.len()
}
/// The segments, in drawing order.
pub fn segments(&self) -> impl ExactSizeIterator<Item = [Point; 2]> + '_ {
self.segs.iter().map(seg)
}
/// Snap endpoints: each edge's or curve's two ends, in drawing order.
pub fn endpoints(&self) -> impl Iterator<Item = Point> + '_ {
self.tgts.iter().flat_map(|t| [pt(t[0], t[1]), pt(t[2], t[3])])
}
/// Snap midpoints: each edge's or curve's middle, in drawing order.
pub fn midpoints(&self) -> impl Iterator<Item = Point> + '_ {
self.tgts.iter().map(|t| pt(t[4], t[5]))
}
/// Dense intersection searches are capped independently from path extraction.
pub fn intersection_limited(&self, at: Point, tolerance: f64) -> bool {
self.segments.iter().filter(|[a, b]| distance(at, nearest(at, *a, *b)) <= tolerance).take(257).count() > 256
if tolerance.is_nan() || tolerance < 0.0 {
return false;
}
let (segs, _) = self.grid.near(self, at, tolerance);
segs.iter()
.filter_map(|&i| self.segs.get(i as usize))
.map(seg)
.filter(|[a, b]| distance(at, nearest(at, *a, *b)) <= tolerance)
.take(257)
.count()
> 256
}
pub fn snap(&self, at: Point, tolerance: f64, options: SnapOptions) -> Result<Option<Snap>> {
check_points(&[at])?;
@@ -106,19 +172,25 @@ impl Geometry {
*best = Some(Snap { point: p, kind, distance });
}
};
// Only what the grid finds near the pointer: everything within `tolerance` is there, in
// drawing order, so the result is the one a scan of the whole page would give.
let (segs, tgts) = self.grid.near(self, at, tolerance);
let targets: Vec<&[f32; 6]> = tgts.iter().filter_map(|&i| self.tgts.get(i as usize)).collect();
if options.endpoints {
for &p in &self.endpoints {
consider(&mut best, p, SnapKind::Endpoint);
for t in &targets {
consider(&mut best, pt(t[0], t[1]), SnapKind::Endpoint);
consider(&mut best, pt(t[2], t[3]), SnapKind::Endpoint);
}
}
if options.midpoints {
for &p in &self.midpoints {
consider(&mut best, p, SnapKind::Midpoint);
for t in &targets {
consider(&mut best, pt(t[4], t[5]), SnapKind::Midpoint);
}
}
let near: Vec<[Point; 2]> = segs.iter().filter_map(|&i| self.segs.get(i as usize)).map(seg).collect();
// Only segments in the pointer's neighbourhood participate in intersection tests.
// Cap the local quadratic work even on a malicious densely overlapping drawing.
let local: Vec<_> = self.segments.iter().filter(|[a, b]| distance(at, nearest(at, *a, *b)) <= tolerance).take(256).collect();
let local: Vec<_> = near.iter().filter(|[a, b]| distance(at, nearest(at, *a, *b)) <= tolerance).take(256).collect();
if options.intersections {
for (i, s) in local.iter().enumerate() {
for t in local.iter().skip(i + 1) {
@@ -129,35 +201,36 @@ impl Geometry {
}
}
if options.paths && best.is_none() {
for [a, b] in &self.segments {
for [a, b] in &near {
consider(&mut best, nearest(at, *a, *b), SnapKind::Path);
}
}
Ok(best)
}
/// Endpoints and midpoints together.
fn targets(&self) -> usize {
self.endpoints.len().saturating_add(self.midpoints.len())
self.tgts.len().saturating_mul(3)
}
/// Whether the segment was accepted (finite, non-degenerate and within the cap).
fn segment(&mut self, a: Point, b: Point) -> bool {
if self.segments.len() >= MAX_SEGMENTS {
if self.segs.len() >= self.limits.segments {
self.truncated = true;
return false;
}
if check_points(&[a, b]).is_ok() && distance(a, b) > 1e-9 {
self.segments.push([a, b]);
self.segs.push([a[0] as f32, a[1] as f32, b[0] as f32, b[1] as f32]);
return true;
}
false
}
/// Snap targets are recorded only for accepted geometry, and are capped.
fn target(&mut self, ends: [Point; 2], mid: Point) {
if self.targets() >= MAX_TARGETS {
if self.targets().saturating_add(3) > self.limits.targets {
self.truncated = true;
return;
}
self.endpoints.extend(ends);
self.midpoints.push(mid);
let [a, b] = ends;
self.tgts.push([a[0] as f32, a[1] as f32, b[0] as f32, b[1] as f32, mid[0] as f32, mid[1] as f32]);
}
fn edge(&mut self, a: Point, b: Point) {
if self.segment(a, b) {
@@ -166,7 +239,7 @@ impl Geometry {
}
fn curve(&mut self, points: [Point; 4], depth: usize) {
let [a, b, c, d] = points;
if self.segments.len() >= MAX_SEGMENTS {
if self.segs.len() >= self.limits.segments {
self.truncated = true;
return;
}
@@ -203,6 +276,7 @@ struct Walker<'a> {
/// expands and then shrinks pays for the expansion each time it is drawn.
budget: DecodeBudget,
streams: usize,
limits: Limits,
}
impl Walker<'_> {
fn walk(&mut self, object: &Object, resources: &Dict, initial: Matrix, depth: usize) -> Result<()> {
@@ -261,7 +335,7 @@ impl Walker<'_> {
let mut start = None;
let mut current = None;
let mut closures = Vec::new();
let mut path = Geometry::default();
let mut path = Geometry::with_limits(self.limits);
// Read on the first `Do` and kept: resolving a direct dictionary copies it.
let mut xobjects = None;
let point = |m: Matrix, x: f64, y: f64| {
@@ -270,8 +344,8 @@ impl Walker<'_> {
};
// One operator at a time: a stream of millions of short operators is never held whole.
for op in ops(data) {
if self.geometry.segments.len().saturating_add(path.segments.len()) >= MAX_SEGMENTS
|| self.geometry.targets().saturating_add(path.targets()) >= MAX_TARGETS
if self.geometry.segs.len().saturating_add(path.segs.len()) >= self.limits.segments
|| self.geometry.targets().saturating_add(path.targets()) >= self.limits.targets
{
self.geometry.truncated = true;
break;
@@ -327,9 +401,9 @@ impl Walker<'_> {
if let Some([a, b, c, d]) = curve
&& check_points(&[a, b, c, d]).is_ok()
{
let before = path.segments.len();
let before = path.segs.len();
path.curve([a, b, c, d], 0);
if path.segments.len() > before {
if path.segs.len() > before {
path.target([a, d], [(a[0] + 3.0 * b[0] + 3.0 * c[0] + d[0]) / 8.0, (a[1] + 3.0 * b[1] + 3.0 * c[1] + d[1]) / 8.0]);
}
current = Some(d);
@@ -369,8 +443,8 @@ impl Walker<'_> {
// Filling implicitly closes every open subpath, not just the last one.
// Stroke-only S leaves its open subpaths unchanged.
for [a, b] in closures.drain(..) {
if self.geometry.segments.len().saturating_add(path.segments.len()) >= MAX_SEGMENTS
|| self.geometry.targets().saturating_add(path.targets()) >= MAX_TARGETS
if self.geometry.segs.len().saturating_add(path.segs.len()) >= self.limits.segments
|| self.geometry.targets().saturating_add(path.targets()) >= self.limits.targets
{
path.truncated = true;
break;
@@ -382,9 +456,8 @@ impl Walker<'_> {
{
path.edge(a, b);
}
self.geometry.segments.append(&mut path.segments);
self.geometry.endpoints.append(&mut path.endpoints);
self.geometry.midpoints.append(&mut path.midpoints);
self.geometry.segs.append(&mut path.segs);
self.geometry.tgts.append(&mut path.tgts);
self.geometry.truncated |= path.truncated;
closures.clear();
current = None;
@@ -392,7 +465,7 @@ impl Walker<'_> {
}
"n" => {
self.geometry.truncated |= path.truncated;
path = Geometry::default();
path = Geometry::with_limits(self.limits);
closures.clear();
current = None;
start = None;
@@ -450,16 +523,296 @@ thread_local! {
pub(crate) static XOBJECT_DICT_READS: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
}
/// The page's geometry for snapping, within [`Limits::DEFAULT`].
pub fn geometry(doc: &Document, page_index: usize) -> Result<Geometry> {
geometry_with_limits(doc, page_index, Limits::DEFAULT)
}
/// The page's geometry for snapping, within `limits`.
pub fn geometry_with_limits(doc: &Document, page_index: usize, limits: Limits) -> Result<Geometry> {
let limits = limits.indexable();
let p = crate::page(doc, page_index)?;
let resources = p.dict.get(b"Resources").map(|o| doc.resolve(o));
let empty = Dict::new();
let resources = resources.as_ref().and_then(|o| o.as_dict()).unwrap_or(&empty);
let mut walker =
Walker { doc, geometry: Geometry::default(), seen: HashSet::new(), budget: DecodeBudget::new(MAX_BYTES, usize::MAX), streams: 0 };
let mut walker = Walker {
doc,
geometry: Geometry::with_limits(limits),
seen: HashSet::new(),
budget: DecodeBudget::new(limits.bytes, usize::MAX),
streams: 0,
limits,
};
// Extraction is lenient: limits and unreadable streams are reported in the geometry.
if let Some(contents) = p.dict.get(b"Contents") {
walker.walk(contents, resources, Matrix::IDENTITY, 0)?;
}
Ok(walker.geometry)
let mut geometry = walker.geometry;
geometry.segs.shrink_to_fit();
geometry.tgts.shrink_to_fit();
geometry.grid = Grid::build(&geometry, MAX_INDEX_ENTRIES);
Ok(geometry)
}
/// The most cells a side of the grid has.
const GRID_SIDE: usize = 1024;
/// A segment whose box covers more cells than this is kept in one list checked at every snap
/// (a long line would otherwise fill a whole row of cells).
const WIDE_CELLS: usize = 64;
/// The most cells one snap reads; a larger search area (a huge tolerance) scans everything.
const MAX_QUERY_CELLS: usize = 4096;
/// The most entries the segment lists of the cells hold (4 bytes each). Past it, shorter segments
/// go in the list checked at every snap: snaps get slower, and find the same.
const MAX_INDEX_ENTRIES: u64 = 64_000_000;
/// A uniform grid over the geometry's box: per cell, the segments and targets touching it.
#[derive(Clone, Debug, Default)]
struct Grid {
x0: f32,
y0: f32,
cell: f32,
cols: usize,
rows: usize,
/// Per cell, where its segment indices start in `seg_items` (one more entry than cells).
seg_start: Vec<u32>,
seg_items: Vec<u32>,
tgt_start: Vec<u32>,
tgt_items: Vec<u32>,
/// Segments too long for the cells.
wide: Vec<u32>,
}
impl Grid {
fn build(g: &Geometry, max_entries: u64) -> Self {
let mut bounds: Option<[f32; 4]> = None;
let mut grow = |x: f32, y: f32| {
let b = bounds.get_or_insert([x, y, x, y]);
*b = [b[0].min(x), b[1].min(y), b[2].max(x), b[3].max(y)];
};
for s in &g.segs {
grow(s[0], s[1]);
grow(s[2], s[3]);
}
for t in &g.tgts {
grow(t[0], t[1]);
grow(t[2], t[3]);
grow(t[4], t[5]);
}
let Some([x0, y0, x1, y1]) = bounds else { return Self::default() };
let items = g.segs.len().max(g.tgts.len()).max(1);
// About eight items a cell.
let side = ((items as f64 / 8.0).sqrt().ceil() as usize).clamp(1, GRID_SIDE);
let extent = (x1 - x0).max(y1 - y0).max(1e-3);
let cell = extent / side as f32;
let cols = (((x1 - x0) / cell) as usize + 1).min(GRID_SIDE);
let rows = (((y1 - y0) / cell) as usize + 1).min(GRID_SIDE);
let mut grid = Self { x0, y0, cell, cols, rows, ..Self::default() };
let cells = cols * rows;
// Segments: count, then fill (compressed rows).
let seg_cells = |s: &[f32; 4]| grid.range(s[0].min(s[2]), s[1].min(s[3]), s[0].max(s[2]), s[1].max(s[3]));
let covered = |s: &[f32; 4]| {
let (c0, r0, c1, r1) = seg_cells(s);
(c1 - c0 + 1) * (r1 - r0 + 1)
};
// A segment of one cell adds one entry, so the last step fits whatever the sheet.
let most_cells = [WIDE_CELLS, 16, 4]
.into_iter()
.find(|&most| g.segs.iter().map(covered).filter(|&n| n <= most).map(|n| n as u64).sum::<u64>() <= max_entries)
.unwrap_or(1);
let mut counts = vec![0u32; cells];
let mut wide = Vec::new();
for (i, s) in g.segs.iter().enumerate() {
let (c0, r0, c1, r1) = seg_cells(s);
if (c1 - c0 + 1) * (r1 - r0 + 1) > most_cells {
wide.push(i as u32);
continue;
}
for r in r0..=r1 {
for c in c0..=c1 {
counts[r * cols + c] += 1;
}
}
}
let (seg_start, mut fill) = Self::starts(&counts);
let mut seg_items = vec![0u32; seg_start.last().copied().unwrap_or(0) as usize];
for (i, s) in g.segs.iter().enumerate() {
let (c0, r0, c1, r1) = seg_cells(s);
if (c1 - c0 + 1) * (r1 - r0 + 1) > most_cells {
continue;
}
for r in r0..=r1 {
for c in c0..=c1 {
let k = r * cols + c;
seg_items[fill[k] as usize] = i as u32;
fill[k] += 1;
}
}
}
// Targets: the cells of their three points.
let tgt_cells = |t: &[f32; 6]| {
let mut cs = [grid.cell_of(t[0], t[1]), grid.cell_of(t[2], t[3]), grid.cell_of(t[4], t[5])];
cs.sort_unstable();
cs
};
let mut counts = vec![0u32; cells];
for t in &g.tgts {
let cs = tgt_cells(t);
for (j, &k) in cs.iter().enumerate() {
if j == 0 || cs[j - 1] != k {
counts[k] += 1;
}
}
}
let (tgt_start, mut fill) = Self::starts(&counts);
let mut tgt_items = vec![0u32; tgt_start.last().copied().unwrap_or(0) as usize];
for (i, t) in g.tgts.iter().enumerate() {
let cs = tgt_cells(t);
for (j, &k) in cs.iter().enumerate() {
if j == 0 || cs[j - 1] != k {
tgt_items[fill[k] as usize] = i as u32;
fill[k] += 1;
}
}
}
grid.seg_start = seg_start;
grid.seg_items = seg_items;
grid.tgt_start = tgt_start;
grid.tgt_items = tgt_items;
grid.wide = wide;
grid
}
/// Prefix sums of `counts` (the starts), and a copy to fill from.
fn starts(counts: &[u32]) -> (Vec<u32>, Vec<u32>) {
let mut start = Vec::with_capacity(counts.len() + 1);
let mut total = 0u32;
start.push(0);
for &c in counts {
total = total.saturating_add(c);
start.push(total);
}
let fill = start[..counts.len()].to_vec();
(start, fill)
}
fn col(&self, x: f32) -> usize {
(((x - self.x0) / self.cell).max(0.0) as usize).min(self.cols.saturating_sub(1))
}
fn row(&self, y: f32) -> usize {
(((y - self.y0) / self.cell).max(0.0) as usize).min(self.rows.saturating_sub(1))
}
fn cell_of(&self, x: f32, y: f32) -> usize {
self.row(y) * self.cols + self.col(x)
}
/// The cells (first and last column, first and last row) a box covers.
fn range(&self, x0: f32, y0: f32, x1: f32, y1: f32) -> (usize, usize, usize, usize) {
(self.col(x0), self.row(y0), self.col(x1), self.row(y1))
}
/// The segments and targets near `at` (all within `tolerance` of it, and maybe more), each
/// list in drawing order without repeats.
fn near(&self, g: &Geometry, at: Point, tolerance: f64) -> (Vec<u32>, Vec<u32>) {
let all = || ((0..g.segs.len() as u32).collect(), (0..g.tgts.len() as u32).collect());
if self.cols == 0 || self.rows == 0 {
return if g.segs.is_empty() && g.tgts.is_empty() { (Vec::new(), Vec::new()) } else { all() };
}
// Rounding to `f32` can stop the box short of a candidate's cell: this much wider can't.
let t = (tolerance + (at[0].abs().max(at[1].abs()) + tolerance) * 2.0 * f64::from(f32::EPSILON)) as f32;
let (x, y) = (at[0] as f32, at[1] as f32);
if !(x.is_finite() && y.is_finite() && t.is_finite()) {
return all();
}
let (c0, r0, c1, r1) = self.range(x - t, y - t, x + t, y + t);
// Entirely outside the grid: nothing is near (except wide segments).
let outside =
x + t < self.x0 || y + t < self.y0 || x - t > self.x0 + self.cell * self.cols as f32 || y - t > self.y0 + self.cell * self.rows as f32;
if (c1 - c0 + 1) * (r1 - r0 + 1) > MAX_QUERY_CELLS {
return all();
}
let mut segs: Vec<u32> = self.wide.clone();
let mut tgts: Vec<u32> = Vec::new();
if !outside {
for r in r0..=r1 {
for c in c0..=c1 {
let k = r * self.cols + c;
let range = |start: &[u32]| (start[k] as usize)..(start[k + 1] as usize);
segs.extend_from_slice(self.seg_items.get(range(&self.seg_start)).unwrap_or(&[]));
tgts.extend_from_slice(self.tgt_items.get(range(&self.tgt_start)).unwrap_or(&[]));
}
}
}
segs.sort_unstable();
segs.dedup();
tgts.sort_unstable();
tgts.dedup();
(segs, tgts)
}
}
#[cfg(test)]
mod grid_tests {
use super::*;
fn geometry(segs: Vec<[f32; 4]>, tgts: Vec<[f32; 6]>, max_entries: u64) -> Geometry {
let mut g = Geometry { segs, tgts, ..Geometry::default() };
g.grid = Grid::build(&g, max_entries);
g
}
/// Past the allowance the index holds fewer entries and finds the same.
#[test]
fn a_bounded_index_finds_what_a_full_one_does() {
let mut segs = Vec::new();
for i in 0..4000u16 {
let (x, y) = (f32::from(i % 64) * 15.0, f32::from(i / 64) * 15.0);
segs.push([x, y, x + 37.0, y + 41.0]);
}
let tgts = Vec::new();
let full = geometry(segs.clone(), tgts.clone(), u64::MAX);
let budget = segs.len() as u64;
let bounded = geometry(segs, tgts, budget);
assert!(full.grid.seg_items.len() as u64 > budget, "the sheet needs more than the allowance");
assert!(bounded.grid.seg_items.len() as u64 <= budget, "{} entries", bounded.grid.seg_items.len());
let options = SnapOptions { endpoints: false, midpoints: false, intersections: true, paths: true };
for i in 0..400u16 {
let at = [f64::from(i) * 2.37, f64::from(i) * 1.13];
assert_eq!(bounded.snap(at, 6.0, options).unwrap(), full.snap(at, 6.0, options).unwrap(), "at {at:?}");
}
}
#[test]
fn limits_stay_within_what_the_index_can_number() {
let limits = Limits { segments: usize::MAX, targets: usize::MAX, bytes: 1 }.indexable();
assert!(limits.segments <= u32::MAX as usize && limits.targets <= u32::MAX as usize);
assert_eq!(Limits::DEFAULT.indexable(), Limits::DEFAULT);
}
/// A tolerance below zero finds nothing; it must not reverse the cells searched.
#[test]
fn a_negative_tolerance_finds_nothing() {
let segs = (0..4000u16)
.map(|i| [f32::from(i % 64) * 15.0, f32::from(i / 64) * 15.0, f32::from(i % 64) * 15.0 + 5.0, f32::from(i / 64) * 15.0])
.collect();
let g = geometry(segs, Vec::new(), u64::MAX);
assert!(g.grid.cols > 1 && g.grid.rows > 1);
assert!(!g.intersection_limited([400.0, 400.0], -200.0));
}
/// Near 1e8 one `f32` step is 8 units, which must not hide an end just within tolerance.
#[test]
fn rounding_doesnt_hide_a_target_at_the_edge_of_the_grid() {
let g = geometry(Vec::new(), vec![[100_000_008.0, 0.0, 100_000_016.0, 0.0, 100_000_012.0, 0.0]], u64::MAX);
let options = SnapOptions { endpoints: true, midpoints: false, intersections: false, paths: false };
let hit = g.snap([100_000_003.999_999_9, 0.0], 4.000_000_11, options).unwrap();
assert_eq!(hit.map(|h| h.point), Some([100_000_008.0, 0.0]));
}
/// A target is three points, and all three must fit.
#[test]
fn a_target_is_kept_only_if_all_its_points_fit() {
let mut g = Geometry::with_limits(Limits { segments: 10, targets: 2, bytes: 1 });
g.target([[0.0, 0.0], [1.0, 0.0]], [0.5, 0.0]);
assert_eq!(g.targets(), 0);
assert!(g.truncated);
}
}
crates/measure/src/tests.rs
+31 -27
View File
@@ -112,7 +112,7 @@ fn filled_open_triangle_snaps_to_closing_edge() {
for (op, closed) in [("f", true), ("F", true), ("f*", true), ("B", true), ("B*", true), ("h f", true), ("S", false)] {
let d = fixture(&format!("40 50 m 160 50 l 100 170 l {op}"), "", &[]);
let paths = snap::geometry(&d, 0).unwrap();
assert_eq!(paths.segments.len(), if closed { 3 } else { 2 }, "{op}");
assert_eq!(paths.segment_count(), if closed { 3 } else { 2 }, "{op}");
let hit = paths.snap([70.0, 110.0], 5.0, snap::SnapOptions::default()).unwrap();
assert_eq!(hit.is_some(), closed, "{op}");
}
@@ -121,7 +121,7 @@ fn filled_open_triangle_snaps_to_closing_edge() {
fn snap_paths_endpoints_midpoints_and_intersections() {
let d = fixture("10 20 m 110 20 l S 60 0 m 60 80 l S 10 100 100 40 re S", "", &[]);
let paths = snap::geometry(&d, 0).unwrap();
assert_eq!(paths.segments.len(), 6);
assert_eq!(paths.segment_count(), 6);
let opts = snap::SnapOptions::default();
let hit = paths.snap([11.0, 20.0], 3.0, opts).unwrap().unwrap();
assert_eq!(hit.kind, snap::SnapKind::Endpoint);
@@ -147,9 +147,9 @@ fn filled_open_subpath_offers_the_same_snap_edges_as_explicit_close() {
let explicit = snap::geometry(&closed, 0).unwrap();
let stroked = snap::geometry(&stroke, 0).unwrap();
assert_eq!(filled.segments, explicit.segments);
assert_eq!(filled.segments.len(), 3);
assert_eq!(stroked.segments.len(), 2);
assert!(filled.segments().eq(explicit.segments()));
assert_eq!(filled.segment_count(), 3);
assert_eq!(stroked.segment_count(), 2);
let paths_only = snap::SnapOptions { endpoints: false, midpoints: false, intersections: false, paths: true };
let hit = filled.snap([55.0, 80.0], 5.0, paths_only).unwrap().unwrap();
assert_eq!(hit.kind, snap::SnapKind::Path);
@@ -163,12 +163,12 @@ fn fills_close_all_open_subpaths_but_strokes_do_not() {
for paint in ["f", "F", "f*", "B", "B*", "b", "b*"] {
let doc = fixture(&format!("{triangles} {paint}"), "", &[]);
let geometry = snap::geometry(&doc, 0).unwrap();
assert_eq!(geometry.segments.len(), 6, "{paint}");
assert!(geometry.segments.contains(&[[10.0, 20.0], [0.0, 0.0]]), "{paint}");
assert!(geometry.segments.contains(&[[50.0, 20.0], [40.0, 0.0]]), "{paint}");
assert_eq!(geometry.segment_count(), 6, "{paint}");
assert!(geometry.segments().any(|s| s == [[10.0, 20.0], [0.0, 0.0]]), "{paint}");
assert!(geometry.segments().any(|s| s == [[50.0, 20.0], [40.0, 0.0]]), "{paint}");
}
let stroked = fixture(&format!("{triangles} S"), "", &[]);
assert_eq!(snap::geometry(&stroked, 0).unwrap().segments.len(), 4);
assert_eq!(snap::geometry(&stroked, 0).unwrap().segment_count(), 4);
}
#[test]
@@ -180,12 +180,12 @@ fn nested_forms_transforms_curves_and_cycles_are_bounded() {
);
let d = fixture("q 1 0 0 1 100 200 cm /Nested Do Q", "", &[&form]);
let geometry = snap::geometry(&d, 0).unwrap();
assert_eq!(geometry.segments[0], [[100.0, 200.0], [120.0, 200.0]]);
assert!(geometry.endpoints.contains(&[120.0, 200.0]));
assert!(geometry.midpoints.contains(&[110.0, 222.5]));
assert!(geometry.segments.len() > 4 && geometry.segments.len() < 200);
assert_eq!(geometry.segments().next(), Some([[100.0, 200.0], [120.0, 200.0]]));
assert!(geometry.endpoints().any(|p| p == [120.0, 200.0]));
assert!(geometry.midpoints().any(|p| p == [110.0, 222.5]));
assert!(geometry.segment_count() > 4 && geometry.segment_count() < 200);
let d = fixture(&"0 0 m 1 1 l S ".repeat(20_001), "", &[]);
assert!(snap::geometry(&d, 0).unwrap().truncated);
assert!(snap::geometry_with_limits(&d, 0, SMALL).unwrap().truncated);
}
#[test]
fn measurement_csv_quotes_newlines_and_blocks_formula_cells() {
@@ -242,35 +242,39 @@ fn imported_measurement_appearance_is_not_silently_replaced() {
close(list(&d).measurements[0].reading.value, 100.0);
}
/// The limits snapping had before heavy sheets, which the cap tests below were written for.
const SMALL: snap::Limits = snap::Limits { segments: 20_000, targets: 40_000, bytes: 16 << 20 };
#[test]
fn degenerate_and_dense_paths_keep_snap_targets_bounded() {
// Rejected (zero-length) segments record no endpoints or midpoints at all.
let d = fixture(&"0 0 0 0 re f ".repeat(100_000), "", &[]);
let g = snap::geometry(&d, 0).unwrap();
assert!(g.segments.is_empty() && g.endpoints.is_empty() && g.midpoints.is_empty(), "{} {}", g.endpoints.len(), g.midpoints.len());
let g = snap::geometry_with_limits(&d, 0, SMALL).unwrap();
assert!(g.segment_count() == 0 && g.endpoints().next().is_none() && g.midpoints().next().is_none());
// Every paint drains into the page geometry; the page-wide target cap still holds.
let d = fixture(&"0 0 1 1 re f ".repeat(30_000), "", &[]);
let g = snap::geometry(&d, 0).unwrap();
let g = snap::geometry_with_limits(&d, 0, SMALL).unwrap();
assert!(g.truncated);
assert!(g.endpoints.len() + g.midpoints.len() <= 40_000 + 12, "{}", g.endpoints.len() + g.midpoints.len());
assert!(g.segments.len() <= 20_000 + 4);
let targets = g.endpoints().count() + g.midpoints().count();
assert!(targets <= 40_000 + 12, "{targets}");
assert!(g.segment_count() <= 20_000 + 4);
// Unpainted curves and lines are capped too.
let d = fixture(&"0 0 m 5 9 1 9 7 0 c 0 3 l ".repeat(30_000), "", &[]);
let g = snap::geometry(&d, 0).unwrap();
assert!(g.truncated && g.endpoints.is_empty());
let g = snap::geometry_with_limits(&d, 0, SMALL).unwrap();
assert!(g.truncated && g.endpoints().next().is_none());
}
#[test]
fn deep_graphics_state_and_unreadable_streams_are_skipped_not_fatal() {
let d = fixture(&format!("0 0 m 10 0 l S {}", "q ".repeat(300)), "", &[]);
let g = snap::geometry(&d, 0).unwrap();
assert!(g.truncated);
assert_eq!(g.segments.len(), 1);
assert_eq!(g.segment_count(), 1);
// An unsupported filter cannot be decoded at all (corrupt Flate is decoded tolerantly).
let broken = "<< /Length 10 /Filter /NoSuchDecode >>\nstream\nnot-zlib!!\nendstream";
let d = fixture_with("[4 0 R 5 0 R]", "0 0 m 10 0 l S", "", &[broken]);
let g = snap::geometry(&d, 0).unwrap();
assert_eq!(g.unreadable, 1);
assert_eq!(g.segments.len(), 1);
assert_eq!(g.segment_count(), 1);
let d = fixture_with("5 0 R", "", "", &[broken]);
assert_eq!(snap::geometry(&d, 0).unwrap().unreadable, 1);
}
@@ -410,7 +414,7 @@ fn xobject_resources_are_read_once_per_stream() {
let d = fixture(&"/Missing Do\n".repeat(10_000), "", &[]);
snap::XOBJECT_DICT_READS.with(|c| c.set(0));
let paths = snap::geometry(&d, 0).unwrap();
assert!(paths.segments.is_empty());
assert_eq!(paths.segment_count(), 0);
let reads = snap::XOBJECT_DICT_READS.with(std::cell::Cell::get);
assert!(reads <= 1, "{reads} reads of the page's XObject resources");
}
@@ -466,10 +470,10 @@ fn a_stream_over_the_byte_budget_uses_it_up_once() {
let bomb = String::from_utf8(pdfcraft_testkit::stream_obj("/Filter [/ASCIIHexDecode /FlateDecode]", hex)).unwrap();
let contents = format!("[4 0 R{}]", " 5 0 R".repeat(50));
let d = fixture_with(&contents, "0 0 m 10 0 l S", "", &[bomb.as_str()]);
let g = snap::geometry(&d, 0).unwrap();
let g = snap::geometry_with_limits(&d, 0, SMALL).unwrap();
assert_eq!(g.unreadable, 1);
assert!(g.truncated);
assert_eq!(g.segments.len(), 1, "the parts before it still snap");
assert_eq!(g.segment_count(), 1, "the parts before it still snap");
}
/// A content stream drawn again and again pays for every filter stage's output each time, not
@@ -484,7 +488,7 @@ fn repeated_content_pays_for_every_filter_stage() {
.obj("<< /Type /Page /Parent 2 0 R /Contents [4 0 R 4 0 R 4 0 R 4 0 R 4 0 R 4 0 R 4 0 R 4 0 R] >>")
.stream("/Filter [/ASCIIHexDecode /RunLengthDecode]", &hex);
let d = Document::open(Arc::new(pdf.build())).unwrap();
let g = snap::geometry(&d, 0).unwrap();
let g = snap::geometry_with_limits(&d, 0, SMALL).unwrap();
assert!(g.truncated, "eight draws decode past the page's bound");
}
crates/measure/tests/snap_property.rs
+101 -7
View File
@@ -131,18 +131,20 @@ fn finite(point: [f64; 2]) -> bool {
fn geometry_of(content: &[u8]) -> Geometry {
let doc = Document::open(Arc::new(one_page_pdf(content))).unwrap();
snap::geometry(&doc, 0).unwrap()
// The caps these properties were written for.
let limits = snap::Limits { segments: MAX_GEOMETRY_SEGMENTS, targets: 2 * MAX_GEOMETRY_SEGMENTS, bytes: 16 << 20 };
snap::geometry_with_limits(&doc, 0, limits).unwrap()
}
/// Checks the bounds and finiteness that snapping relies on, and returns the extracted geometry.
fn check_snap_invariants(content: &[u8]) -> Geometry {
assert!(content.len() <= MAX_CONTENT_BYTES, "generated content exceeded the byte cap");
let geometry = geometry_of(content);
assert!(geometry.segments.len() <= MAX_GEOMETRY_SEGMENTS, "geometry exceeded the segment cap: {}", geometry.segments.len());
assert!(geometry.endpoints.len().saturating_add(geometry.midpoints.len()) <= MAX_SNAP_TARGETS, "geometry exceeded the snap-target cap");
assert!(geometry.segments.iter().flatten().copied().all(finite), "geometry contains a non-finite segment coordinate");
assert!(geometry.endpoints.iter().copied().all(finite), "geometry contains a non-finite endpoint");
assert!(geometry.midpoints.iter().copied().all(finite), "geometry contains a non-finite midpoint");
assert!(geometry.segment_count() <= MAX_GEOMETRY_SEGMENTS, "geometry exceeded the segment cap: {}", geometry.segment_count());
assert!(geometry.endpoints().count().saturating_add(geometry.midpoints().count()) <= MAX_SNAP_TARGETS, "geometry exceeded the snap-target cap");
assert!(geometry.segments().flatten().all(finite), "geometry contains a non-finite segment coordinate");
assert!(geometry.endpoints().all(finite), "geometry contains a non-finite endpoint");
assert!(geometry.midpoints().all(finite), "geometry contains a non-finite midpoint");
geometry.intersection_limited([0.0, 0.0], 10_000.0);
let snap = geometry.snap([0.0, 0.0], 10_000.0, SnapOptions::default()).unwrap();
@@ -173,7 +175,7 @@ fn mutated_snap_geometry_stays_bounded_and_finite() {
fn high_curvature_geometry_hits_segment_cap() {
let stress_curve = b"0 0 m 0 100000000 100000000 100000000 100000000 0 c S ".repeat(6);
let geometry = check_snap_invariants(&stress_curve);
assert!(geometry.truncated, "high-curvature geometry did not report its extraction cap ({} segments)", geometry.segments.len());
assert!(geometry.truncated, "high-curvature geometry did not report its extraction cap ({} segments)", geometry.segment_count());
}
fn two_piece_pdf(first: &[u8], second: &[u8]) -> Vec<u8> {
@@ -216,3 +218,95 @@ fn trailing_comment_in_previous_piece_hides_next_piece_first_line() {
assert!(snap_at_box_corner(b"q % tail comment, no end of line").is_none(), "snapped to a rectangle the renderer comments out");
assert!(snap_at_box_corner(b"q % tail comment\n").is_some(), "control: the rectangle is drawn");
}
/// A snap computed by scanning the whole page, as snapping did before its grid index: the
/// reference the indexed snap must equal.
fn snap_by_scan(g: &Geometry, at: [f64; 2], tolerance: f64, options: SnapOptions) -> Option<snap::Snap> {
let distance = |a: [f64; 2], b: [f64; 2]| (a[0] - b[0]).hypot(a[1] - b[1]);
let nearest = |p: [f64; 2], a: [f64; 2], b: [f64; 2]| {
let (dx, dy) = (b[0] - a[0], b[1] - a[1]);
let len2 = dx * dx + dy * dy;
if len2 <= f64::EPSILON {
return a;
}
let t = (((p[0] - a[0]) * dx + (p[1] - a[1]) * dy) / len2).clamp(0.0, 1.0);
[a[0] + t * dx, a[1] + t * dy]
};
let mut best: Option<snap::Snap> = None;
let consider = |best: &mut Option<snap::Snap>, p: [f64; 2], kind: snap::SnapKind| {
let d = distance(at, p);
if d <= tolerance && best.is_none_or(|b| d < b.distance - 1e-9) {
*best = Some(snap::Snap { point: p, kind, distance: d });
}
};
if options.endpoints {
g.endpoints().for_each(|p| consider(&mut best, p, snap::SnapKind::Endpoint));
}
if options.midpoints {
g.midpoints().for_each(|p| consider(&mut best, p, snap::SnapKind::Midpoint));
}
let local: Vec<[[f64; 2]; 2]> = g.segments().filter(|[a, b]| distance(at, nearest(at, *a, *b)) <= tolerance).take(256).collect();
if options.intersections {
for (i, s) in local.iter().enumerate() {
for t in local.iter().skip(i + 1) {
if let Some(p) = snap::intersection(s[0], s[1], t[0], t[1]) {
consider(&mut best, p, snap::SnapKind::Intersection);
}
}
}
}
if options.paths && best.is_none() {
g.segments().for_each(|[a, b]| consider(&mut best, nearest(at, a, b), snap::SnapKind::Path));
}
best
}
/// Whether more than 256 segments pass within `tolerance` of `at`, by a scan of the whole page.
fn limited_by_scan(g: &Geometry, at: [f64; 2], tolerance: f64) -> bool {
g.segments()
.filter(|[a, b]| {
let (dx, dy) = (b[0] - a[0], b[1] - a[1]);
let len2 = dx * dx + dy * dy;
let t = if len2 <= f64::EPSILON { 0.0 } else { (((at[0] - a[0]) * dx + (at[1] - a[1]) * dy) / len2).clamp(0.0, 1.0) };
(at[0] - (a[0] + t * dx)).hypot(at[1] - (a[1] + t * dy)) <= tolerance
})
.count()
> 256
}
#[test]
fn indexed_snapping_matches_a_scan_of_the_whole_page() {
let mut rng = Rng(SEED ^ 0x5eed);
let mut checked = 0usize;
for case in 0..200 {
let seed = SEEDS[rng.below(SEEDS.len())];
// A drawing with many short lines, some long ones (kept outside the cells) and curves.
let mut content = mutate(&mut rng, seed);
for i in 0..200 + rng.below(800) {
let (x, y) = (rng.below(600) as f64, rng.below(800) as f64);
let line = match i % 7 {
0 => format!("{x} {y} m {} {} l S ", x + rng.below(600) as f64, y),
1 => format!("{x} {y} m {} {} {} {} {} {} c S ", x + 5.0, y + 9.0, x + 9.0, y + 5.0, x + 12.0, y),
_ => format!("{x} {y} m {} {} l S ", x + rng.below(20) as f64, y + rng.below(20) as f64),
};
content.extend_from_slice(line.as_bytes());
}
let g = geometry_of(&content);
for _ in 0..40 {
let at = [rng.below(700) as f64 - 50.0 + 0.37, rng.below(900) as f64 - 50.0 + 0.61];
let tolerance = [0.5, 2.0, 7.5, 40.0, 400.0][rng.below(5)];
let options = SnapOptions {
endpoints: rng.below(4) != 0,
midpoints: rng.below(4) != 0,
intersections: rng.below(3) != 0,
paths: rng.below(2) == 0,
};
let indexed = g.snap(at, tolerance, options).unwrap();
let scanned = snap_by_scan(&g, at, tolerance, options);
assert_eq!(indexed, scanned, "case {case} at {at:?} tolerance {tolerance} {options:?}");
assert_eq!(g.intersection_limited(at, tolerance), limited_by_scan(&g, at, tolerance), "case {case} at {at:?} tolerance {tolerance}");
checked += 1;
}
}
assert!(checked >= 8000);
}
crates/organize/src/import.rs
+2 -2
View File
@@ -103,8 +103,8 @@ fn added_objects(dst: &Document, before: &HashSet<u32>) -> Vec<ObjRef> {
/// Insert Pages from File: [`import_pages_with`], and the inserted file's bookmarks come along
/// under one new bookmark titled `title` (the file's name) at its first inserted page, as Combine
/// Files does for each file. That bookmark goes among the top-level bookmarks where the pages went:
/// before the first one that points after them. Bookmarks to pages left out lose their destination;
/// a file without bookmarks adds none.
/// before the first one that points after them. Bookmarks to pages left out stay behind, unless a
/// bookmark below them leads to an inserted page; a file without bookmarks adds none.
pub fn insert_file_pages(
dst: &mut Document,
src: &Document,
crates/organize/src/import/carry.rs
+5 -1
View File
@@ -98,9 +98,13 @@ pub(super) fn source_outline(src: &Document, pages: HashMap<ObjRef, ObjRef>) ->
.and_then(|r| src.get(r).as_dict().cloned())
.and_then(|c| c.get(b"Outlines").map(|o| src.resolve(o)))
.and_then(|o| o.as_dict().and_then(|d| d.reference(b"First")));
let whole = crate::page_count(src).is_ok_and(|n| pages.len() >= n);
let mapper = Copier::new(src, HashMap::new(), pages);
let mut budget = 10_000usize;
outline_level(&mapper, first, 0, &mut budget)
let outline = outline_level(&mapper, first, 0, &mut budget);
// Of some of the pages, only the bookmarks that lead to a copied page and those above them:
// the others would carry their titles along into a file without their pages.
if whole { outline } else { kept_bookmarks(outline) }
}
/// `nodes` without the bookmarks whose page was not copied (no destination left), unless a bookmark
crates/organize/src/import/combine.rs
+15 -7
View File
@@ -1,5 +1,5 @@
use super::carry::{Mark, add_outline, collect_attachments, kept_bookmarks, set_attachments, source_outline, write_outline_level};
use super::{Dict, Document, FieldConflicts, HashMap, ObjRef, Object, OrganizeError, import_pages_mapped, rebuild};
use super::{Dict, Document, FieldConflicts, HashMap, HashSet, ObjRef, Object, OrganizeError, import_pages_mapped, rebuild};
/// A new document containing copies of `pages` from `src` (Extract Pages / Split).
/// Document information (title, author…) is carried over, and so is the print standard the
@@ -95,7 +95,8 @@ pub fn combine_with(sources: &[(&str, &Document)], conflicts: FieldConflicts) ->
}
/// Combine Files with chosen pages: each source contributes `pages` (0-based, in that order;
/// `None` for all of them). Bookmarks that point at pages left out lose their destination.
/// `None` for all of them). A source whose pages are not all used brings only the bookmarks that
/// lead to its copied pages (and those above them), and none of its attachments.
pub fn combine_selected(sources: &[(&str, &Document, Option<&[usize]>)]) -> Result<Document, OrganizeError> {
combine_selected_with(sources, FieldConflicts::Rename).map(|o| o.doc)
}
@@ -187,7 +188,11 @@ impl Combiner {
let outline = source_outline(src, imported.page_map);
self.marks.push(Mark { title: title.to_string(), page: *first, outline });
}
collect_attachments(&mut self.out, src, &mut self.attachments)?;
// Attachments belong to the whole file: they come along with all of its pages, not with a
// selection of them.
if pages.iter().collect::<HashSet<_>>().len() == n {
collect_attachments(&mut self.out, src, &mut self.attachments)?;
}
let declared = crate::pdfx::PrintStandard::of(src);
match self.standard.as_ref().map(|first| first.declared.same_as(&declared)) {
Some(same) => self.agreed &= same,
@@ -227,9 +232,9 @@ pub type Run<'a> = (usize, &'a str, &'a Document, Option<&'a [usize]>);
/// Combine Files where a file's pages may be split into several runs with other files' pages
/// between them. Each file (group) is copied once, all its pages together, then the pages are put
/// in the runs' order: its links between its own pages, its form fields and its attachments stay
/// whole, and it gets one top-level bookmark (the title of its first run) at its first page, its
/// own bookmarks nested. Otherwise as [`combine_selected`] (print standard, shared resources).
/// in the runs' order: its links between its own pages and its form fields stay whole across its
/// runs, its attachments come along when its runs use all of its pages, and it gets one top-level
/// bookmark (the title of its first run) at its first page, its own bookmarks nested. Otherwise as [`combine_selected`] (print standard, shared resources).
pub fn combine_grouped(runs: &[Run<'_>]) -> Result<Document, OrganizeError> {
combine_grouped_with(runs, FieldConflicts::Rename).map(|o| o.doc)
}
@@ -281,7 +286,10 @@ pub fn combine_grouped_with(runs: &[Run<'_>], conflicts: FieldConflicts) -> Resu
if let Some(first) = imported.pages.first() {
marks.push(Mark { title: file.title.to_string(), page: *first, outline: source_outline(file.src, imported.page_map) });
}
collect_attachments(&mut out, file.src, &mut attachments)?;
// Attachments come along with all of a file's pages, not with a selection of them.
if file.pages.iter().collect::<HashSet<_>>().len() == crate::page_count(file.src)? {
collect_attachments(&mut out, file.src, &mut attachments)?;
}
let declared = crate::pdfx::PrintStandard::of(file.src);
match &standard {
None => standard = Some((declared, file.src, imported.objects)),
crates/organize/src/tests.rs
+41 -3
View File
@@ -1591,13 +1591,12 @@ fn inserting_a_file_brings_its_bookmarks_under_one_named_after_it() {
let page_of = |title: &str| marks.iter().find(|b| b.title == title).and_then(|b| b.page);
assert_eq!((page_of("annex"), page_of("Chapter one"), page_of("Section"), page_of("End")), (Some(1), Some(1), Some(2), Some(4)));
assert!(!marks.iter().find(|b| b.title == "annex").unwrap().open, "the file's bookmarks start collapsed");
// Inserted last, after every bookmark; only the pages chosen keep a destination.
// Inserted last, after every bookmark; a bookmark to a page left out stays behind with it.
let mut host = open(fixture());
add_bookmark(&mut host, &[], 0, "Start", 0).unwrap();
insert_file_pages(&mut host, &doc_c(), "annex", &[0], 9, FieldConflicts::Rename).unwrap();
let marks = bookmarks(&host);
assert_eq!(titles(&marks), ["Start", "annex[Chapter one[Section]]"]);
assert_eq!(marks.iter().find(|b| b.title == "Section").unwrap().page, None, "its page was left out");
assert_eq!(titles(&marks), ["Start", "annex[Chapter one]"], "Section's page was left out");
// A file without bookmarks adds none, and a host without an outline gets one only when needed.
let mut host = open(fixture());
insert_file_pages(&mut host, &doc_b(), "plain", &[0], 0, FieldConflicts::Rename).unwrap();
@@ -4453,3 +4452,42 @@ fn a_form_action_brings_no_field_from_a_page_left_behind() {
// With every page, the actions still list it.
assert!(holds_value(&full_roundtrip(&extract_pages(&src, &[0, 1]).unwrap()), "90000"));
}
/// The titles of a bookmark's children, each with its own children's titles.
fn bookmark_titles(doc: &Document, first: Option<pdfcraft_cos::ObjRef>) -> Vec<(String, Vec<String>)> {
let mut out = Vec::new();
let mut at = first;
while let Some(r) = at {
let item = doc.get(r).as_dict().cloned().unwrap();
let title = item.get(b"Title").and_then(|t| t.as_string()).map(|t| t.to_text()).unwrap_or_default();
let children = bookmark_titles(doc, item.reference(b"First")).into_iter().map(|(t, _)| t).collect();
out.push((title, children));
at = item.reference(b"Next");
}
out
}
/// Combining some of a file's pages brings only the bookmarks that lead to them (and those above
/// them), and none of the file's attachments.
#[test]
fn combining_some_pages_brings_only_their_bookmarks_and_no_attachments() {
let source_bookmarks = |out: &Document| {
let outlines = out.get(catalog(out).reference(b"Outlines").unwrap()).as_dict().cloned().unwrap();
let file = out.get(outlines.reference(b"First").unwrap()).as_dict().cloned().unwrap();
bookmark_titles(out, file.reference(b"First"))
};
let attachments = |out: &Document| {
catalog(out).get(b"Names").map(|n| out.resolve(n)).is_some_and(|n| n.as_dict().is_some_and(|d| d.contains(b"EmbeddedFiles")))
};
// Page 1 only: "Section" leads to page 2.
let first = full_roundtrip(&combine_selected(&[("C", &doc_c(), Some(&[0]))]).unwrap());
assert_eq!(source_bookmarks(&first), [("Chapter one".to_string(), vec![])]);
assert!(!attachments(&first), "an attachment came along with a selection of pages");
// Page 2 only: "Chapter one" stays, as the way to "Section".
let second = full_roundtrip(&combine_selected(&[("C", &doc_c(), Some(&[1]))]).unwrap());
assert_eq!(source_bookmarks(&second), [("Chapter one".to_string(), vec!["Section".to_string()])]);
// Every page: all of it, attachments too.
let all = full_roundtrip(&combine_selected(&[("C", &doc_c(), Some(&[0, 1]))]).unwrap());
assert_eq!(source_bookmarks(&all), [("Chapter one".to_string(), vec!["Section".to_string()])]);
assert!(attachments(&all));
}
crates/preflight/src/gstate.rs
+235
View File
@@ -0,0 +1,235 @@
//! PDF/A-2 6.2.5, the extended graphics state: device-dependent settings a conforming file may not
//! rely on.
//!
//! - no transfer function: no `/TR` entry, and `/TR2` only as `/Default` (ISO 32000-1 10.4);
//! - no `/HTP` (halftone phase) entry, a PDF 1.2 key ISO 32000-1 no longer defines;
//! - halftones of type 1 or 5 only, without a `/HalftoneName` (ISO 32000-1 10.5.5); the halftones
//! a type 5 halftone holds for its colorants are checked the same way;
//! - a halftone's `/TransferFunction` only where one is needed: a type 5 halftone's component for
//! a non-primary colorant has one, and no other halftone does. The primary colour components
//! are those of the standard device spaces: Gray; Red, Green and Blue; Cyan, Magenta, Yellow
//! and Black (ISO 32000-1 10.5.5.6; ISO 32000-2 10.6.5.6). The `Default` component is left as
//! it is (ISO 32000-1 Table 130);
//! - a rendering intent (`/RI`) that is one of the four of ISO 32000-1 8.6.5.8, Table 70.
//!
//! Graphics state dictionaries are found as the `/ExtGState` entries of resource dictionaries
//! (`/Resources` of pages, forms, Type 3 fonts, patterns and annotation appearances, and an
//! interactive form's `/DR`), and as the `/ExtGState` of a shading pattern (ISO 32000-1 8.7.4.3,
//! Table 76). They are looked for in what the document reaches from its trailer, following
//! references, but not into private application data (`/PieceInfo`), so an object only that data
//! refers to isn't read. A map shared by reference is read once.
use std::collections::{BTreeSet, HashSet, VecDeque};
use pdfcraft_cos::{Dict, Document, ObjRef, Object};
/// How deep a direct object is followed: as deep as the parser nests them.
const MAX_DEPTH: u8 = 128;
/// The rendering intents of ISO 32000-1 Table 70.
const INTENTS: [&[u8]; 4] = [b"AbsoluteColorimetric", b"RelativeColorimetric", b"Saturation", b"Perceptual"];
/// The entries of a type 5 halftone that aren't colorant halftones.
const HALFTONE_KEYS: [&[u8]; 3] = [b"Type", b"HalftoneType", b"HalftoneName"];
/// The primary colour components of DeviceGray, DeviceRGB and DeviceCMYK, whose halftones take
/// the graphics state's transfer function.
const PRIMARY: [&[u8]; 8] = [b"Gray", b"Red", b"Green", b"Blue", b"Cyan", b"Magenta", b"Yellow", b"Black"];
/// The entries that hold resource dictionaries.
const RESOURCES: [&[u8]; 2] = [b"Resources", b"DR"];
/// What the graphics states of a document break of 6.2.5.
#[derive(Debug, Default, PartialEq, Eq)]
pub(crate) struct Findings {
/// A graphics state has a `/TR` entry.
pub(crate) transfer: bool,
/// A graphics state has a `/TR2` other than `/Default`.
pub(crate) transfer2: bool,
/// A graphics state has an `/HTP`.
pub(crate) phase: bool,
/// Halftone types other than 1 and 5, in order.
pub(crate) halftone_types: BTreeSet<i64>,
/// A halftone's `/HalftoneType` is missing or isn't an integer.
pub(crate) untyped: bool,
/// A halftone has a `/HalftoneName`.
pub(crate) halftone_name: bool,
/// A halftone that may not have a `/TransferFunction` has one.
pub(crate) halftone_transfer: bool,
/// Non-primary colorants whose halftone has no `/TransferFunction`, in name order.
pub(crate) missing_transfer: BTreeSet<String>,
/// Rendering intents outside Table 70, in name order.
pub(crate) intents: BTreeSet<String>,
/// A rendering intent isn't a name.
pub(crate) intent_not_name: bool,
}
/// The 6.2.5 findings over what `doc` reaches from its trailer.
pub(crate) fn check(doc: &Document) -> Findings {
scan(doc).findings
}
/// How many objects [`check`] reads.
#[cfg(test)]
pub(crate) fn reached(doc: &Document) -> usize {
scan(doc).reached.len()
}
fn scan(doc: &Document) -> Scan<'_> {
let mut scan = Scan { doc, read: HashSet::new(), reached: HashSet::new(), queue: VecDeque::new(), findings: Findings::default() };
scan.walk(&Object::Dict(doc.trailer().clone()), 0);
// References are followed from a queue, not by recursion: a chain of them can be as long as
// the file.
while let Some(r) = scan.queue.pop_front() {
let o = doc.get(r);
scan.walk(&o, 0);
}
scan
}
struct Scan<'a> {
doc: &'a Document,
/// Resource dictionaries, `/ExtGState` maps and graphics states read so far, by object number.
read: HashSet<u32>,
/// Object numbers reached so far, queued or walked. Both sets go by number, as
/// [`Document::get`] does: references differing only in generation reach the same object.
reached: HashSet<u32>,
queue: VecDeque<ObjRef>,
findings: Findings,
}
impl Scan<'_> {
fn walk(&mut self, o: &Object, depth: u8) {
if depth > MAX_DEPTH {
return;
}
let d = match o {
Object::Ref(r) => {
if self.reached.insert(r.num) {
self.queue.push_back(*r);
}
return;
}
Object::Array(a) => {
for v in a {
self.walk(v, depth.saturating_add(1));
}
return;
}
Object::Dict(d) => d,
Object::Stream(s) => &s.dict,
_ => return,
};
self.holder(d);
for (k, v) in d.iter() {
if k.as_slice() != b"PieceInfo" {
self.walk(v, depth.saturating_add(1));
}
}
}
/// Whether `o` is a reference read before; marks it read.
fn read_before(&mut self, o: &Object) -> bool {
matches!(o, Object::Ref(r) if !self.read.insert(r.num))
}
/// A dictionary that holds graphics states: through its resources, or as a shading pattern.
fn holder(&mut self, d: &Dict) {
for key in RESOURCES {
if let Some(resources) = d.get(key) {
self.resources(resources);
}
}
if d.get(b"PatternType").is_some()
&& let Some(gs) = d.get(b"ExtGState")
&& !self.read_before(gs)
&& let Some(gs) = self.doc.resolve(gs).as_dict()
{
self.gstate(gs);
}
}
fn resources(&mut self, resources: &Object) {
if self.read_before(resources) {
return;
}
let resources = self.doc.resolve(resources);
let Some(states) = resources.as_dict().and_then(|r| r.get(b"ExtGState")) else { return };
if self.read_before(states) {
return;
}
let states = self.doc.resolve(states);
let Some(states) = states.as_dict() else { return };
for (_, gs) in states.iter() {
if !self.read_before(gs)
&& let Some(gs) = self.doc.resolve(gs).as_dict()
{
self.gstate(gs);
}
}
}
/// `key` of `d`, resolved, unless it is missing or null (ISO 32000-1 7.3.9).
fn present(&self, d: &Dict, key: &[u8]) -> Option<std::sync::Arc<Object>> {
d.get(key).map(|v| self.doc.resolve(v)).filter(|v| !matches!(**v, Object::Null))
}
fn gstate(&mut self, gs: &Dict) {
if self.present(gs, b"TR").is_some() {
self.findings.transfer = true;
}
if self.present(gs, b"TR2").is_some_and(|f| f.as_name() != Some(b"Default")) {
self.findings.transfer2 = true;
}
if self.present(gs, b"HTP").is_some() {
self.findings.phase = true;
}
if let Some(ht) = self.present(gs, b"HT")
&& let Some(ht) = ht.as_dict()
{
self.halftone(ht, None);
}
if let Some(ri) = self.present(gs, b"RI") {
match ri.as_name() {
Some(ri) if !INTENTS.contains(&ri) => {
self.findings.intents.insert(String::from_utf8_lossy(ri).into_owned());
}
Some(_) => {}
None => self.findings.intent_not_name = true,
}
}
}
/// A halftone dictionary or stream: the graphics state's own (`colorant` is `None`), or the
/// one a type 5 halftone holds for `colorant`.
fn halftone(&mut self, ht: &Dict, colorant: Option<&[u8]>) {
// An integer (ISO 32000-1 Table 128), not a real of the same value.
let kind = self.present(ht, b"HalftoneType").and_then(|t| if let Object::Int(t) = *t { Some(t) } else { None });
match kind {
None => self.findings.untyped = true,
Some(1 | 5) => {}
Some(kind) => {
self.findings.halftone_types.insert(kind);
}
}
if self.present(ht, b"HalftoneName").is_some() {
self.findings.halftone_name = true;
}
let transfer = self.present(ht, b"TransferFunction").is_some();
match colorant {
Some(b"Default") => {}
Some(c) if !PRIMARY.contains(&c) => {
if !transfer {
self.findings.missing_transfer.insert(String::from_utf8_lossy(c).into_owned());
}
}
_ => self.findings.halftone_transfer |= transfer,
}
if colorant.is_none() && kind == Some(5) {
for (k, v) in ht.iter() {
if HALFTONE_KEYS.contains(&k.as_slice()) {
continue;
}
if let Some(component) = self.doc.resolve(v).as_dict() {
self.halftone(component, Some(k));
}
}
}
}
}
crates/preflight/src/header.rs
+71
View File
@@ -0,0 +1,71 @@
//! PDF/A-2 6.1.2, the file header: the file starts at its first byte with `%PDF-1.n`, `n` from 0
//! to 7, and an end-of-line marker; the next line is a comment, `%` and at least four bytes each
//! above 127, which marks the file as binary (ISO 32000-1 7.5.2).
//!
//! A document whose next save rewrites it in full is judged by the header that save writes: the
//! version it keeps, followed by the writer's own binary comment.
use pdfcraft_cos::Document;
/// What the header of `doc`, as it will be saved, breaks of 6.1.2.
#[derive(Debug, Default, PartialEq, Eq)]
pub(crate) struct Header {
/// The first line isn't `%PDF-1.0` to `%PDF-1.7` at the start of the file.
pub(crate) version: bool,
/// The second line isn't a comment of at least four bytes above 127.
pub(crate) comment: bool,
}
/// What the header of `doc` breaks of 6.1.2: as stored, or, when the next save rewrites the
/// file, as that save will write it.
pub(crate) fn check(doc: &Document) -> Header {
if doc.full_save_required() || doc.encryption_changed() {
let written = doc.version().get(..3).unwrap_or_default();
return Header { version: !allowed(written.as_bytes()), comment: false };
}
let data = doc.bytes();
let Some(rest) = data.strip_prefix(b"%PDF-") else { return Header { version: true, comment: false } };
let version = rest.get(..3).unwrap_or_default();
let Some(rest) = rest.get(3..).and_then(eol) else {
return Header { version: true, comment: !binary_comment(after_line(rest)) };
};
Header { version: !allowed(version), comment: !binary_comment(rest) }
}
/// Whether a version is `1.0` to `1.7`.
fn allowed(version: &[u8]) -> bool {
matches!(version, [b'1', b'.', b'0'..=b'7'])
}
/// What follows an end-of-line marker (CR, LF or CR LF) at the start of `data`.
fn eol(data: &[u8]) -> Option<&[u8]> {
match data {
[b'\r', b'\n', rest @ ..] | [b'\r', rest @ ..] | [b'\n', rest @ ..] => Some(rest),
_ => None,
}
}
/// What follows the first end-of-line marker in `data`, or nothing.
fn after_line(data: &[u8]) -> &[u8] {
let end = data.iter().position(|&b| b == b'\r' || b == b'\n').unwrap_or(data.len());
data.get(end..).and_then(eol).unwrap_or_default()
}
/// Whether `data` starts with `%` and four bytes above 127.
fn binary_comment(data: &[u8]) -> bool {
matches!(data, [b'%', a, b, c, d, ..] if [a, b, c, d].iter().all(|&&x| x > 127))
}
/// Makes the next save of `doc` write a header 6.1.2 allows, when the current one breaks it:
/// a full save, at version 1.7 when the version kept isn't 1.0 to 1.7. What it fixed, if anything.
pub(crate) fn rewrite(doc: &mut Document) -> Option<String> {
let header = check(doc);
if !header.version && !header.comment {
return None;
}
if !allowed(doc.version().as_bytes().get(..3).unwrap_or_default()) {
doc.set_version("1.7");
}
doc.require_full_save();
Some("Rewrote the file header".into())
}
crates/preflight/src/lib.rs
+400 -41
View File
@@ -10,7 +10,9 @@
//! - every font embedded;
//! - annotations: allowed types, printable, not hidden, with appearance streams;
//! - actions: none of the forbidden kinds (JavaScript, Launch, Sound, Movie, ResetForm,
//! ImportData, Hide, SetOCGState, Rendition, Trans, GoTo3DView); no additional actions on the
//! ImportData, Hide, SetOCGState, Rendition, Trans, GoTo3DView, and the PDF 1.2 SetState and
//! NOP that ISO 32000-1 no longer defines); Named actions only for NextPage, PrevPage,
//! FirstPage and LastPage (ISO 32000-1 12.6.4.11, Table 212); no additional actions on the
//! catalog, pages, fields or widgets; `NeedAppearances` not true;
//! - images: no `Interpolate`, no `Alternates`, no OPI; no PostScript XObjects;
//! - embedded files: PDF/A-2 only allows embedded PDF/A files; PDF/A-3 needs `AFRelationship`.
@@ -21,8 +23,15 @@
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
mod gstate;
mod header;
pub mod icc;
mod limits;
mod spot;
mod trailer;
mod transparency;
mod xmp;
mod xmp_extension;
mod xmp_types;
use pdfcraft_cos::{Dict, Document, ObjRef, Object, PdfString, Stream};
@@ -153,14 +162,46 @@ pub const WORDINGS: &[&str] = &[
"The title in the metadata and the document information differ",
"The XMP property {property} in metadata stream {object} should have the value type {type}",
"The XMP property {property} in metadata stream {object}, written {written}, should have the value type {type}",
"The XMP extension schema {schema} in metadata stream {object} has no {entry} entry",
"The XMP extension schema {schema} in metadata stream {object} describes {item} without a {entry} entry",
"The XMP extension schema {schema} in metadata stream {object} has an empty {entry} entry",
"The XMP extension schema {schema} in metadata stream {object} describes {item} with an empty {entry} entry",
"The XMP extension schema container in metadata stream {object} holds no rdf:Bag",
"The XMP extension schema container in metadata stream {object} holds {found}, not one rdf:Bag",
"Metadata stream {object} writes the namespace {namespace} without a prefix, not with {prefix}",
"Metadata stream {object} writes the namespace {namespace} with the prefix {written}, not {prefix}",
// The document as a whole.
"The file doesn't start with a %PDF-1.0 to %PDF-1.7 header line",
"The file header isn't followed by a comment of four bytes above 127",
"The trailer has no file identifier (ID)",
"The file identifier (ID) in the trailer isn't two non-empty strings",
"The document is encrypted",
"An integer is outside the 32-bit range",
"A real number is outside ±3.403 × 10³⁸, or nonzero but within ±1.175 × 10⁻³⁸",
"A CMap maps a character code to a CID above 65535",
"A string is longer than 32767 bytes",
"A name is longer than 127 bytes",
"Page {} has a boundary smaller than 3 or larger than 14400 units",
"Device colour is used, but there is no PDF/A output intent",
"Page {} has transparency but no device-independent blending colour space or PDF/A output intent",
"A graphics state uses the blend mode {}, which ISO 32000-1 doesn't define",
"DeviceCMYK is used: it needs a CMYK output intent (not added automatically)",
"DeviceCMYK is used, but the PDF/A output intent isn't a CMYK profile",
"DeviceRGB is used, but the PDF/A output intent isn't an RGB profile",
"Separation colour spaces named {} differ in their alternate space or tint transform",
"The spot colour {} of a DeviceN colour space isn't in its Colorants dictionary",
"The document has additional actions (/AA)",
"The document has document-level JavaScript",
"A graphics state has a transfer function (TR)",
"A graphics state has a transfer function (TR2) other than Default",
"A graphics state has a halftone phase (HTP)",
"A halftone has type {}: only types 1 and 5 are allowed",
"A halftone's HalftoneType is missing or isn't an integer",
"A halftone has a HalftoneName",
"A halftone has a transfer function, which only a non-primary colorant's halftone may have",
"The halftone for the colorant {} has no transfer function",
"A graphics state has the rendering intent {}, which isn't a standard one",
"A graphics state has a rendering intent that isn't a name",
"The form asks viewers to make appearances (NeedAppearances)",
// Pages, actions, annotations and fields.
"Page {} has additional actions",
@@ -192,8 +233,23 @@ pub struct Declared {
pub output_intents: Vec<String>,
}
const FORBIDDEN_ACTIONS: &[&[u8]] =
&[b"JavaScript", b"Launch", b"Sound", b"Movie", b"ResetForm", b"ImportData", b"Hide", b"SetOCGState", b"Rendition", b"Trans", b"GoTo3DView"];
const FORBIDDEN_ACTIONS: &[&[u8]] = &[
b"JavaScript",
b"Launch",
b"Sound",
b"Movie",
b"ResetForm",
b"ImportData",
b"Hide",
b"SetOCGState",
b"Rendition",
b"Trans",
b"GoTo3DView",
b"SetState",
b"NOP",
];
/// The named actions of ISO 32000-1 Table 212, the only ones a Named action may name.
const NAMED_ACTIONS: &[&[u8]] = &[b"NextPage", b"PrevPage", b"FirstPage", b"LastPage"];
const FORBIDDEN_ANNOTS: &[&[u8]] = &[b"Sound", b"Movie", b"Screen", b"3D", b"RichMedia"];
fn catalog(doc: &Document) -> Dict {
@@ -250,6 +306,19 @@ fn dict_of(o: &Object) -> Option<&Dict> {
}
}
/// Whether the document has a PDF/A output intent whose `/DestOutputProfile` is a stream: one with
/// a null or missing profile describes no colour, so it doesn't stand in for a blending colour
/// space (6.2.10).
fn pdfa_intent_has_profile(doc: &Document) -> bool {
let cat = catalog(doc);
let list = cat.get(b"OutputIntents").map(|o| doc.resolve(o)).and_then(|o| o.as_array().cloned()).unwrap_or_default();
list.iter().any(|oi| {
let oi = doc.resolve(oi);
let Some(d) = oi.as_dict() else { return false };
d.name(b"S") == Some(b"GTS_PDFA1") && d.get(b"DestOutputProfile").is_some_and(|p| matches!(&*doc.resolve(p), Object::Stream(_)))
})
}
fn has_pdfa_intent(doc: &Document) -> bool {
pdfa_intent_profile(doc).is_some()
}
@@ -369,6 +438,37 @@ fn referenced_content(doc: &Document, objs: &[(ObjRef, std::sync::Arc<Object>)])
out
}
/// The fonts the document draws with: those a resource dictionary names (`/Resources`, of a page or
/// a page tree node, a form XObject or an annotation's appearance stream, a pattern or a Type 3
/// font). PDF/A asks that every font "used for rendering" be embedded (ISO 19005-2, 6.2.11.4.1). A
/// font that only the interactive form's default resources hold (`/AcroForm /DR`, ISO 32000-1
/// 12.7.2) draws nothing, so it needn't be: an appearance stream that does draw with it names it in
/// its own `/Resources`.
fn drawing_fonts(doc: &Document, objs: &[(ObjRef, std::sync::Arc<Object>)]) -> std::collections::HashSet<ObjRef> {
let mut out = std::collections::HashSet::new();
for (_, o) in objs {
let Some(resources) = dict_of(o).and_then(|d| d.get(b"Resources")).map(|r| doc.resolve(r)) else { continue };
let Some(fonts) = resources.as_dict().and_then(|r| r.get(b"Font")).map(|f| doc.resolve(f)) else { continue };
if let Some(fonts) = fonts.as_dict() {
out.extend(fonts.iter().filter_map(|(_, f)| f.as_ref()).map(|r| font_object(doc, r)));
}
}
out
}
/// How many references [`font_object`] follows, as the document's own `resolve` does.
const MAX_REFERENCE_HOPS: usize = 32;
/// The object a font entry's reference `r` ends at: an object can be nothing but a reference to
/// another (ISO 32000-1 7.3.10), and the font dictionary is the last one in that chain.
fn font_object(doc: &Document, mut r: ObjRef) -> ObjRef {
for _ in 0..MAX_REFERENCE_HOPS {
let Object::Ref(next) = &*doc.get(r) else { break };
r = *next;
}
r
}
/// Whether a stream's own dictionary says it holds content: a form XObject, or a stream with a
/// `/BBox` and no `/Subtype` (a tiling pattern, or an appearance stream that leaves out
/// `/Subtype /Form`). Mesh shadings may carry a `/BBox` too, but their data isn't content.
@@ -486,7 +586,48 @@ pub fn verify(doc: &Document, level: Level) -> Vec<Issue> {
let message = fill(pattern, None, &values);
push(Issue { clause, message, pattern, value: None, values, page: None, object: Some(object), fixable });
};
let cat = catalog(doc);
check_identification(doc, level, &mut issue);
let objs = objects(doc);
check_xmp_packets(&objs, &located_issue);
check_output_intent(doc, &objs, &mut issue);
check_catalog_actions(doc, &mut issue);
// Pages: annotations and page actions.
let pages = pdfcraft_model::pages(doc);
// Each annotation's page: the first page that lists it.
let mut annot_pages: std::collections::HashMap<ObjRef, usize> = std::collections::HashMap::new();
for (i, p) in pages.iter().enumerate() {
if let Some(a) = p.dict.get(b"Annots").map(|a| doc.resolve(a))
&& let Some(a) = a.as_array()
{
#[cfg(test)]
ANNOT_ENTRIES_READ.with(|n| n.set(n.get() + a.len()));
for r in a.iter().filter_map(Object::as_ref) {
annot_pages.entry(r).or_insert(i);
}
}
}
let page_of = |r: ObjRef| annot_pages.get(&r).copied();
for (i, p) in pages.iter().enumerate() {
if p.dict.get(b"AA").is_some() {
issue("6.5.2", "Page {} has additional actions", Some((i + 1).to_string()), Some(i), true);
}
}
check_transparency(doc, &pages, &mut issue);
let drawing = drawing_fonts(doc, &objs);
check_objects(doc, level, &objs, &drawing, &page_of, &mut issue);
check_limits(doc, &objs, &pages, &mut issue);
check_spots(doc, &objs, &mut issue);
check_graphics_states(doc, &mut issue);
// One message per kind of problem is enough for repeated ones (as preflight summaries do).
let mut out = out.into_inner();
let mut seen = std::collections::HashSet::new();
out.retain(|i| seen.insert((i.clause, i.message.clone(), i.page)));
out
}
/// 6.1.2, 6.1.3 and 6.6: identification and metadata, the file header, the file identifier and
/// encryption.
fn check_identification(doc: &Document, level: Level, issue: &mut impl FnMut(&'static str, &'static str, Option<String>, Option<usize>, bool)) {
// 6.1.3 / 6.6: identification and metadata.
match metadata(doc) {
None => issue("6.6.2.1", "The document has no XMP metadata", None, None, true),
@@ -503,16 +644,37 @@ pub fn verify(doc: &Document, level: Level) -> Vec<Issue> {
}
}
}
// 6.1.3: encryption.
// 6.1.2: the file header.
let header = header::check(doc);
if header.version {
issue("6.1.2", "The file doesn't start with a %PDF-1.0 to %PDF-1.7 header line", None, None, true);
}
if header.comment {
issue("6.1.2", "The file header isn't followed by a comment of four bytes above 127", None, None, true);
}
// 6.1.3: the file identifier and encryption.
match trailer::check(doc) {
Some(trailer::IdProblem::Missing) => issue("6.1.3", "The trailer has no file identifier (ID)", None, None, true),
Some(trailer::IdProblem::Malformed) => {
issue("6.1.3", "The file identifier (ID) in the trailer isn't two non-empty strings", None, None, true)
}
None => {}
}
if doc.trailer().get(b"Encrypt").is_some() {
issue("6.1.3", "The document is encrypted", None, None, true);
}
// 6.2.2: output intent when device colour is used.
let objs = objects(doc);
// 6.6.2.3.1: the value types of predefined XMP properties, in every metadata stream.
let mut mistyped = Vec::new();
xmp_types::for_each_packet(&objs, |object, packet| {
}
/// 6.6.2.3.1 and 6.6.2.3.3: predefined XMP value types and extension schema descriptions, in
/// every metadata stream.
fn check_xmp_packets(
objs: &[(ObjRef, std::sync::Arc<Object>)],
located_issue: &impl Fn(&'static str, &'static str, Vec<(&'static str, String)>, ObjRef, bool),
) {
let (mut mistyped, mut extension) = (Vec::new(), xmp_extension::Findings::default());
xmp_types::for_each_packet(objs, |object, packet| {
mistyped.extend(xmp_types::mistyped(packet).into_iter().map(|((name, written), ty)| (object, name, written, ty)));
extension.add(object, packet);
});
for (object, name, written, ty) in mistyped {
let mut values = vec![("{property}", name.clone()), ("{object}", object.to_string()), ("{type}", ty)];
@@ -535,7 +697,104 @@ pub fn verify(doc: &Document, level: Level) -> Vec<Issue> {
);
}
}
let used = uses_device_colour(doc, &objs);
report_xmp_extension(extension, located_issue);
}
/// 6.6.2.3.3: what the extension schema descriptions of the metadata streams lack.
fn report_xmp_extension(
extension: xmp_extension::Findings,
located_issue: &impl Fn(&'static str, &'static str, Vec<(&'static str, String)>, ObjRef, bool),
) {
for m in extension.missing {
let mut values = vec![("{schema}", m.schema), ("{object}", m.object.to_string()), ("{entry}", m.entry)];
match (m.item, m.empty) {
(None, false) => located_issue(
"6.6.2.3.3",
"The XMP extension schema {schema} in metadata stream {object} has no {entry} entry",
values,
m.object,
false,
),
(None, true) => located_issue(
"6.6.2.3.3",
"The XMP extension schema {schema} in metadata stream {object} has an empty {entry} entry",
values,
m.object,
false,
),
(Some(item), false) => {
values.push(("{item}", item));
located_issue(
"6.6.2.3.3",
"The XMP extension schema {schema} in metadata stream {object} describes {item} without a {entry} entry",
values,
m.object,
false,
);
}
(Some(item), true) => {
values.push(("{item}", item));
located_issue(
"6.6.2.3.3",
"The XMP extension schema {schema} in metadata stream {object} describes {item} with an empty {entry} entry",
values,
m.object,
false,
);
}
}
}
for c in extension.not_bag {
let object = c.object;
if c.found.is_empty() {
located_issue(
"6.6.2.3.3",
"The XMP extension schema container in metadata stream {object} holds no rdf:Bag",
vec![("{object}", object.to_string())],
object,
false,
);
} else {
let values = vec![("{object}", object.to_string()), ("{found}", c.found)];
located_issue(
"6.6.2.3.3",
"The XMP extension schema container in metadata stream {object} holds {found}, not one rdf:Bag",
values,
object,
false,
);
}
}
for p in extension.prefixes {
let mut values = vec![("{object}", p.object.to_string()), ("{namespace}", p.namespace.to_owned()), ("{prefix}", p.own.to_owned())];
if p.written.is_empty() {
located_issue(
"6.6.2.3.3",
"Metadata stream {object} writes the namespace {namespace} without a prefix, not with {prefix}",
values,
p.object,
false,
);
} else {
values.push(("{written}", p.written));
located_issue(
"6.6.2.3.3",
"Metadata stream {object} writes the namespace {namespace} with the prefix {written}, not {prefix}",
values,
p.object,
false,
);
}
}
}
/// 6.2.3 and 6.2.4.3: the output intent device colour needs.
fn check_output_intent(
doc: &Document,
objs: &[(ObjRef, std::sync::Arc<Object>)],
issue: &mut impl FnMut(&'static str, &'static str, Option<String>, Option<usize>, bool),
) {
let used = uses_device_colour(doc, objs);
match pdfa_intent_profile(doc) {
None if used.any() => {
issue("6.2.3", "Device colour is used, but there is no PDF/A output intent", None, None, !used.cmyk);
@@ -554,7 +813,11 @@ pub fn verify(doc: &Document, level: Level) -> Vec<Issue> {
}
None => {}
}
}
fn check_catalog_actions(doc: &Document, issue: &mut impl FnMut(&'static str, &'static str, Option<String>, Option<usize>, bool)) {
// 6.1.3 / 6.6.1: catalog actions.
let cat = catalog(doc);
if cat.get(b"AA").is_some() {
issue("6.5.2", "The document has additional actions (/AA)", None, None, true);
}
@@ -568,61 +831,137 @@ pub fn verify(doc: &Document, level: Level) -> Vec<Issue> {
{
issue("6.4.1", "The form asks viewers to make appearances (NeedAppearances)", None, None, true);
}
// Pages: annotations and page actions.
let pages = pdfcraft_model::pages(doc);
// Each annotation's page: the first page that lists it.
let mut annot_pages: std::collections::HashMap<ObjRef, usize> = std::collections::HashMap::new();
}
fn check_transparency(
doc: &Document,
pages: &[pdfcraft_model::Page],
issue: &mut impl FnMut(&'static str, &'static str, Option<String>, Option<usize>, bool),
) {
// 6.2.10: blend modes are those ISO 32000-1 defines, and without a PDF/A output intent (one with
// a profile) a page with transparency names a blending colour space.
let intent = pdfa_intent_has_profile(doc);
// Pages share resources, forms and fonts: what one page found under them, the next takes as is.
let mut shared = transparency::Shared::default();
for (i, p) in pages.iter().enumerate() {
if let Some(a) = p.dict.get(b"Annots").map(|a| doc.resolve(a))
&& let Some(a) = a.as_array()
{
#[cfg(test)]
ANNOT_ENTRIES_READ.with(|n| n.set(n.get() + a.len()));
for r in a.iter().filter_map(Object::as_ref) {
annot_pages.entry(r).or_insert(i);
}
let painting = transparency::page_painting(doc, &p.dict, &mut shared);
for name in painting.unknown_blend_modes {
issue("6.2.10", "A graphics state uses the blend mode {}, which ISO 32000-1 doesn't define", Some(name), Some(i), false);
}
if painting.transparency && !intent && !transparency::page_names_blending_space(doc, &p.dict) {
let n = Some((i + 1).to_string());
issue("6.2.10", "Page {} has transparency but no device-independent blending colour space or PDF/A output intent", n, Some(i), false);
}
}
let page_of = |r: ObjRef| annot_pages.get(&r).copied();
}
fn check_limits(
doc: &Document,
objs: &[(ObjRef, std::sync::Arc<Object>)],
pages: &[pdfcraft_model::Page],
issue: &mut impl FnMut(&'static str, &'static str, Option<String>, Option<usize>, bool),
) {
// 6.1.13: the implementation limits of ISO 32000-1 Annex C.
let over = limits::exceeded(doc, objs, &referenced_content(doc, objs));
if over.int {
issue("6.1.13", "An integer is outside the 32-bit range", None, None, false);
}
if over.real {
issue("6.1.13", "A real number is outside ±3.403 × 10³⁸, or nonzero but within ±1.175 × 10⁻³⁸", None, None, false);
}
if over.string {
issue("6.1.13", "A string is longer than 32767 bytes", None, None, false);
}
if limits::cid_over_limit(doc, objs) {
issue("6.1.13", "A CMap maps a character code to a CID above 65535", None, None, false);
}
if over.name {
issue("6.1.13", "A name is longer than 127 bytes", None, None, false);
}
for (i, p) in pages.iter().enumerate() {
if p.dict.get(b"AA").is_some() {
issue("6.5.2", "Page {} has additional actions", Some((i + 1).to_string()), Some(i), true);
if limits::page_box_out_of_range(doc, &p.dict) {
issue("6.1.13", "Page {} has a boundary smaller than 3 or larger than 14400 units", Some((i + 1).to_string()), Some(i), false);
}
}
check_objects(doc, level, &objs, &page_of, &mut issue);
// One message per kind of problem is enough for repeated ones (as preflight summaries do).
let mut out = out.into_inner();
let mut seen = std::collections::HashSet::new();
out.retain(|i| seen.insert((i.clause, i.message.clone(), i.page)));
out
}
fn check_spots(
doc: &Document,
objs: &[(ObjRef, std::sync::Arc<Object>)],
issue: &mut impl FnMut(&'static str, &'static str, Option<String>, Option<usize>, bool),
) {
// 6.2.4.4: Separation and DeviceN colour spaces.
let spots = spot::check(doc, objs);
for name in spots.differing {
issue("6.2.4.4", "Separation colour spaces named {} differ in their alternate space or tint transform", Some(name), None, false);
}
for name in spots.undefined {
issue("6.2.4.4", "The spot colour {} of a DeviceN colour space isn't in its Colorants dictionary", Some(name), None, false);
}
}
fn check_graphics_states(doc: &Document, issue: &mut impl FnMut(&'static str, &'static str, Option<String>, Option<usize>, bool)) {
// 6.2.5: the extended graphics state.
let gs = gstate::check(doc);
if gs.transfer {
issue("6.2.5", "A graphics state has a transfer function (TR)", None, None, false);
}
if gs.transfer2 {
issue("6.2.5", "A graphics state has a transfer function (TR2) other than Default", None, None, false);
}
if gs.phase {
issue("6.2.5", "A graphics state has a halftone phase (HTP)", None, None, false);
}
for kind in gs.halftone_types {
issue("6.2.5", "A halftone has type {}: only types 1 and 5 are allowed", Some(kind.to_string()), None, false);
}
if gs.untyped {
issue("6.2.5", "A halftone's HalftoneType is missing or isn't an integer", None, None, false);
}
if gs.halftone_name {
issue("6.2.5", "A halftone has a HalftoneName", None, None, false);
}
if gs.halftone_transfer {
issue("6.2.5", "A halftone has a transfer function, which only a non-primary colorant's halftone may have", None, None, false);
}
for colorant in gs.missing_transfer {
issue("6.2.5", "The halftone for the colorant {} has no transfer function", Some(colorant), None, false);
}
for intent in gs.intents {
issue("6.2.5", "A graphics state has the rendering intent {}, which isn't a standard one", Some(intent), None, false);
}
if gs.intent_not_name {
issue("6.2.5", "A graphics state has a rendering intent that isn't a name", None, None, false);
}
}
fn check_objects(
doc: &Document,
level: Level,
objects: &[(ObjRef, std::sync::Arc<Object>)],
drawing: &std::collections::HashSet<ObjRef>,
page_of: &impl Fn(ObjRef) -> Option<usize>,
issue: &mut impl FnMut(&'static str, &'static str, Option<String>, Option<usize>, bool),
) {
for (r, o) in objects {
let Some(d) = dict_of(o) else { continue };
check_one_object(doc, level, *r, d, o, page_of, issue);
check_one_object(doc, level, d, o, drawing.contains(r), page_of(*r), issue);
}
}
fn check_one_object(
doc: &Document,
level: Level,
reference: ObjRef,
dict: &Dict,
object: &Object,
page_of: &impl Fn(ObjRef) -> Option<usize>,
draws: bool,
page: Option<usize>,
issue: &mut impl FnMut(&'static str, &'static str, Option<String>, Option<usize>, bool),
) {
let ty = dict.name(b"Type");
let subtype = dict.name(b"Subtype");
check_actions(doc, dict, issue);
if is_annotation(dict, ty, subtype) && check_annotation(doc, dict, subtype, page_of(reference), issue) {
if is_annotation(dict, ty, subtype) && check_annotation(doc, dict, subtype, page, issue) {
// A forbidden annotation stops every later check for its object, as the scan did before
// it was split into helpers: the rest of its own checks and the form field, font, stream
// and embedded file below all came after that `continue`.
@@ -631,7 +970,8 @@ fn check_one_object(
if dict.get(b"FT").is_some() && dict.get(b"AA").is_some() && ty != Some(b"Annot") {
issue("6.5.2", "A form field has additional actions", None, None, true);
}
if ty == Some(b"Font") || dict.get(b"BaseFont").is_some() && subtype.is_some() {
// A Type 0 font's descendant is checked through it.
if draws && (ty == Some(b"Font") || dict.get(b"BaseFont").is_some() && subtype.is_some()) {
check_font(doc, dict, issue);
}
if let Object::Stream(stream) = object {
@@ -829,10 +1169,8 @@ fn walk_chain(doc: &Document, a: &Dict, read: &mut usize, seen: &mut std::collec
if *read > MAX_ACTION_CHAIN {
return Some("too deeply chained".into());
}
if let Some(s) = d.name(b"S")
&& FORBIDDEN_ACTIONS.contains(&s)
{
return Some(String::from_utf8_lossy(s).into_owned());
if let Some(s) = action_forbidden(doc, d) {
return Some(s);
}
match d.get(b"Next") {
Some(Object::Dict(next)) => stack.push(next),
@@ -849,6 +1187,24 @@ fn walk_chain(doc: &Document, a: &Dict, read: &mut usize, seen: &mut std::collec
None
}
/// The forbidden kind of the action `d` itself, if it is one: a forbidden type, or a Named
/// action for anything but the four of [`NAMED_ACTIONS`]. The type and the name may each be given
/// directly or by reference.
fn action_forbidden(doc: &Document, d: &Dict) -> Option<String> {
let s = doc.resolve(d.get(b"S")?);
let s = s.as_name()?;
if FORBIDDEN_ACTIONS.contains(&s) {
return Some(String::from_utf8_lossy(s).into_owned());
}
let n = d.get(b"N").map(|n| doc.resolve(n));
match n.as_deref().and_then(Object::as_name) {
_ if s != b"Named" => None,
Some(n) if NAMED_ACTIONS.contains(&n) => None,
Some(n) => Some(format!("Named {}", String::from_utf8_lossy(n))),
None => Some("Named".into()),
}
}
/// Whether `o` is (or refers to) a forbidden action or one chaining to it, or to nothing.
fn forbidden(doc: &Document, o: &Object) -> bool {
match &*doc.resolve(o) {
@@ -874,6 +1230,9 @@ pub fn convert(doc: &mut Document, level: Level) -> Result<Report, pdfcraft_cos:
doc.remove_encryption();
fixed.push("Removed encryption".into());
}
// A full save writes the header afresh, with the version it keeps and a binary comment.
fixed.extend(header::rewrite(doc));
fixed.extend(trailer::fix(doc));
let root = doc.root().ok_or(pdfcraft_cos::CosError::Syntax { offset: 0, detail: "no catalog".into() })?;
// Catalog: actions, JavaScript, NeedAppearances, output intent, metadata.
let mut cat = catalog(doc);
crates/preflight/src/limits.rs
+227
View File
@@ -0,0 +1,227 @@
//! PDF/A-2 6.1.13: a file stays within the implementation limits of ISO 32000-1 Annex C (Table C.1
//! and the page size limits after it): integers that fit in 32 bits, strings of at most 32767
//! bytes, names of at most 127 bytes, and page boundaries between 3 and 14400 units wide and high.
//!
//! The values are read from the trailer, the indirect objects and the content streams, as parsed:
//! a name written with `#xx` escapes counts its decoded bytes. A page's `/Contents` streams are read
//! joined, as one content stream (an operator's operands may end one stream and the operator start
//! the next, ISO 32000-1 7.8.2). Content is decoded within one allowance for the whole document.
//! Real numbers beyond ±3.403 × 10³⁸ are reported, and so are nonzero ones nearer zero than
//! ±1.175 × 10⁻³⁸ (ISO 32000-1 Table C.1 lets a reader round them to zero; PDF/A-2 does not allow
//! them, PDF Association TechNote 0010 A018), and so is an embedded CMap of a Type 0 font
//! (or one it uses through `/UseCMap`) that maps a code to a CID above 65535. An integer token too
//! long even for a real reads as one past that range, and is reported as a real.
use std::collections::HashSet;
use pdfcraft_cos::{DecodeBudget, Dict, Document, ObjRef, Object};
/// Table C.1: the smallest and largest integer values.
const MIN_INT: i64 = i32::MIN as i64;
const MAX_INT: i64 = i32::MAX as i64;
/// Table C.1: the largest real value (approximate).
const MAX_REAL: f64 = 3.403e38;
/// Table C.1: the nonzero real value closest to zero (approximate).
const MIN_REAL: f64 = 1.175e-38;
/// Table C.1: the largest character identifier.
const MAX_CID: u64 = 65535;
/// The decoded CMap data, across all the document's embedded CMaps, the scan reads.
const MAX_CMAP_BYTES: usize = 16 << 20;
/// How many CMaps a `/UseCMap` chain is followed through.
const MAX_CMAP_CHAIN: usize = 8;
/// Table C.1: the longest string, in bytes.
const MAX_STRING: usize = 32767;
/// Table C.1: the longest name, in bytes.
const MAX_NAME: usize = 127;
/// The smallest and largest page boundary, in default user space units.
const MIN_PAGE: f64 = 3.0;
const MAX_PAGE: f64 = 14400.0;
/// How deep a direct object is followed (arrays and dictionaries inside one another): as deep as
/// the parser nests them.
const MAX_DEPTH: u8 = 128;
/// The decoded content, across all the document's content streams, the scan reads.
const MAX_CONTENT_BYTES: usize = 256 << 20;
/// The page boundaries, of which only the media and crop boxes are inherited.
const BOXES: [&[u8]; 5] = [b"MediaBox", b"CropBox", b"BleedBox", b"TrimBox", b"ArtBox"];
/// Which limits a value went past.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub(crate) struct Exceeded {
pub(crate) int: bool,
pub(crate) real: bool,
pub(crate) string: bool,
pub(crate) name: bool,
}
impl Exceeded {
fn all(self) -> bool {
self.int && self.real && self.string && self.name
}
fn name(&mut self, n: &[u8]) {
self.name |= n.len() > MAX_NAME;
}
/// One value and, within [`MAX_DEPTH`], what it holds: a dictionary's keys count as names.
fn value(&mut self, o: &Object, depth: u8) {
if self.all() || depth > MAX_DEPTH {
return;
}
match o {
Object::Int(i) => self.int |= !(MIN_INT..=MAX_INT).contains(i),
Object::Real(r) => self.real |= !r.is_finite() || r.abs() > MAX_REAL || (*r != 0.0 && r.abs() < MIN_REAL),
Object::String(s) => self.string |= s.bytes.len() > MAX_STRING,
Object::Name(n) => self.name(n),
Object::Array(a) => {
for v in a {
self.value(v, depth.saturating_add(1));
}
}
Object::Dict(d) => self.dict(d, depth),
Object::Stream(s) => self.dict(&s.dict, depth),
Object::Null | Object::Bool(_) | Object::Ref(_) => {}
}
}
/// The operands of one content stream, inline images' dictionaries included.
fn content(&mut self, data: &[u8]) {
for op in pdfcraft_content::ops(data) {
if self.all() {
return;
}
for v in &op.operands {
self.value(v, 0);
}
if let Some((image, _)) = &op.inline {
self.dict(image, 0);
}
}
}
fn dict(&mut self, d: &Dict, depth: u8) {
for (k, v) in d.iter() {
self.name(k);
self.value(v, depth.saturating_add(1));
}
}
}
/// The limits that the trailer, the indirect objects `objs` and the content streams go past.
/// `content` holds the content streams only a reference identifies (a page's `/Contents`, Type 3
/// glyph procedures), besides form XObjects and appearance streams, which say so themselves.
pub(crate) fn exceeded(doc: &Document, objs: &[(ObjRef, std::sync::Arc<Object>)], content: &HashSet<ObjRef>) -> Exceeded {
let mut found = Exceeded::default();
found.dict(doc.trailer(), 0);
for (_, o) in objs {
if found.all() {
return found;
}
found.value(o, 0);
}
let mut budget = DecodeBudget::new(MAX_CONTENT_BYTES, 0);
let mut read: HashSet<ObjRef> = HashSet::new();
for page in pdfcraft_model::pages(doc) {
if found.all() || budget.remaining() == 0 {
return found;
}
let refs = contents(doc, &page.dict);
let mut joined = Vec::new();
for r in &refs {
if let Object::Stream(s) = &*doc.get(*r)
&& let Ok(data) = budget.decode(s)
{
joined.extend_from_slice(&data);
joined.push(b' ');
}
}
read.extend(refs);
found.content(&joined);
}
for (r, o) in objs {
if found.all() || budget.remaining() == 0 {
break;
}
if let Object::Stream(s) = &**o
&& !read.contains(r)
&& (content.contains(r) || crate::is_content(&s.dict))
&& let Ok(data) = budget.decode(s)
{
found.content(&data);
}
}
found
}
/// A page's `/Contents` streams, in order.
fn contents(doc: &Document, page: &Dict) -> Vec<ObjRef> {
match page.get(b"Contents") {
Some(Object::Ref(r)) => match &*doc.get(*r) {
Object::Array(a) => a.iter().filter_map(Object::as_ref).collect(),
_ => vec![*r],
},
Some(Object::Array(a)) => a.iter().filter_map(Object::as_ref).collect(),
_ => Vec::new(),
}
}
/// Whether one of a page's boundaries (with the media and crop boxes it inherits) is narrower or
/// lower than 3 units, or wider or higher than 14400.
pub(crate) fn page_box_out_of_range(doc: &Document, page: &Dict) -> bool {
BOXES.iter().filter_map(|k| page.get(k)).any(|b| {
let b = doc.resolve(b);
let v: Vec<f64> = b.as_array().map(|a| a.iter().filter_map(|x| doc.resolve(x).as_f64()).collect()).unwrap_or_default();
let [x0, y0, x1, y1] = v.as_slice() else { return false };
let (w, h) = ((x1 - x0).abs(), (y1 - y0).abs());
// A size past `f64` (corners near ±1e308) is as much too large as any other.
[w, h].iter().any(|s| !(MIN_PAGE..=MAX_PAGE).contains(s))
})
}
/// Whether an embedded CMap of a Type 0 font among `objs`, or one it uses through `/UseCMap`, maps
/// a code to a CID above 65535: a `begincidrange` entry `<low> <high> cid` reaches
/// `cid + high − low`, a `begincidchar` entry `<code> cid` reaches `cid` (ISO 32000-1 9.7.5.4). A
/// CMap named rather than embedded is a predefined one, within the limit. The CMaps are decoded
/// within one allowance for the document.
pub(crate) fn cid_over_limit(doc: &Document, objs: &[(ObjRef, std::sync::Arc<Object>)]) -> bool {
let mut budget = DecodeBudget::new(MAX_CMAP_BYTES, 0);
let mut read: HashSet<ObjRef> = HashSet::new();
for (_, o) in objs {
let Some(d) = o.as_dict() else { continue };
if d.name(b"Subtype") != Some(b"Type0") {
continue;
}
let mut next = d.get(b"Encoding").and_then(Object::as_ref);
for _ in 0..MAX_CMAP_CHAIN {
let Some(r) = next.filter(|r| read.insert(*r)) else { break };
if budget.remaining() == 0 {
return false;
}
let Object::Stream(cmap) = &*doc.get(r) else { break };
if budget.decode(cmap).is_ok_and(|data| maps_past_limit(&data)) {
return true;
}
next = cmap.dict.get(b"UseCMap").and_then(Object::as_ref);
}
}
false
}
/// Whether the CMap program `data` maps a code to a CID above [`MAX_CID`]. The entries of a
/// `begincidrange` … `endcidrange` block are the operands of its `endcidrange`, read as content is.
fn maps_past_limit(data: &[u8]) -> bool {
let value = |o: &Object| -> Option<u64> {
match o {
Object::String(s) if s.bytes.len() <= 4 => Some(s.bytes.iter().fold(0u64, |v, b| (v << 8) | u64::from(*b))),
Object::Int(i) => u64::try_from(*i).ok(),
_ => None,
}
};
pdfcraft_content::ops(data).any(|op| match op.op.as_slice() {
b"endcidrange" => op.operands.as_chunks::<3>().0.iter().any(|[low, high, cid]| match (value(low), value(high), value(cid)) {
(Some(low), Some(high), Some(cid)) => cid.saturating_add(high.saturating_sub(low)) > MAX_CID,
_ => false,
}),
b"endcidchar" => op.operands.as_chunks::<2>().0.iter().any(|[_, cid]| value(cid).is_some_and(|cid| cid > MAX_CID)),
_ => false,
})
}
crates/preflight/src/spot.rs
+212
View File
@@ -0,0 +1,212 @@
//! PDF/A-2 6.2.4.4, Separation and DeviceN colour spaces:
//!
//! - Separation spaces of the same colorant name agree on their alternate space and tint
//! transform, also where a DeviceN space's `/Colorants` dictionary holds them. They are compared
//! as objects, whether stored directly or by reference, at any depth (ISO 32000-1 7.3.10);
//! - a DeviceN space names each spot colorant it uses in the `/Colorants` dictionary of its
//! attributes (ISO 32000-1 8.6.6.5, Table 71); a null or missing entry defines nothing. The
//! process colorants (`Cyan`, `Magenta`, `Yellow`, `Black`, and those an NChannel space's
//! `/Process` lists) and `None` are not spot colorants.
//!
//! Colour spaces are found wherever they are written: in resource dictionaries, images, shadings
//! and other colour spaces, direct or by reference.
use std::collections::{BTreeSet, HashMap, HashSet};
use pdfcraft_cos::{DecodeBudget, Dict, Document, ObjRef, Object, Stream};
/// How deep a direct object is followed (arrays and dictionaries inside one another): as deep as
/// the parser nests them.
const MAX_DEPTH: u8 = 128;
/// How deep two definitions are compared, references followed.
const MAX_COMPARE_DEPTH: u8 = 32;
/// How many object pairs the comparisons of one document visit in all. Past it, a comparison
/// claims no difference.
const MAX_COMPARE_STEPS: u32 = 1 << 20;
/// The decoded bytes the stream comparisons of one document may spend in all.
const MAX_COMPARE_BYTES: usize = 64 << 20;
/// The stream entries that say how the data is stored, not what it is.
const ENCODING: [&[u8]; 3] = [b"Length", b"Filter", b"DecodeParms"];
/// The colorants of DeviceCMYK, which a DeviceN space may name without defining them.
const PROCESS: [&[u8]; 4] = [b"Cyan", b"Magenta", b"Yellow", b"Black"];
/// What the colour spaces of a document break of 6.2.4.4.
#[derive(Debug, Default, PartialEq, Eq)]
pub(crate) struct Findings {
/// Colorant names whose Separation spaces differ, in name order.
pub(crate) differing: BTreeSet<String>,
/// Spot colorants a DeviceN space uses without a `/Colorants` entry, in name order.
pub(crate) undefined: BTreeSet<String>,
}
/// The 6.2.4.4 findings over the indirect objects `objs`.
pub(crate) fn check(doc: &Document, objs: &[(ObjRef, std::sync::Arc<Object>)]) -> Findings {
let compare = Compare { doc, equal: HashSet::new(), steps: MAX_COMPARE_STEPS, budget: DecodeBudget::new(MAX_COMPARE_BYTES, 0) };
let mut scan = Scan { doc, compare, separations: HashMap::new(), findings: Findings::default() };
for (_, o) in objs {
scan.walk(o, 0);
}
scan.findings
}
struct Scan<'a> {
doc: &'a Document,
compare: Compare<'a>,
/// Each colorant name's first Separation space: its alternate space and tint transform.
separations: HashMap<Vec<u8>, (Object, Object)>,
findings: Findings,
}
impl Scan<'_> {
fn walk(&mut self, o: &Object, depth: u8) {
if depth > MAX_DEPTH {
return;
}
match o {
Object::Array(a) => {
match a.first().and_then(Object::as_name) {
Some(b"Separation") => self.separation(a),
Some(b"DeviceN") => self.devicen(a),
_ => {}
}
for v in a {
self.walk(v, depth.saturating_add(1));
}
}
Object::Dict(d) => {
for (_, v) in d.iter() {
self.walk(v, depth.saturating_add(1));
}
}
Object::Stream(s) => {
for (_, v) in s.dict.iter() {
self.walk(v, depth.saturating_add(1));
}
}
_ => {}
}
}
/// `[/Separation name alternateSpace tintTransform]`: the same name must keep the same
/// alternate space and tint transform.
fn separation(&mut self, a: &[Object]) {
let [_, name, alternate, tint, ..] = a else { return };
let name = self.doc.resolve(name);
let Some(name) = name.as_name() else { return };
let this = (alternate.clone(), tint.clone());
match self.separations.get(name) {
Some((alt, fun)) => {
if !self.compare.same(alt, &this.0, 0) || !self.compare.same(fun, &this.1, 0) {
self.findings.differing.insert(String::from_utf8_lossy(name).into_owned());
}
}
None => {
self.separations.insert(name.to_vec(), this);
}
}
}
/// `[/DeviceN names alternateSpace tintTransform attributes]`: each spot colorant in `names`
/// needs an entry in the attributes' `/Colorants`.
fn devicen(&mut self, a: &[Object]) {
let Some(names) = a.get(1).map(|n| self.doc.resolve(n)) else { return };
let Some(names) = names.as_array() else { return };
let attributes = a.get(4).map(|d| self.doc.resolve(d));
let attributes = attributes.as_deref().and_then(Object::as_dict);
let colorants = attributes.and_then(|d| d.get(b"Colorants")).map(|c| self.doc.resolve(c));
let colorants = colorants.as_deref().and_then(Object::as_dict);
let nchannel = attributes.and_then(|d| d.get(b"Subtype")).map(|t| self.doc.resolve(t)).is_some_and(|t| t.as_name() == Some(b"NChannel"));
let process = attributes
.filter(|_| nchannel)
.and_then(|d| d.get(b"Process"))
.map(|p| self.doc.resolve(p))
.and_then(|p| p.as_dict().and_then(|p| p.get(b"Components")).map(|c| self.doc.resolve(c)));
let process: Vec<Vec<u8>> = process
.as_deref()
.and_then(Object::as_array)
.map(|c| c.iter().filter_map(|n| self.doc.resolve(n).as_name().map(<[u8]>::to_vec)).collect())
.unwrap_or_default();
for n in names.iter().map(|n| self.doc.resolve(n)) {
let Some(n) = n.as_name() else { continue };
let spot = n != b"None" && !PROCESS.contains(&n) && !process.iter().any(|p| p == n);
// Each entry is a Separation space for the colorant it is keyed by (ISO 32000-1 8.6.6.5,
// Table 71); a null entry, or one referring to no object, defines nothing (7.3.9).
let defined = colorants.and_then(|c| c.get(n)).is_some_and(|e| self.separation_named(e, n));
if spot && !defined {
self.findings.undefined.insert(String::from_utf8_lossy(n).into_owned());
}
}
}
/// Whether `o` is `[/Separation name ...]` for the colorant `name`.
fn separation_named(&self, o: &Object, name: &[u8]) -> bool {
let o = self.doc.resolve(o);
let Some([kind, n, ..]) = o.as_array().map(Vec::as_slice) else { return false };
kind.as_name() == Some(b"Separation") && self.doc.resolve(n).as_name() == Some(name)
}
}
/// Compares definitions, remembering which reference pairs it found equal, so a structure shared
/// by many definitions is compared once, within [`MAX_COMPARE_STEPS`] and [`MAX_COMPARE_BYTES`].
struct Compare<'a> {
doc: &'a Document,
/// Reference pairs compared equal. A depth cut-off only ever answers "different", so an
/// equal answer holds at any depth.
equal: HashSet<(ObjRef, ObjRef)>,
steps: u32,
budget: DecodeBudget,
}
impl Compare<'_> {
/// Whether `a` and `b` are the same object, wherever their parts are stored: references are
/// followed (to [`MAX_COMPARE_DEPTH`]), so two identical streams under different object
/// numbers are the same, and streams compare by their decoded data. Once the steps or the
/// decode allowance run out, nothing more is called different.
fn same(&mut self, a: &Object, b: &Object, depth: u8) -> bool {
let pair = match (a, b) {
(Object::Ref(x), Object::Ref(y)) if x == y || self.equal.contains(&(*x, *y)) => return true,
(Object::Ref(x), Object::Ref(y)) => Some((*x, *y)),
_ => None,
};
let Some(steps) = self.steps.checked_sub(1) else { return true };
self.steps = steps;
if depth > MAX_COMPARE_DEPTH {
return false;
}
let (a, b) = (self.doc.resolve(a), self.doc.resolve(b));
let deeper = depth.saturating_add(1);
let same = match (&*a, &*b) {
(Object::Array(x), Object::Array(y)) => x.len() == y.len() && x.iter().zip(y).all(|(p, q)| self.same(p, q, deeper)),
(Object::Dict(x), Object::Dict(y)) => self.same_dict(x, y, &[], deeper),
(Object::Stream(x), Object::Stream(y)) => self.same_data(x, y, deeper) && self.same_dict(&x.dict, &y.dict, &ENCODING, deeper),
(x, y) => x == y,
};
if same && let Some(pair) = pair {
self.equal.insert(pair);
}
same
}
/// Whether two dictionaries hold the same entries, those keyed in `skip` aside.
fn same_dict(&mut self, a: &Dict, b: &Dict, skip: &[&[u8]], depth: u8) -> bool {
let kept = |d: &Dict| d.iter().filter(|(k, _)| !skip.contains(&k.as_slice())).count();
kept(a) == kept(b) && a.iter().filter(|(k, _)| !skip.contains(&k.as_slice())).all(|(k, v)| b.get(k).is_some_and(|w| self.same(v, w, depth)))
}
/// Whether two streams hold the same data, however each is encoded. Data that can't be
/// decoded within the allowance is not called different.
fn same_data(&mut self, a: &Stream, b: &Stream, depth: u8) -> bool {
let stored_alike = |c: &mut Self, k: &[u8]| match (a.dict.get(k), b.dict.get(k)) {
(None, None) => true,
(Some(x), Some(y)) => c.same(x, y, depth),
_ => false,
};
if a.raw == b.raw && stored_alike(self, b"Filter") && stored_alike(self, b"DecodeParms") {
return true;
}
match (self.budget.decode(a), self.budget.decode(b)) {
(Ok(x), Ok(y)) => x == y,
_ => true,
}
}
}
crates/preflight/src/tests.rs
+1331 -2
View File
File diff suppressed because it is too large. Load diff
crates/preflight/src/trailer.rs
+55
View File
@@ -0,0 +1,55 @@
//! PDF/A-2 6.1.3, the file trailer: it has an `/ID`, a file identifier as ISO 32000-1 14.4
//! defines it, an array of two byte strings.
//!
//! The trailer read is the one the file's last `startxref` points to: the latest update's, or a
//! linearized file's first-page trailer (ISO 32000-1 F.3.4). Trailers earlier in the chain don't
//! stand in for it.
use pdfcraft_cos::{Document, Object};
/// The longest string a conforming file may hold (ISO 32000-1 Annex C, Table C.1).
const MAX_STRING: usize = 32767;
/// What is wrong with the file identifier.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(crate) enum IdProblem {
/// The trailer has no `/ID`, or a null one.
Missing,
/// The `/ID` isn't an array of two non-empty strings.
Malformed,
}
/// What 6.1.3 finds wrong with the file identifier of `doc`, if anything.
pub(crate) fn check(doc: &Document) -> Option<IdProblem> {
let id = doc.trailer().get(b"ID").map(|id| doc.resolve(id)).filter(|id| !matches!(**id, Object::Null));
let Some(id) = id else { return Some(IdProblem::Missing) };
let strings = |parts: &[Object]| parts.iter().all(|p| doc.resolve(p).as_string().is_some_and(|s| !s.bytes.is_empty()));
match id.as_array() {
Some(parts) if parts.len() == 2 && strings(parts) => None,
_ => Some(IdProblem::Malformed),
}
}
/// Gives `doc` a file identifier when it lacks a good one; what it fixed, if anything. A first
/// string a conforming file can keep (non-empty, within [`MAX_STRING`]) is the permanent
/// identifier (ISO 32000-1 14.4): it is kept, and only the second is made afresh. Otherwise both
/// are new, and the report says the identifier was added rather than completed.
pub(crate) fn fix(doc: &mut Document) -> Option<String> {
check(doc)?;
let Object::Array(mut id) = pdfcraft_cos::new_file_id(doc.bytes(), 0) else { return None };
let permanent = doc
.trailer()
.get(b"ID")
.map(|old| doc.resolve(old))
.and_then(|old| old.as_array().and_then(|parts| parts.first().map(|p| doc.resolve(p))))
.filter(|first| first.as_string().is_some_and(|s| (1..=MAX_STRING).contains(&s.bytes.len())));
let kept = match (permanent, id.first_mut()) {
(Some(permanent), Some(slot)) => {
*slot = (*permanent).clone();
true
}
_ => false,
};
doc.trailer_mut().set(b"ID".to_vec(), Object::Array(id));
Some(if kept { "Completed the file identifier (ID)" } else { "Added a file identifier (ID)" }.into())
}
crates/preflight/src/transparency.rs
+391
View File
@@ -0,0 +1,391 @@
//! PDF/A-2 6.2.10: a graphics state's blend mode is one ISO 32000-1 defines, and a page that
//! contains transparency, in a file without a PDF/A output intent, names its blending colour
//! space: its `/Group` dictionary has a `/CS` entry, which as any colour space in such a file is
//! not a device space without a default (6.2.4.3).
//!
//! A page contains transparency when something its resources reach can paint transparently (ISO
//! 32000-1 11.6.4.4 to 11.6.6): a graphics state with a constant alpha (`/CA`, `/ca`) below 1,
//! a blend mode other than `Normal` or `Compatible`, or a soft mask; an image with a soft mask
//! (`/SMask`, or `/SMaskInData` above 0); or a transparency group XObject. The resources looked
//! through are the page's own, those of the form XObjects, tiling patterns and Type 3 fonts they
//! hold, and those of the appearance streams of the page's annotations. Whether the content uses
//! a resource is not asked: a resource a page holds is one it may paint with. Blend modes are read
//! from the same graphics states, so private data (`/PieceInfo`) is never taken for one.
use std::collections::HashMap;
use std::sync::Arc;
use pdfcraft_cos::{Dict, Document, ObjRef, Object};
/// How many dictionaries one page's search reads, across its resources, XObjects, patterns, Type
/// 3 fonts and appearance streams. An object [`Shared`] holds counts as one read.
const MAX_VISITS: usize = 4096;
/// How deep resources nest (a form XObject in a form XObject, ...).
const MAX_DEPTH: usize = 32;
/// The resource categories that hold what a page paints with, read in this order.
const CATEGORIES: [&[u8]; 4] = [b"ExtGState", b"XObject", b"Pattern", b"Font"];
/// How many unknown blend mode names one search gathers: enough to name them in the report,
/// without a `/BM` array of a million names comparing each with all the others.
const MAX_BLEND_NAMES: usize = 32;
/// How many objects [`Shared`] keeps what was found under.
const MAX_SHARED: usize = 1 << 16;
/// What the graphics a page's resources reach paint with.
#[derive(Clone, Debug, Default)]
pub(crate) struct Painting {
/// Something paints transparently.
pub(crate) transparency: bool,
/// The blend modes graphics states name that ISO 32000-1 doesn't define.
pub(crate) unknown_blend_modes: Vec<String>,
}
impl Painting {
fn add(&mut self, other: &Painting) {
self.transparency |= other.transparency;
for name in &other.unknown_blend_modes {
self.add_blend_mode(name);
}
}
/// Whether no more unknown blend mode names are gathered.
fn blend_modes_full(&self) -> bool {
self.unknown_blend_modes.len() >= MAX_BLEND_NAMES
}
fn add_blend_mode(&mut self, name: &str) {
if !self.blend_modes_full() && !self.unknown_blend_modes.iter().any(|n| n == name) {
self.unknown_blend_modes.push(name.to_owned());
}
}
}
/// What was found under the indirect objects (resources, form XObjects, patterns, Type 3 fonts,
/// graphics states) that pages share, so the next page that names one takes it from here instead
/// of walking it again. Only an object read to the end goes in: none whose reading stopped below
/// it at the budget, at the depth limit, or at an object already read on that page (a cycle).
#[derive(Default)]
pub(crate) struct Shared(HashMap<ObjRef, Painting>);
/// An object read in place, or the object a reference names as the document holds it: either way
/// nothing is copied.
enum Held<'o> {
Direct(&'o Object),
Resolved(Arc<Object>),
}
impl std::ops::Deref for Held<'_> {
type Target = Object;
fn deref(&self) -> &Object {
match self {
Held::Direct(o) => o,
Held::Resolved(o) => o,
}
}
}
/// `o`, or the object it refers to, without copying either ([`Document::resolve`] copies a direct
/// object).
fn held<'o>(doc: &Document, o: &'o Object) -> Held<'o> {
match o {
Object::Ref(_) => Held::Resolved(doc.resolve(o)),
o => Held::Direct(o),
}
}
/// What the page `page` paints with: its resources, and the appearance streams of its
/// annotations. `shared` carries what earlier pages found under the objects they share.
pub(crate) fn page_painting(doc: &Document, page: &Dict, shared: &mut Shared) -> Painting {
walk_page(doc, page, shared).found
}
/// How many reads the search of `page` took, each object taken from `shared` counting as one.
#[cfg(test)]
pub(crate) fn page_reads(doc: &Document, page: &Dict, shared: &mut Shared) -> usize {
walk_page(doc, page, shared).visits
}
fn walk_page<'a>(doc: &'a Document, page: &Dict, shared: &'a mut Shared) -> Walk<'a> {
let mut walk = Walk { doc, shared, seen: HashMap::new(), visits: 0, cuts: 0, found: Painting::default() };
if let Some(r) = page.get(b"Resources") {
walk.visit(r, 0, Walk::resources);
}
let annots = page.get(b"Annots").map(|a| held(doc, a));
for a in annots.as_deref().and_then(Object::as_array).into_iter().flatten() {
let a = held(doc, a);
let Some(ap) = a.as_dict().and_then(|a| a.get(b"AP")).map(|ap| held(doc, ap)) else { continue };
let Some(ap) = ap.as_dict() else { continue };
// Each appearance (normal, rollover, down) is a stream, or a dictionary of streams by state.
for (_, v) in ap.iter() {
match &*held(doc, v) {
Object::Stream(_) => walk.visit(v, 0, Walk::xobject),
Object::Dict(states) => {
for (_, s) in states.iter() {
walk.visit(s, 0, Walk::xobject);
}
}
_ => {}
}
}
}
walk
}
/// Whether `page` names a blending colour space PDF/A accepts without an output intent: its
/// `/Group` has a `/CS` that is a blending colour space (ISO 32000-1 and 32000-2 11.3.4:
/// `CalGray`, `CalRGB`, or an `ICCBased` space of 1, 3 or 4 components whose profile describes
/// gray, RGB or CMYK; not Lab, Indexed, Pattern, Separation or DeviceN), and not a device colour
/// space unless the page's resources give that device space a default (`DefaultGray`,
/// `DefaultRGB`, `DefaultCMYK`) that is itself such a space with as many components (ISO
/// 32000-1 8.6.5.6), as PDF/A-2 6.2.4.3 asks of any device colour.
pub(crate) fn page_names_blending_space(doc: &Document, page: &Dict) -> bool {
let group = page.get(b"Group").map(|g| doc.resolve(g));
let Some(cs) = group.as_deref().and_then(Object::as_dict).and_then(|g| g.get(b"CS")).map(|c| doc.resolve(c)) else { return false };
let (default, components): (&[u8], i64) = match &*cs {
Object::Name(n) if n == b"DeviceGray" => (b"DefaultGray", 1),
Object::Name(n) if n == b"DeviceRGB" => (b"DefaultRGB", 3),
Object::Name(n) if n == b"DeviceCMYK" => (b"DefaultCMYK", 4),
Object::Array(a) => return independent_blending_space(doc, a).is_some(),
_ => return false,
};
let resources = page.get(b"Resources").map(|r| doc.resolve(r));
let spaces = resources.as_deref().and_then(Object::as_dict).and_then(|r| r.get(b"ColorSpace")).map(|c| doc.resolve(c));
let space = spaces.as_deref().and_then(Object::as_dict).and_then(|s| s.get(default)).map(|d| doc.resolve(d));
space.as_deref().and_then(Object::as_array).and_then(|a| independent_blending_space(doc, a)) == Some(components)
}
/// The number of components of the colour space array `a` when it is a device-independent
/// blending colour space: `CalGray` (1), `CalRGB` (3), or `ICCBased` with 1, 3 or 4 components
/// whose profile header, where it can be read (within the output intent's profile cap), says
/// gray, RGB or CMYK to match.
fn independent_blending_space(doc: &Document, a: &[Object]) -> Option<i64> {
let family = a.first().map(|f| doc.resolve(f));
// A CalGray or CalRGB space needs its dictionary, with the required `/WhitePoint` [X Y Z]: X
// and Z positive, Y 1 (ISO 32000-1 Tables 63 and 64).
let calibrated = || {
let params = a.get(1).map(|p| doc.resolve(p));
let white = params.as_deref().and_then(Object::as_dict).and_then(|d| d.get(b"WhitePoint")).map(|w| doc.resolve(w));
let values: Option<Vec<f64>> = white.as_deref().and_then(Object::as_array).and_then(|w| w.iter().map(|v| doc.resolve(v).as_f64()).collect());
matches!(values.as_deref(), Some(&[x, y, z]) if x > 0.0 && y == 1.0 && z > 0.0)
};
match family.as_deref().and_then(Object::as_name)? {
b"CalGray" => calibrated().then_some(1),
b"CalRGB" => calibrated().then_some(3),
b"ICCBased" => {
let profile = a.get(1).map(|p| doc.resolve(p));
let Some(Object::Stream(s)) = profile.as_deref() else { return None };
let components = s.dict.get(b"N").and_then(|n| doc.resolve(n).as_int())?;
// ICC.1 header: the data colour space is bytes 16–19.
let header: Option<i64> = match s.decoded_within(crate::MAX_ICC_PROFILE).ok().and_then(|data| data.get(16..20).map(<[u8]>::to_vec)) {
Some(space) => match space.as_slice() {
b"GRAY" => Some(1),
b"RGB " => Some(3),
b"CMYK" => Some(4),
_ => return None,
},
None => None,
};
(matches!(components, 1 | 3 | 4) && header.is_none_or(|h| h == components)).then_some(components)
}
_ => None,
}
}
struct Walk<'a> {
doc: &'a Document,
shared: &'a mut Shared,
/// The indirect objects already read on this page, each with the shallowest depth it was read
/// at: an object is read again only from a shallower place, so a cycle ends and a deep first
/// visit doesn't hide what a shallower one would read.
seen: HashMap<ObjRef, usize>,
visits: usize,
/// How many times reading stopped short (the budget, the depth limit, an object already read
/// on this page). An object with a stop below it doesn't go in [`Shared`]: what was found
/// under it may be incomplete.
cuts: usize,
found: Painting,
}
impl Walk<'_> {
/// Read `o`, reached `depth` levels down, with `read`: from [`Shared`] when an earlier page
/// read it to the end, else when it is within the budget and the depth, and not an indirect
/// object read before on this page from as shallow a place.
fn visit(&mut self, o: &Object, depth: usize, read: fn(&mut Self, &Object, usize)) {
if self.visits >= MAX_VISITS || depth > MAX_DEPTH {
self.cuts = self.cuts.saturating_add(1);
return;
}
let Object::Ref(r) = o else {
self.visits += 1;
read(self, o, depth);
return;
};
if let Some(found) = self.shared.0.get(r) {
self.visits += 1;
self.found.add(found);
return;
}
if self.seen.get(r).is_some_and(|&d| d <= depth) {
self.cuts = self.cuts.saturating_add(1);
return;
}
self.seen.insert(*r, depth);
self.visits += 1;
// Gather what this object holds on its own, to keep for the next page, then add it to what
// the page holds.
let outer = std::mem::take(&mut self.found);
let cuts = self.cuts;
read(self, o, depth);
let below = std::mem::replace(&mut self.found, outer);
self.found.add(&below);
if self.cuts == cuts && self.shared.0.len() < MAX_SHARED {
self.shared.0.insert(*r, below);
}
}
fn resources(&mut self, r: &Object, depth: usize) {
let doc = self.doc;
let res = held(doc, r);
let Some(res) = res.as_dict() else { return };
for category in CATEGORIES {
let entries = res.get(category).map(|d| held(doc, d));
let Some(entries) = entries.as_deref().and_then(Object::as_dict) else { continue };
for (_, v) in entries.iter() {
// Once the budget is spent nothing more is read, so nothing more is gathered either.
if self.visits >= MAX_VISITS {
self.cuts = self.cuts.saturating_add(1);
return;
}
match category {
b"ExtGState" => self.visit(v, depth, Self::ext_gstate),
b"XObject" => self.visit(v, depth, Self::xobject),
b"Pattern" => self.visit(v, depth, Self::pattern),
_ => self.visit(v, depth, Self::type3),
}
}
}
}
/// A graphics state (ISO 32000-1 Table 58): whether it paints transparently, and the blend
/// modes it names that ISO 32000-1 doesn't define.
fn ext_gstate(&mut self, g: &Object, _depth: usize) {
let doc = self.doc;
let g = held(doc, g);
let Some(g) = g.as_dict() else { return };
let below_one = |key: &[u8]| g.get(key).and_then(|v| held(doc, v).as_f64()).is_some_and(|a| a < 1.0);
let bm = g.get(b"BM").map(|b| held(doc, b));
let blend = bm.as_deref().is_some_and(|b| blends(doc, b));
// A soft mask is a dictionary; `/None`, null or anything else is none (ISO 32000-1 Table 58).
let mask = g.get(b"SMask").map(|m| held(doc, m)).is_some_and(|m| matches!(&*m, Object::Dict(_)));
self.found.transparency |= below_one(b"CA") || below_one(b"ca") || blend || mask;
if let Some(bm) = bm.as_deref() {
unknown_blend_modes(doc, bm, &mut self.found);
}
}
/// An image with a soft mask, or a form XObject that is a transparency group or holds
/// transparency in its resources.
fn xobject(&mut self, x: &Object, depth: usize) {
let doc = self.doc;
let x = held(doc, x);
let Object::Stream(s) = &*x else { return };
let d = &s.dict;
if d.name(b"Subtype") == Some(b"Image") {
let soft_mask = d.get(b"SMask").is_some_and(|m| matches!(&*held(doc, m), Object::Stream(_)));
let in_data = d.get(b"SMaskInData").and_then(|v| held(doc, v).as_int()).is_some_and(|v| v > 0);
self.found.transparency |= soft_mask || in_data;
return;
}
let group = d.get(b"Group").map(|g| held(doc, g)).is_some_and(|g| g.as_dict().is_some_and(|g| g.name(b"S") == Some(b"Transparency")));
self.found.transparency |= group;
if let Some(r) = d.get(b"Resources") {
self.visit(r, depth + 1, Self::resources);
}
}
/// A tiling pattern's resources, or a shading pattern's graphics state.
fn pattern(&mut self, p: &Object, depth: usize) {
let p = held(self.doc, p);
let Some(d) = (match &*p {
Object::Stream(s) => Some(&s.dict),
Object::Dict(d) => Some(d),
_ => None,
}) else {
return;
};
if let Some(g) = d.get(b"ExtGState") {
self.visit(g, depth + 1, Self::ext_gstate);
}
if let Some(r) = d.get(b"Resources") {
self.visit(r, depth + 1, Self::resources);
}
}
/// A Type 3 font's resources, which its glyph procedures paint with.
fn type3(&mut self, f: &Object, depth: usize) {
let f = held(self.doc, f);
let Some(d) = f.as_dict() else { return };
if d.name(b"Subtype") != Some(b"Type3") {
return;
}
if let Some(r) = d.get(b"Resources") {
self.visit(r, depth + 1, Self::resources);
}
}
}
/// The blend modes of ISO 32000-1 11.3.5 (Tables 136 and 137), and `Compatible` (Table 58).
const BLEND_MODES: &[&[u8]] = &[
b"Normal",
b"Compatible",
b"Multiply",
b"Screen",
b"Overlay",
b"Darken",
b"Lighten",
b"ColorDodge",
b"ColorBurn",
b"HardLight",
b"SoftLight",
b"Difference",
b"Exclusion",
b"Hue",
b"Saturation",
b"Color",
b"Luminosity",
];
/// Add to `found` the names in a `/BM` value (a name, or an array of them) that are not blend
/// modes ISO 32000-1 defines, up to [`MAX_BLEND_NAMES`] of them.
fn unknown_blend_modes(doc: &Document, bm: &Object, found: &mut Painting) {
let one = |found: &mut Painting, n: &[u8]| {
if !BLEND_MODES.contains(&n) {
found.add_blend_mode(&String::from_utf8_lossy(n));
}
};
match bm {
Object::Name(n) => one(found, n),
Object::Array(a) => {
for m in a {
if found.blend_modes_full() {
return;
}
if let Some(n) = held(doc, m).as_name() {
one(found, n);
}
}
}
_ => {}
}
}
/// Whether a `/BM` value blends: a name other than `Normal` or `Compatible`, or an array whose
/// first name is not one of those.
fn blends(doc: &Document, b: &Object) -> bool {
let normal = |n: &[u8]| n == b"Normal" || n == b"Compatible";
match b {
Object::Name(n) => !normal(n),
Object::Array(a) => a.iter().find_map(|m| doc.resolve(m).as_name().map(|n| !normal(n))).unwrap_or(false),
_ => false,
}
}
crates/preflight/src/xmp_extension.rs
+251
View File
@@ -0,0 +1,251 @@
//! PDF/A-2 6.6.2.3.3, the extension schema container schema: a packet describes each schema it
//! extends XMP with in `pdfaExtension:schemas`, a bag of schema descriptions, and each description
//! is complete.
//!
//! The schemas, their entries and their prefixes are those of PDF Association TechNote 0009, "XMP
//! Extension Schemas in PDF/A-1" (2008), section 4, for PDF/A-2 6.6.2.3.3:
//! - a schema (`pdfaSchema`, 4.3) has `schema`, `namespaceURI` and `prefix`; its `property` and
//! `valueType` lists are optional. TechNote 0009, written for PDF/A-1, calls `schema` an
//! optional description; PDF/A-2 requires it, as its conformance tests show;
//! - a property (`pdfaProperty`, 4.4) has `name`, `valueType`, `category` and `description`;
//! - a value type (`pdfaType`, 4.5) has `type`, `namespaceURI`, `prefix`, `description` and
//! `field`. TechNote 0009 calls `field` optional; PDF/A-2 requires it, as its conformance tests
//! show;
//! - a field (`pdfaField`, 4.6) has `name`, `valueType` and `description`;
//! - each of the five namespaces is written with its own prefix (`pdfaExtension`, `pdfaSchema`,
//! `pdfaProperty`, `pdfaType`, `pdfaField`): "Unlike predefined XMP schemas, the namespace
//! prefixes are not only preferred, but required" (4.1).
//!
//! An entry counts as present written as an element or as an attribute of the description. An
//! entry that names something is empty when it holds no value: a namespace URI or prefix, a
//! property, value type or field name, a value type reference, a property category. TechNote 0009
//! asks for names that are valid XML names (4.4, 4.6) and a category that is internal or external
//! (4.4), and an empty prefix or namespace binds nothing. The human-readable descriptions
//! (`pdfaSchema:schema` and the `description` entries) are Text and may be empty.
use pdfcraft_cos::ObjRef;
use roxmltree::{Document, Node};
const RDF: &str = "http://www.w3.org/1999/02/22-rdf-syntax-ns#";
const EXTENSION: &str = "http://www.aiim.org/pdfa/ns/extension/";
const SCHEMA: &str = "http://www.aiim.org/pdfa/ns/schema#";
const PROPERTY: &str = "http://www.aiim.org/pdfa/ns/property#";
const TYPE: &str = "http://www.aiim.org/pdfa/ns/type#";
const FIELD: &str = "http://www.aiim.org/pdfa/ns/field#";
/// Each container namespace with its required prefix (TechNote 0009, 4.1).
const PREFIXES: [(&str, &str); 5] =
[(EXTENSION, "pdfaExtension"), (SCHEMA, "pdfaSchema"), (PROPERTY, "pdfaProperty"), (TYPE, "pdfaType"), (FIELD, "pdfaField")];
/// How many findings a document's packets are read for. A document past this many has plainly
/// failed the clause, and listing every one of a hostile packet's descriptions helps nobody.
const MAX_FINDINGS: usize = 1000;
/// How much of a name the packet gives (a namespace URI, a property) a finding repeats.
const MAX_SHOWN_CHARS: usize = 200;
/// What the extension schema descriptions of a document break of 6.6.2.3.3, in the order the
/// metadata streams and their descriptions come.
#[derive(Debug, Default, PartialEq, Eq)]
pub(crate) struct Findings {
/// Entries descriptions lack.
pub(crate) missing: Vec<Missing>,
/// Description namespaces written with another prefix, each once per stream and prefix.
pub(crate) prefixes: Vec<Prefix>,
/// `pdfaExtension:schemas` containers that aren't one `rdf:Bag` (TechNote 0009, 4.2: "bag
/// Schema").
pub(crate) not_bag: Vec<NotBag>,
}
/// A description without an entry it needs, or with the entry empty.
#[derive(Debug, PartialEq, Eq)]
pub(crate) struct Missing {
/// The metadata stream.
pub(crate) object: ObjRef,
/// The schema the description is in: its `pdfaSchema:namespaceURI`, or `#n` for the n-th
/// schema of its container when it has none.
pub(crate) schema: String,
/// The property, value type or field description, when it is one of those and not the schema
/// description itself: its `pdfaProperty:name` or `pdfaType:type`, a field as `type/name`, or
/// `#n` for the n-th of its list when it has no name.
pub(crate) item: Option<String>,
/// The entry it lacks or holds empty, as `prefix:name`.
pub(crate) entry: String,
/// Whether the entry is there but empty, rather than absent.
pub(crate) empty: bool,
}
/// A description namespace written with another prefix than its own.
#[derive(Debug, PartialEq, Eq)]
pub(crate) struct Prefix {
pub(crate) object: ObjRef,
pub(crate) namespace: &'static str,
/// The prefix it needs.
pub(crate) own: &'static str,
/// The prefix it is written with, empty when it is written as the default namespace.
pub(crate) written: String,
}
/// A `pdfaExtension:schemas` container that isn't one `rdf:Bag`.
#[derive(Debug, PartialEq, Eq)]
pub(crate) struct NotBag {
pub(crate) object: ObjRef,
/// The elements it holds instead, by their names as the packet writes them, comma-separated;
/// empty when it holds none.
pub(crate) found: String,
}
/// Where a description is: its stream, its schema and, below the schema, its own name.
struct Place {
object: ObjRef,
schema: String,
item: Option<String>,
}
impl Findings {
/// Adds what the packet `doc`, from the metadata stream `object`, breaks.
pub(crate) fn add(&mut self, object: ObjRef, doc: &Document<'_>) {
let containers = doc.descendants().filter(|n| is(n, EXTENSION, "schemas"));
for container in containers {
self.prefixes_in(object, doc, &container);
let children: Vec<Node<'_, '_>> = container.children().filter(Node::is_element).collect();
if !matches!(children.as_slice(), [bag] if is(bag, RDF, "Bag")) && self.room() {
let found = children.iter().take(MAX_SHOWN_CHARS).map(|c| written_name(doc, c)).collect::<Vec<_>>().join(", ");
self.not_bag.push(NotBag { object, found: shown(&found) });
}
for (n, schema) in items(&container).iter().enumerate() {
let id = value(schema, SCHEMA, "namespaceURI").unwrap_or_else(|| position(n));
let at = |item: Option<String>| Place { object, schema: id.clone(), item };
self.need(&at(None), schema, SCHEMA, "pdfaSchema", &["schema", "namespaceURI", "prefix"]);
let properties = entries(schema, SCHEMA, "property").flat_map(|e| items(&e));
for (n, property) in properties.enumerate() {
let name = value(&property, PROPERTY, "name").unwrap_or_else(|| position(n));
self.need(&at(Some(name)), &property, PROPERTY, "pdfaProperty", &["name", "valueType", "category", "description"]);
}
let value_types = entries(schema, SCHEMA, "valueType").flat_map(|e| items(&e));
for (n, value_type) in value_types.enumerate() {
let name = value(&value_type, TYPE, "type").unwrap_or_else(|| position(n));
self.need(&at(Some(name.clone())), &value_type, TYPE, "pdfaType", &["type", "namespaceURI", "prefix", "description", "field"]);
for (n, field) in entries(&value_type, TYPE, "field").flat_map(|e| items(&e)).enumerate() {
let field_name = value(&field, FIELD, "name").unwrap_or_else(|| position(n));
self.need(&at(Some(format!("{name}/{field_name}"))), &field, FIELD, "pdfaField", &["name", "valueType", "description"]);
}
}
}
}
}
/// Whether another finding may be recorded (see [`MAX_FINDINGS`]).
fn room(&self) -> bool {
self.missing.len().saturating_add(self.prefixes.len()).saturating_add(self.not_bag.len()) < MAX_FINDINGS
}
/// Notes the entries of `ns` in `names` that the description `d`, at `at`, lacks, or holds
/// empty when they are entries that name something (see [`NAMING`]).
fn need(&mut self, at: &Place, d: &Node<'_, '_>, ns: &str, prefix: &str, names: &[&str]) {
for name in names {
let element = entry(d, ns, name);
let attribute = d.attribute((ns, *name));
let empty = match (element, attribute) {
(None, None) => false,
(Some(e), _) => NAMING.contains(name) && is_empty(&e),
(None, Some(a)) => NAMING.contains(name) && a.trim().is_empty(),
};
if (empty || (element.is_none() && attribute.is_none())) && self.room() {
let Place { object, schema, item } = at;
let entry = format!("{prefix}:{name}");
self.missing.push(Missing { object: *object, schema: schema.clone(), item: item.clone(), entry, empty });
}
}
}
/// Notes container namespaces written in `container` (itself included) with another prefix
/// than their own.
fn prefixes_in(&mut self, object: ObjRef, doc: &Document<'_>, container: &Node<'_, '_>) {
let text = doc.input_text();
for n in container.descendants().filter(Node::is_element) {
let element = (n.tag_name().namespace(), text.get(n.range()).unwrap_or_default().strip_prefix('<').unwrap_or_default());
let attributes = n.attributes().map(|a| (a.namespace(), text.get(a.range_qname()).unwrap_or_default()));
for (ns, qname) in std::iter::once(element).chain(attributes) {
let Some(&(namespace, own)) = PREFIXES.iter().find(|(uri, _)| Some(*uri) == ns) else { continue };
let written = shown(written_prefix(qname));
if written != own
&& self.room()
&& !self.prefixes.iter().any(|p| p.object == object && p.namespace == namespace && p.written == written)
{
self.prefixes.push(Prefix { object, namespace, own, written });
}
}
}
}
}
/// The entries that name something, which an empty value leaves without a name: namespace URIs
/// and prefixes, property, value type and field names, value type references and the property
/// category.
const NAMING: [&str; 6] = ["namespaceURI", "prefix", "name", "type", "valueType", "category"];
/// Whether the element entry `e` holds no value: no element, no text but white space, and no
/// `rdf:resource` or `rdf:value` attribute with a value.
fn is_empty(e: &Node<'_, '_>) -> bool {
let blank = |v: Option<&str>| v.is_none_or(|v| v.trim().is_empty());
!e.children().any(|c| c.is_element() || (c.is_text() && c.text().is_some_and(|t| !t.trim().is_empty())))
&& blank(e.attribute((RDF, "resource")))
&& blank(e.attribute((RDF, "value")))
}
/// `#n` for the 0-based position `n`, counted from 1.
fn position(n: usize) -> String {
format!("#{}", n.saturating_add(1))
}
/// At most [`MAX_SHOWN_CHARS`] of `s`, cut at a char boundary, with `…` when cut.
fn shown(s: &str) -> String {
let mut out: String = s.chars().take(MAX_SHOWN_CHARS).collect();
if out.len() < s.len() {
out.push('…');
}
out
}
/// The text of the entry `name` of `ns` in the description `d`, written as an element or an
/// attribute, trimmed and [`shown`]; `None` when it is absent or blank.
fn value(d: &Node<'_, '_>, ns: &str, name: &str) -> Option<String> {
let text = entry(d, ns, name).and_then(|e| e.text()).or_else(|| d.attribute((ns, name)))?.trim();
(!text.is_empty()).then(|| shown(text))
}
/// The name of the element `n` as the packet writes it, prefix included.
fn written_name<'i>(doc: &Document<'i>, n: &Node<'_, '_>) -> &'i str {
qualified_name(doc.input_text().get(n.range()).unwrap_or_default().strip_prefix('<').unwrap_or_default())
}
/// The qualified name the text of a start tag (after its `<`) or of an attribute begins with.
fn qualified_name(text: &str) -> &str {
text.split(|c: char| c.is_whitespace() || c == '>' || c == '/' || c == '=').next().unwrap_or_default()
}
/// The prefix of the qualified name `text` begins with, empty when it has none.
fn written_prefix(text: &str) -> &str {
qualified_name(text).split_once(':').map_or("", |(p, _)| p)
}
fn is(n: &Node<'_, '_>, ns: &str, name: &str) -> bool {
n.is_element() && n.tag_name().namespace() == Some(ns) && n.tag_name().name() == name
}
/// Every element entry `name` of `ns` in the description `d`: a list written twice is read twice.
fn entries<'a, 'i>(d: &Node<'a, 'i>, ns: &'a str, name: &'a str) -> impl Iterator<Item = Node<'a, 'i>> {
d.children().filter(move |c| is(c, ns, name))
}
/// The element entry `name` of `ns` in the description `d`.
fn entry<'a, 'i>(d: &Node<'a, 'i>, ns: &str, name: &str) -> Option<Node<'a, 'i>> {
d.children().find(|c| is(c, ns, name))
}
/// The descriptions in the array `e` holds: each item's `rdf:parseType="Resource"` node or
/// `rdf:Description`.
fn items<'a, 'i>(e: &Node<'a, 'i>) -> Vec<Node<'a, 'i>> {
let Some(array) = e.children().find(|c| is(c, RDF, "Bag") || is(c, RDF, "Seq") || is(c, RDF, "Alt")) else { return Vec::new() };
array.children().filter(|i| is(i, RDF, "li")).map(|li| li.children().find(|c| is(c, RDF, "Description")).unwrap_or(li)).collect()
}
crates/preflight/src/xmp_types.rs
+158 -5
View File
@@ -4,7 +4,10 @@
//! The types come from ISO 16684-1:2011 (XMP Part 1) clause 8 for the Dublin Core, XMP Basic and
//! XMP Rights Management namespaces, and from XMP Specification Part 2 (Additional Properties) for
//! the Media Management, Basic Job Ticket, Paged-Text, Dynamic Media, Adobe PDF, Photoshop and
//! Camera Raw namespaces, with the structures they use. A value is read as RDF/XML serializes it
//! Camera Raw namespaces, with the structures they use. The TIFF, EXIF and Additional EXIF (`aux`)
//! namespaces, which the current Part 2 leaves to CIPA DC-010, come from the September 2005 XMP
//! Specification that PDF/A-2 cites: "EXIF Schemas" (pages 58–70) and "EXIF Schema Value Types"
//! (pages 83–87). A value is read as RDF/XML serializes it
//! (ISO 16684-1 7.9): a simple value as text or an attribute, an array as `rdf:Seq`, `rdf:Bag` or
//! `rdf:Alt`, a structure as `rdf:parseType="Resource"`, an `rdf:Description` or attributes, and a
//! qualified value through its `rdf:value` (element or attribute). Every metadata stream of the
@@ -12,7 +15,8 @@
//!
//! The simple types are checked by their lexical form (ISO 16684-1 8.2.1): Boolean is `True` or
//! `False`; Integer an optionally signed run of digits; Real a decimal; Date one of the six forms
//! of 8.2.1.2; Rational two integers around a `/`, as Part 2's `xmpDM:scale` example writes it.
//! of 8.2.1.2; Rational two integers around a `/`, as Part 2's `xmpDM:scale` example writes it and
//! the 2005 edition defines it (page 84); GPSCoordinate `DDD,MM,SSk` or `DDD,MM.mmk` (page 85).
//! The other simple types are text. A field a structure doesn't define, and a property a
//! namespace doesn't define, are left to the clauses about them.
@@ -43,6 +47,8 @@ enum Ty {
Boolean,
Date,
Rational,
/// `DDD,MM,SSk` or `DDD,MM.mmk`, `k` one of N, S, E, W (2005 edition, page 85).
GpsCoordinate,
/// An alternative array of text items, each with an `xml:lang` (ISO 16684-1 8.2.2.4).
LangAlt,
Seq(&'static Ty),
@@ -203,6 +209,19 @@ const ANCESTOR: &[Field] = &[(PHOTOSHOP, "AncestorID", Text)];
/// Part 2 Table 32.
const LAYER: &[Field] = &[(PHOTOSHOP, "LayerName", Text), (PHOTOSHOP, "LayerText", Text)];
const EXIF: &str = "http://ns.adobe.com/exif/1.0/";
/// Flash (2005 edition, page 85).
const FLASH: &[Field] =
&[(EXIF, "Fired", Boolean), (EXIF, "Return", Integer), (EXIF, "Mode", Integer), (EXIF, "Function", Boolean), (EXIF, "RedEyeMode", Boolean)];
/// OECF/SFR (2005 edition, page 86).
const OECF_SFR: &[Field] =
&[(EXIF, "Columns", Integer), (EXIF, "Rows", Integer), (EXIF, "Names", Ty::Seq(&Text)), (EXIF, "Values", Ty::Seq(&Rational))];
/// CFAPattern (2005 edition, page 87).
const CFA_PATTERN: &[Field] = &[(EXIF, "Columns", Integer), (EXIF, "Rows", Integer), (EXIF, "Values", Ty::Seq(&Integer))];
/// DeviceSettings (2005 edition, page 87).
const DEVICE_SETTINGS: &[Field] = &[(EXIF, "Columns", Integer), (EXIF, "Rows", Integer), (EXIF, "Settings", Ty::Seq(&Text))];
const SCHEMAS: &[Schema] = &[
// ISO 16684-1 Table 4.
Schema {
@@ -452,6 +471,126 @@ const SCHEMAS: &[Schema] = &[
("WhiteBalance", Text),
],
},
// 2005 edition, "EXIF Schema for TIFF Properties", pages 58–60. A Closed Choice of Integer is an
// Integer.
Schema {
uri: "http://ns.adobe.com/tiff/1.0/",
prefix: "tiff",
properties: &[
("ImageWidth", Integer),
("ImageLength", Integer),
("BitsPerSample", Ty::Seq(&Integer)),
("Compression", Integer),
("PhotometricInterpretation", Integer),
("Orientation", Integer),
("SamplesPerPixel", Integer),
("PlanarConfiguration", Integer),
("YCbCrSubSampling", Ty::Seq(&Integer)),
("YCbCrPositioning", Integer),
("XResolution", Rational),
("YResolution", Rational),
("ResolutionUnit", Integer),
("TransferFunction", Ty::Seq(&Integer)),
("WhitePoint", Ty::Seq(&Rational)),
("PrimaryChromaticities", Ty::Seq(&Rational)),
("YCbCrCoefficients", Ty::Seq(&Rational)),
("ReferenceBlackWhite", Ty::Seq(&Rational)),
("DateTime", Date),
("ImageDescription", LangAlt),
("Make", Text),
("Model", Text),
("Software", Text),
("Artist", Text),
("Copyright", LangAlt),
],
},
// 2005 edition, "EXIF Schema for EXIF-specific Properties", pages 60–69.
Schema {
uri: EXIF,
prefix: "exif",
properties: &[
("ExifVersion", Text),
("FlashpixVersion", Text),
("ColorSpace", Integer),
("ComponentsConfiguration", Ty::Seq(&Integer)),
("CompressedBitsPerPixel", Rational),
("PixelXDimension", Integer),
("PixelYDimension", Integer),
("UserComment", LangAlt),
("RelatedSoundFile", Text),
("DateTimeOriginal", Date),
("DateTimeDigitized", Date),
("ExposureTime", Rational),
("FNumber", Rational),
("ExposureProgram", Integer),
("SpectralSensitivity", Text),
("ISOSpeedRatings", Ty::Seq(&Integer)),
("OECF", Ty::Struct("OECF/SFR", OECF_SFR)),
("ShutterSpeedValue", Rational),
("ApertureValue", Rational),
("BrightnessValue", Rational),
("ExposureBiasValue", Rational),
("MaxApertureValue", Rational),
("SubjectDistance", Rational),
("MeteringMode", Integer),
("LightSource", Integer),
("Flash", Ty::Struct("Flash", FLASH)),
("FocalLength", Rational),
("SubjectArea", Ty::Seq(&Integer)),
("FlashEnergy", Rational),
("SpatialFrequencyResponse", Ty::Struct("OECF/SFR", OECF_SFR)),
("FocalPlaneXResolution", Rational),
("FocalPlaneYResolution", Rational),
("FocalPlaneResolutionUnit", Integer),
("SubjectLocation", Ty::Seq(&Integer)),
("ExposureIndex", Rational),
("SensingMethod", Integer),
("FileSource", Integer),
("SceneType", Integer),
("CFAPattern", Ty::Struct("CFAPattern", CFA_PATTERN)),
("CustomRendered", Integer),
("ExposureMode", Integer),
("WhiteBalance", Integer),
("DigitalZoomRatio", Rational),
("FocalLengthIn35mmFilm", Integer),
("SceneCaptureType", Integer),
("GainControl", Integer),
("Contrast", Integer),
("Saturation", Integer),
("Sharpness", Integer),
("DeviceSettingDescription", Ty::Struct("DeviceSettings", DEVICE_SETTINGS)),
("SubjectDistanceRange", Integer),
("ImageUniqueID", Text),
("GPSVersionID", Text),
("GPSLatitude", Ty::GpsCoordinate),
("GPSLongitude", Ty::GpsCoordinate),
("GPSAltitudeRef", Integer),
("GPSAltitude", Rational),
("GPSTimeStamp", Date),
("GPSSatellites", Text),
("GPSStatus", Text),
("GPSMeasureMode", Text),
("GPSDOP", Rational),
("GPSSpeedRef", Text),
("GPSSpeed", Rational),
("GPSTrackRef", Text),
("GPSTrack", Rational),
("GPSImgDirectionRef", Text),
("GPSImgDirection", Rational),
("GPSMapDatum", Text),
("GPSDestLatitude", Ty::GpsCoordinate),
("GPSDestLongitude", Ty::GpsCoordinate),
("GPSDestBearingRef", Text),
("GPSDestBearing", Rational),
("GPSDestDistanceRef", Text),
("GPSDestDistance", Rational),
("GPSProcessingMethod", Text),
("GPSAreaInformation", Text),
("GPSDifferential", Integer),
],
},
// 2005 edition, "EXIF Schema for Additional EXIF Properties", page 69.
Schema { uri: "http://ns.adobe.com/exif/1.0/aux/", prefix: "aux", properties: &[("Lens", Text), ("SerialNumber", Text)] },
];
/// Calls `f` with every metadata stream among the indirect objects `objs` (ISO 32000-1 14.3.2) that
@@ -535,8 +674,8 @@ fn written(qname: &str, local: &str) -> String {
format!("{out}:{local}")
}
/// `ty` as the XMP Specification writes value types: `Integer`, `Lang Alt`, `seq Rational`, a
/// structure by its type's name, such as `Dimensions`.
/// `ty` as the XMP Specification writes value types: `Integer`, `GPSCoordinate`, `Lang Alt`,
/// `seq Rational`, a structure by its type's name, such as `Flash`.
fn named(ty: Ty) -> String {
let array = |kind: &str, item: &Ty| format!("{kind} {}", named(*item));
match ty {
@@ -547,6 +686,7 @@ fn named(ty: Ty) -> String {
Boolean => "Boolean".into(),
Date => "Date".into(),
Rational => "Rational".into(),
Ty::GpsCoordinate => "GPSCoordinate".into(),
LangAlt => "Lang Alt".into(),
Ty::Seq(item) => array("seq", item),
Ty::Bag(item) => array("bag", item),
@@ -608,7 +748,7 @@ fn fits(ty: Ty, e: &Node<'_, '_>, depth: usize) -> bool {
return blank(e) && blank(&n) && simple(ty, v) && predefined_ok(&n, deeper);
}
match ty {
Ty::Text | Ty::Guid | Ty::Integer | Ty::Real | Ty::Boolean | Ty::Date | Ty::Rational => {
Ty::Text | Ty::Guid | Ty::Integer | Ty::Real | Ty::Boolean | Ty::Date | Ty::Rational | Ty::GpsCoordinate => {
// `rdf:resource` makes the value a URI, which only a text type takes.
// The element is then empty.
if e.attribute((RDF, "resource")).is_some() {
@@ -716,12 +856,25 @@ fn simple(ty: Ty, v: &str) -> bool {
Ty::Boolean => matches!(v, "True" | "False"),
Ty::Date => date(v),
Ty::Rational => v.split_once('/').is_some_and(|(n, d)| integer(n) && integer(d)),
Ty::GpsCoordinate => gps_coordinate(v),
Ty::Text | Ty::Guid => true,
Ty::OneOf(types) => types.iter().any(|t| simple(*t, v)),
_ => false,
}
}
/// `DDD,MM,SSk` or `DDD,MM.mmk`: degrees, minutes and seconds or minutes with any number of
/// fractional digits, then N, S, E or W (2005 edition, page 85).
fn gps_coordinate(v: &str) -> bool {
let Some(v) = v.strip_suffix(['N', 'S', 'E', 'W']) else { return false };
let Some((degrees, rest)) = v.split_once(',') else { return false };
let minutes_ok = match rest.split_once(',') {
Some((minutes, seconds)) => digits(minutes) && digits(seconds),
None => rest.split_once('.').is_some_and(|(m, f)| digits(m) && digits(f)),
};
digits(degrees) && minutes_ok
}
fn digits(s: &str) -> bool {
!s.is_empty() && s.bytes().all(|b| b.is_ascii_digit())
}
crates/preflight/tests/generation_repairs.rs
+3 -1
View File
@@ -4,7 +4,9 @@ use pdfcraft_preflight::{Level, convert};
use std::sync::Arc;
fn image_file(damaged: bool) -> Arc<Vec<u8>> {
let mut bytes = b"%PDF-1.7\n".to_vec();
// A PDF/A-2 header (6.1.2), so that Save as PDF/A has no header to rewrite and only the
// repaired references decide whether the next save is full.
let mut bytes = b"%PDF-1.7\n%\xE2\xE3\xCF\xD3\n".to_vec();
let bodies = [
"<< /Type /Catalog /Pages 2 0 R /HeaderImage 5 1 R /RowImage 5 0 R /StaleImage 5 2 R >>",
"<< /Type /Pages /Kids [3 0 R] /Count 1 >>",
crates/preflight/tests/known_answers.rs
+6 -2
View File
@@ -22,7 +22,8 @@ use pdfcraft_preflight::{Level, convert, declared, icc, verify};
/// A document from hand-written objects and extra trailer entries (copied from the crate's unit
/// tests).
fn build(objs: &[&str], trailer_extra: &str) -> Document {
let mut out = b"%PDF-1.7\n".to_vec();
// ISO 19005-2 6.1.2: the header line is followed by a comment of four bytes above 127.
let mut out = b"%PDF-1.7\n%\xE2\xE3\xCF\xD3\n".to_vec();
let mut offsets = Vec::new();
for (i, o) in objs.iter().enumerate() {
offsets.push(out.len());
@@ -33,7 +34,10 @@ fn build(objs: &[&str], trailer_extra: &str) -> Document {
for o in offsets {
out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes());
}
out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R {trailer_extra} >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes());
// ISO 19005-2 6.1.3: the trailer names a file identifier; a test about another clause gets one
// unless it gives its own.
let id = if trailer_extra.contains("/ID") { "" } else { "/ID [<00112233445566778899AABBCCDDEEFF> <00112233445566778899AABBCCDDEEFF>]" };
out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R {id} {trailer_extra} >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes());
Document::open(std::sync::Arc::new(out)).unwrap()
}
crates/redact/src/interp.rs
+4 -2
View File
@@ -31,7 +31,7 @@ mod prune;
use layers::strip_hidden_layers;
use prune::{clip_out, extgstate_font, prune_form_resources};
pub(crate) use prune::{prune, prune_patterns, prune_properties, resource_ops, unlink_doomed, unlink_pruned_patterns};
pub(crate) use prune::{prune, prune_patterns, prune_properties, resource_ops, unlink_doomed, unlink_pruned_patterns, unlink_undrawn};
/// How deep this pass follows form XObjects. A form past it is removed rather than walked, which
/// is the fail-safe choice for a redaction -- marked content inside a form we cannot read would
@@ -1097,7 +1097,9 @@ fn xobject(
out.error = Some(error);
return None;
}
if let Err(error) = crate::type3::prune_type3_from_resources(doc, &mut cleaned_res, content) {
// No page: annotation appearances draw through a page's resources, never through a
// form's. The form's own soft masks are still counted.
if let Err(error) = crate::type3::prune_type3_from_resources(doc, &mut cleaned_res, content, None) {
out.error = Some(error);
return None;
}
crates/redact/src/interp/prune.rs
+235 -61
View File
@@ -1,6 +1,6 @@
use super::{
CosError, Dict, Document, HashMap, HashSet, MAX_DEPTH, Matrix, Metrics, ObjRef, Object, Op, Pieces, Rc, RedactError, Scope, is_vertical, num,
overlaps, page_streams, res_dict,
CosError, DecodeBudget, Dict, Document, HashMap, HashSet, MAX_DEPTH, Matrix, Metrics, ObjRef, Object, Op, Pieces, Rc, RedactError, Scope,
is_vertical, num, overlaps, page_streams, res_dict,
};
/// Drop every `replaced` resource name whose `Do` no longer appears in the rewritten `content`,
@@ -20,33 +20,231 @@ pub(crate) fn prune(xo: &mut Dict, replaced: &[Vec<u8>], content: &[Vec<u8>]) {
}
}
/// Whether `content` draws the XObject `name` of the resource dictionary `xo`: directly, or
/// through a form that has no /Resources of its own (such a form draws through its caller's).
/// `None` when that cannot be told (a form that will not decode, or nesting past the limit).
fn content_draws(doc: &Document, content: &[Vec<u8>], xo: &Dict, name: &[u8], depth: usize, seen: &mut HashSet<ObjRef>) -> Option<bool> {
let joined = Pieces::join(content);
let ops = joined.parse().ops;
for op in ops.iter().filter(|op| op.is("Do")) {
let Some(n) = op.last_name() else { continue };
if n == name {
return Some(true);
}
let Some(r) = xo.get(n).and_then(Object::as_ref) else { continue };
let obj = doc.get(r);
let Object::Stream(s) = &*obj else { continue };
if s.dict.name(b"Subtype") != Some(b"Form") || s.dict.contains(b"Resources") || !seen.insert(r) {
continue;
}
if depth >= MAX_DEPTH {
return None;
}
let Ok(data) = s.decoded() else { return None };
match content_draws(doc, std::slice::from_ref(&data), xo, name, depth + 1, seen) {
Some(false) => {}
other => return other,
/// The XObject names of the resource dictionary `xo` that `content` draws: directly, or through
/// a form that has no /Resources of its own (such a form draws through its caller's). Each such
/// form is read once, however many names are asked about. The flag says whether some of what
/// `content` draws could not be read (a form that will not decode, or nesting past the limit), so
/// a name missing from the set may still be drawn.
fn names_content_draws(doc: &Document, content: &[Vec<u8>], xo: &Dict) -> (HashSet<Vec<u8>>, bool) {
let mut drawn = HashSet::new();
let mut seen = HashSet::new();
let mut pending = vec![(Pieces::join(content).parse().ops, 0)];
// A form that will not decode makes the answer uncertain, but it does not stop the forms
// already queued from being read: a name drawn only inside one of those need not be unknown.
let mut unknown = false;
while let Some((ops, depth)) = pending.pop() {
for n in ops.iter().filter(|op| op.is("Do")).filter_map(Op::last_name) {
if !drawn.insert(n.to_vec()) {
continue;
}
let Some(r) = xo.get(n).and_then(Object::as_ref) else { continue };
let obj = doc.get(r);
let Object::Stream(s) = &*obj else { continue };
if s.dict.name(b"Subtype") != Some(b"Form") || s.dict.contains(b"Resources") || !seen.insert(r) {
continue;
}
match s.decoded() {
Ok(data) if depth < MAX_DEPTH => pending.push((pdfcraft_content::parse(&data).ops, depth + 1)),
_ => unknown = true,
}
}
}
Some(false)
(drawn, unknown)
}
/// Whether any of `drawn` (from [`names_content_draws`]) draws `name`: `None` when none is known to and
/// one could not be read in full.
fn any_draws(drawn: &[(HashSet<Vec<u8>>, bool)], name: &[u8]) -> Option<bool> {
if drawn.iter().any(|(names, _)| names.contains(name)) {
Some(true)
} else if drawn.iter().any(|(_, unknown)| *unknown) {
None
} else {
Some(false)
}
}
/// Every page-tree node with a /Resources of its own, and the pages that draw through it (the
/// nearest such node above a page, or the page itself). Cycles are cut as in
/// `pdfcraft_model::pages`. The flag is false when a node was reached twice (listed under two
/// parents, or in a cycle) or the walk stopped early: a page may then draw through a holder it
/// isn't listed under.
fn resource_holders(doc: &Document) -> (Vec<(ObjRef, Vec<ObjRef>)>, bool) {
let mut holders: Vec<(ObjRef, Vec<ObjRef>)> = Vec::new();
let mut whole = true;
let Some(top) = doc.root().and_then(|root| doc.get(root).as_dict().and_then(|d| d.reference(b"Pages"))) else { return (holders, whole) };
let mut seen = HashSet::new();
let mut stack: Vec<(ObjRef, Option<usize>)> = vec![(top, None)];
while let Some((node, mut holder)) = stack.pop() {
if !seen.insert(node) || seen.len() > 1_000_000 {
whole = false;
continue;
}
let obj = doc.get(node);
let Some(d) = obj.as_dict() else { continue };
if d.contains(b"Resources") {
holders.push((node, Vec::new()));
holder = Some(holders.len() - 1);
}
let is_pages = d.name(b"Type") == Some(b"Pages") || (d.contains(b"Kids") && d.name(b"Type") != Some(b"Page"));
if is_pages {
let kids = d.get(b"Kids").map(|k| doc.resolve(k)).and_then(|k| k.as_array().cloned()).unwrap_or_default();
stack.extend(kids.iter().filter_map(Object::as_ref).map(|k| (k, holder)));
} else if let Some(h) = holder.and_then(|i| holders.get_mut(i)) {
h.1.push(node);
}
}
(holders, whole)
}
/// Unlink the image and form XObjects nothing draws from the page-tree resource dictionaries
/// that still name them: Delete Image and Replace Image take an image off the page but leave it
/// listed there, and so in a full-rewrite save. A name stays when anything that draws through the
/// dictionary uses it ([`names_drawn`]), and a dictionary whose use can't be told is left as it
/// is, as is every one when a page is listed twice in the page tree. A dictionary that loses an
/// entry is replaced by a private copy on its node, as in [`unlink_doomed`]. Returns how many
/// names went.
pub(crate) fn unlink_undrawn(doc: &mut Document) -> Result<usize, RedactError> {
let index: HashMap<ObjRef, usize> = pdfcraft_model::pages(doc).iter().enumerate().map(|(i, p)| (p.obj, i)).collect();
let (holders, whole) = resource_holders(doc);
if !whole {
return Ok(0);
}
let mut n = 0;
for (node, users) in holders {
let Some(mut res) = doc.get(node).as_dict().and_then(|d| d.get(b"Resources")).and_then(|r| doc.resolve(r).as_dict().cloned()) else {
continue;
};
let mut xo = res_dict(doc, &res, b"XObject");
let listed: Vec<Vec<u8>> = xo
.iter()
.filter(|(_, v)| {
v.as_ref().is_some_and(|r| matches!(&*doc.get(r), Object::Stream(s) if matches!(s.dict.name(b"Subtype"), Some(b"Image" | b"Form"))))
})
.map(|(name, _)| name.to_vec())
.collect();
if listed.is_empty() {
continue;
}
let mut pruned = false;
// A form that goes may have been all that drew another name: look again while names go.
for _ in 0..MAX_DEPTH {
let Some(drawn) = names_drawn(doc, &res, &users, &index) else { break };
let mut gone = 0;
for name in listed.iter().filter(|name| !drawn.contains(*name)) {
gone += usize::from(xo.remove(name).is_some());
}
if gone == 0 {
break;
}
n += gone;
pruned = true;
res.set(b"XObject".to_vec(), Object::Dict(xo.clone()));
}
if pruned {
doc.update_dict(node, |d| d.set(b"Resources".to_vec(), Object::Dict(res)))?;
}
}
Ok(n)
}
/// Every name used with `Do` by what draws through `res`, the resource dictionary of `pages`:
/// their content, their annotations' appearances, and every form, tiling pattern, Type 3 glyph
/// and soft mask those can reach, with /Resources of their own or not. A name such a stream
/// doesn't define is looked up in the resources of whatever draws it, up to the page's, so a `Do`
/// anywhere in there may draw from `res`. Going by name keeps more than it must, never less.
/// Each stream is read once, against one budget. `None` when that can't be told: a page or
/// stream that won't decode, or the budget spent.
fn names_drawn(doc: &Document, res: &Dict, pages: &[ObjRef], index: &HashMap<ObjRef, usize>) -> Option<HashSet<Vec<u8>>> {
let mut drawn = HashSet::new();
let mut note = |data: &[u8]| {
for op in pdfcraft_content::parse(data).ops.iter().filter(|op| op.is("Do")) {
if let Some(name) = op.last_name() {
drawn.insert(name.to_vec());
}
}
};
// Streams still to read, and resource dictionaries still to look through.
let mut streams: Vec<ObjRef> = Vec::new();
let mut dicts: Vec<Dict> = vec![res.clone()];
for page in pages {
let dict = doc.get(*page).as_dict().cloned().unwrap_or_default();
let (_, content) = page_streams(doc, &dict, index.get(page).copied().unwrap_or(0)).ok()?;
// One stream may end with a name its `Do` begins the next with: read them as one.
note(Pieces::join(&content).data());
let annots = dict.get(b"Annots").map(|a| doc.resolve(a)).and_then(|a| a.as_array().cloned()).unwrap_or_default();
for annot in annots {
let ap = doc.resolve(&annot).as_dict().and_then(|a| a.get(b"AP")).map(|ap| doc.resolve(ap)).and_then(|ap| ap.as_dict().cloned());
// Each of /N, /R and /D is an appearance, or a dictionary of them by state.
for (_, look) in ap.unwrap_or_default().iter() {
match look.as_ref() {
Some(r) if matches!(&*doc.resolve(look), Object::Stream(_)) => streams.push(r),
_ => streams.extend(doc.resolve(look).as_dict().into_iter().flat_map(|states| states.iter().filter_map(|(_, v)| v.as_ref()))),
}
}
}
}
let mut budget = DecodeBudget::page();
let mut read = HashSet::new();
let mut looked = 0;
loop {
while let Some(dict) = dicts.get(looked) {
let (found, fonts) = drawn_through(doc, dict);
looked += 1;
streams.extend(found);
// Type 3 fonts' own resources, each look charged like a form so fonts naming each
// other can't go round for ever.
for font_res in fonts {
if !budget.enter_form() {
return None;
}
dicts.push(font_res);
}
}
let Some(r) = streams.pop() else { break };
if !read.insert(r) {
continue;
}
let obj = doc.resolve(&Object::Ref(r));
let Object::Stream(s) = &*obj else { continue };
if !budget.enter_form() {
return None;
}
note(&budget.decode_strict(doc, s).ok()?);
if let Some(own) = s.dict.get(b"Resources").map(|o| doc.resolve(o)).and_then(|o| o.as_dict().cloned()) {
dicts.push(own);
}
}
Some(drawn)
}
/// The content streams the resource dictionary `res` lists: its form XObjects, its tiling
/// patterns, its Type 3 fonts' glyph procedures and its soft masks' groups. Then the /Resources
/// of those Type 3 fonts, which their glyphs look names up in first. An object that holds only a
/// reference stands for what it names, as the renderer reads it.
fn drawn_through(doc: &Document, res: &Dict) -> (Vec<ObjRef>, Vec<Dict>) {
let is_stream = |r: &ObjRef| matches!(&*doc.resolve(&Object::Ref(*r)), Object::Stream(_));
let mut found: Vec<ObjRef> = Vec::new();
let mut fonts: Vec<Dict> = Vec::new();
for (_, x) in res_dict(doc, res, b"XObject").iter() {
found.extend(x.as_ref().filter(|_| matches!(&*doc.resolve(x), Object::Stream(s) if s.dict.name(b"Subtype") == Some(b"Form"))));
}
for (_, p) in res_dict(doc, res, b"Pattern").iter() {
found.extend(p.as_ref().filter(|_| matches!(&*doc.resolve(p), Object::Stream(s) if s.dict.int(b"PatternType") == Some(1))));
}
for (_, f) in res_dict(doc, res, b"Font").iter() {
let font = doc.resolve(f);
let Some(font) = font.as_dict().filter(|f| f.name(b"Subtype") == Some(b"Type3")) else { continue };
let procs = font.get(b"CharProcs").map(|c| doc.resolve(c)).and_then(|c| c.as_dict().cloned()).unwrap_or_default();
found.extend(procs.iter().filter_map(|(_, v)| v.as_ref()).filter(is_stream));
fonts.extend(font.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()));
}
for (_, g) in res_dict(doc, res, b"ExtGState").iter() {
let gs = doc.resolve(g);
let mask = gs.as_dict().and_then(|g| g.get(b"SMask")).map(|m| doc.resolve(m));
found.extend(mask.as_ref().and_then(|m| m.as_dict()).and_then(|m| m.reference(b"G")).filter(is_stream));
}
(found, fonts)
}
/// Unlink the XObjects a rewrite dropped or replaced (`doomed`) from every page-tree resource
@@ -67,31 +265,7 @@ pub(crate) fn unlink_doomed(doc: &mut Document, doomed: &[ObjRef], retired_prope
return Ok(());
}
let doomed: HashSet<ObjRef> = doomed.iter().copied().collect();
let Some(top) = doc.root().and_then(|root| doc.get(root).as_dict().and_then(|d| d.reference(b"Pages"))) else { return Ok(()) };
// Every page-tree node with a /Resources of its own, and the pages that draw through it
// (the nearest such node above a page, or the page itself). Cycles are cut as in
// `pdfcraft_model::pages`.
let mut holders: Vec<(ObjRef, Vec<ObjRef>)> = Vec::new();
let mut seen = HashSet::new();
let mut stack: Vec<(ObjRef, Option<usize>)> = vec![(top, None)];
while let Some((node, mut holder)) = stack.pop() {
if !seen.insert(node) || seen.len() > 1_000_000 {
continue;
}
let obj = doc.get(node);
let Some(d) = obj.as_dict() else { continue };
if d.contains(b"Resources") {
holders.push((node, Vec::new()));
holder = Some(holders.len() - 1);
}
let is_pages = d.name(b"Type") == Some(b"Pages") || (d.contains(b"Kids") && d.name(b"Type") != Some(b"Page"));
if is_pages {
let kids = d.get(b"Kids").map(|k| doc.resolve(k)).and_then(|k| k.as_array().cloned()).unwrap_or_default();
stack.extend(kids.iter().filter_map(Object::as_ref).map(|k| (k, holder)));
} else if let Some(h) = holder.and_then(|i| holders.get_mut(i)) {
h.1.push(node);
}
}
let (holders, _) = resource_holders(doc);
let index: HashMap<ObjRef, usize> = pdfcraft_model::pages(doc).iter().enumerate().map(|(i, p)| (p.obj, i)).collect();
let holder_nodes: Vec<ObjRef> = holders.iter().map(|(node, _)| *node).collect();
for (node, users) in holders {
@@ -112,15 +286,11 @@ pub(crate) fn unlink_doomed(doc: &mut Document, doomed: &[ObjRef], retired_prope
contents.push(page_streams(doc, &page, index.get(user).copied().unwrap_or(0))?.1);
}
let mut pruned = false;
// What each page draws through this dictionary, worked out once for all the names.
let drawn: Vec<_> =
if names.is_empty() { Vec::new() } else { contents.iter().map(|content| names_content_draws(doc, content, &xo)).collect() };
for n in names {
let mut drawn = Some(false);
for content in &contents {
drawn = content_draws(doc, content, &xo, &n, 0, &mut HashSet::new());
if drawn != Some(false) {
break;
}
}
match drawn {
match any_draws(&drawn, &n) {
Some(true) => {}
Some(false) => {
xo.remove(&n);
@@ -230,8 +400,12 @@ fn unlink_from_forms(doc: &mut Document, holders: &[ObjRef], doomed: &HashSet<Ob
fn prune_doomed_xobjects(doc: &Document, content: &[Vec<u8>], xo: &mut Dict, doomed_names: &[Vec<u8>]) -> Result<bool, CosError> {
let mut pruned = false;
if doomed_names.is_empty() {
return Ok(false);
}
let drawn = [names_content_draws(doc, content, xo)];
for name in doomed_names {
match content_draws(doc, content, xo, name, 0, &mut HashSet::new()) {
match any_draws(&drawn, name) {
Some(true) => {}
Some(false) => pruned |= xo.remove(name).is_some(),
None => return Err(CosError::Syntax { offset: 0, detail: "redaction could not fully inspect shared form resources".into() }),
crates/redact/src/lib.rs
+1 -1
View File
@@ -855,7 +855,7 @@ fn redact_page_content(
let painted = if out.keeps_all_patterns { &out.patterns_seen } else { &out.patterns_painted };
report.patterns_removed += interp::prune_patterns(doc, &mut res, &out.patterns_seen, painted, pruned_patterns);
retired_properties.extend(interp::prune_properties(doc, &mut res, &cleaned_data)?);
type3::prune_type3_from_resources(doc, &mut res, &cleaned_data)?;
type3::prune_type3_from_resources(doc, &mut res, &cleaned_data, Some(&page.dict))?;
doc.update_dict(page.obj, |d| {
d.set(b"Contents".to_vec(), Object::Array(new_list));
d.set(b"Resources".to_vec(), Object::Dict(res));
crates/redact/src/sanitize.rs
+156 -16
View File
@@ -410,22 +410,153 @@ fn is_comment(subtype: &[u8]) -> bool {
!matches!(subtype, b"FileAttachment" | b"Redact") && pdfcraft_annot::is_comment_subtype(&String::from_utf8_lossy(subtype))
}
/// The layers that are off in the default configuration.
/// The layers that are off in the default configuration: listed off (or not listed on, under
/// `/BaseState /OFF`), or turned off on screen by one of its usage rules (`/AS`, ISO 32000-2,
/// 8.11.4.4): a rule for the View event in the View category hides a layer whose
/// `/Usage /View /ViewState` is OFF. Rules that turn a layer on, or apply only when printing or
/// exporting, leave it as the lists say.
fn hidden_layers(doc: &Document) -> Vec<ObjRef> {
let Some((_, cat)) = catalog(doc) else { return Vec::new() };
let Some(oc) = sub(doc, &cat, b"OCProperties") else { return Vec::new() };
let Some(d) = sub(doc, &oc, b"D") else { return Vec::new() };
let all: Vec<ObjRef> =
oc.get(b"OCGs").and_then(|g| doc.resolve(g).as_array().map(|a| a.iter().filter_map(Object::as_ref).collect())).unwrap_or_default();
let list = |k: &[u8]| -> Vec<ObjRef> {
d.get(k).and_then(|g| doc.resolve(g).as_array().map(|a| a.iter().filter_map(Object::as_ref).collect())).unwrap_or_default()
let refs = |o: Option<&Object>| -> Vec<ObjRef> {
o.and_then(|g| doc.resolve(g).as_array().map(|a| a.iter().filter_map(Object::as_ref).collect())).unwrap_or_default()
};
let (on, off) = (list(b"ON"), list(b"OFF"));
if d.name(b"BaseState") == Some(b"OFF") {
all.into_iter().filter(|g| !on.contains(g)).collect()
let all = refs(oc.get(b"OCGs"));
let (on, off) = (refs(d.get(b"ON")), refs(d.get(b"OFF")));
let mut hidden: Vec<ObjRef> = if d.name(b"BaseState") == Some(b"OFF") {
all.iter().copied().filter(|g| !on.contains(g)).collect()
} else {
off.into_iter().filter(|g| all.contains(g)).collect()
};
// Each rule, layer list and layer is read once, however often the rules name them: many rules
// can share one long list.
let known: std::collections::HashSet<ObjRef> = all.iter().copied().collect();
let mut looked: std::collections::HashSet<ObjRef> = hidden.iter().copied().collect();
let mut read = std::collections::HashSet::new();
let rules = d.get(b"AS").and_then(|a| doc.resolve(a).as_array().cloned()).unwrap_or_default();
for rule in &rules {
if let Some(r) = rule.as_ref()
&& !read.insert(r)
{
continue;
}
let Some(rule) = doc.resolve(rule).as_dict().cloned() else { continue };
let categories = rule.get(b"Category").and_then(|c| doc.resolve(c).as_array().cloned()).unwrap_or_default();
if rule.name(b"Event") != Some(b"View") || !categories.iter().any(|c| c.as_name() == Some(b"View")) {
continue;
}
if let Some(r) = rule.get(b"OCGs").and_then(Object::as_ref)
&& !read.insert(r)
{
continue;
}
for g in refs(rule.get(b"OCGs")) {
if !known.contains(&g) || !looked.insert(g) {
continue;
}
let usage = doc.get(g).as_dict().and_then(|layer| sub(doc, layer, b"Usage"));
if usage.and_then(|u| sub(doc, &u, b"View")).is_some_and(|v| v.name(b"ViewState") == Some(b"OFF")) {
hidden.push(g);
}
}
}
hidden
}
/// The objects a layer cleanup has reached by reference, and whether the lists naming each keep it.
type Cleaned = std::collections::HashMap<ObjRef, bool>;
/// `o` cleaned by `clean`: in its list when written there, and when written by reference, once and
/// where it is, however many lists name it (they keep the reference, or drop it when `clean` gives
/// `None`). A list that names one object many times costs one step per object, not one per path,
/// and a cycle stops where it began.
fn once(
doc: &mut Document,
o: &Object,
cleaned: &mut Cleaned,
clean: impl FnOnce(&mut Document, &Object, &mut Cleaned) -> Result<Option<Object>, RedactError>,
) -> Result<Option<Object>, RedactError> {
let Object::Ref(r) = o else { return clean(doc, o, cleaned) };
if let Some(&kept) = cleaned.get(r) {
return Ok(kept.then(|| o.clone()));
}
cleaned.insert(*r, true);
let held: &Object = &doc.resolve(o);
let Some(new) = clean(doc, held, cleaned)? else {
cleaned.insert(*r, false);
return Ok(None);
};
if new != *held {
doc.set_existing(*r, new)?;
}
Ok(Some(o.clone()))
}
/// `o` without the entries `keep` refuses, through nested arrays (an `/Order` holds sub-lists, and
/// `/RBGroups` holds groups), each array written by reference cleaned [`once`]. `None` when `keep`
/// refuses `o` itself, or when `o` is a sub-list (`depth` above 0) that named a layer and names
/// none now: it goes, with the heading that labelled it.
fn filtered(
doc: &mut Document,
o: &Object,
keep: &impl Fn(&Object) -> bool,
depth: usize,
cleaned: &mut Cleaned,
) -> Result<Option<Object>, RedactError> {
if depth > MAX_DIRECT_DEPTH {
return Err(RedactError::TooDeep);
}
if !keep(o) {
return Ok(None);
}
once(doc, o, cleaned, |doc, o, cleaned| {
let Object::Array(a) = o else { return Ok(Some(o.clone())) };
let mut out = Vec::new();
for x in a {
out.extend(filtered(doc, x, keep, depth + 1, cleaned)?);
}
// Anything but a text string (a heading) is a layer or a sub-list.
let names = |l: &[Object]| l.iter().any(|x| !matches!(x, Object::String(_)));
let emptied = depth > 0 && names(a.as_slice()) && !names(out.as_slice());
Ok((!emptied).then_some(Object::Array(out)))
})
}
/// An optional-content configuration without the layers `keep` refuses, in every list that names
/// them: on, off, order, locked, radio-button groups and usage rules (a rule, group or sub-list left
/// with no layer goes). A removed layer still named in any of them would stay in the file, its name
/// included.
fn config_without(doc: &mut Document, mut d: Dict, keep: &impl Fn(&Object) -> bool, cleaned: &mut Cleaned) -> Result<Dict, RedactError> {
for k in [&b"ON"[..], b"OFF", b"Order", b"Locked", b"RBGroups"] {
if let Some(v) = d.get(k).cloned() {
let v = filtered(doc, &v, keep, 0, cleaned)?.unwrap_or_else(|| Object::Array(Vec::new()));
d.set(k.to_vec(), v);
}
}
let Some(rules) = d.get(b"AS").cloned() else { return Ok(d) };
let rules = once(doc, &rules, cleaned, |doc, rules, cleaned| {
let Object::Array(rules) = rules else { return Ok(Some(rules.clone())) };
let mut kept = Vec::new();
for rule in rules {
kept.extend(once(doc, rule, cleaned, |doc, rule, cleaned| {
let Some(mut rule) = rule.as_dict().cloned() else { return Ok(None) };
if let Some(layers) = rule.get(b"OCGs").cloned() {
let Some(layers) = filtered(doc, &layers, keep, 0, cleaned)? else { return Ok(None) };
if doc.resolve(&layers).as_array().is_some_and(Vec::is_empty) {
return Ok(None);
}
rule.set(b"OCGs".to_vec(), layers);
}
Ok(Some(Object::Dict(rule)))
})?);
}
Ok(Some(Object::Array(kept)))
})?;
if let Some(rules) = rules {
d.set(b"AS".to_vec(), rules);
}
Ok(d)
}
/// Glyphs of hidden text, or content of hidden layers, on every page (`write` = remove them).
@@ -511,7 +642,7 @@ fn content_pass(doc: &mut Document, text: bool, layers: bool, write: bool) -> Re
res.set(b"XObject".to_vec(), Object::Dict(xo));
}
retired_properties.extend(crate::interp::prune_properties(doc, &mut res, &page_content)?);
crate::type3::prune_type3_from_resources(doc, &mut res, &page_content)?;
crate::type3::prune_type3_from_resources(doc, &mut res, &page_content, Some(&page.dict))?;
doc.update_dict(page.obj, |d| {
d.set(b"Contents".to_vec(), Object::Array(new_list));
d.set(b"Resources".to_vec(), Object::Dict(res));
@@ -755,13 +886,19 @@ pub fn remove_hidden(doc: &mut Document, which: &[Hidden]) -> Result<Vec<(Hidden
if let Some(Object::Array(a)) = oc.get(b"OCGs").map(|g| (*doc.resolve(g)).clone()) {
oc.set(b"OCGs".to_vec(), Object::Array(a.into_iter().filter(keep).collect()));
}
if let Some(mut d) = sub(doc, &oc, b"D") {
for k in [&b"ON"[..], b"OFF", b"Order", b"Locked"] {
if let Some(Object::Array(a)) = d.get(k).map(|g| (*doc.resolve(g)).clone()) {
d.set(k.to_vec(), Object::Array(a.into_iter().filter(keep).collect()));
}
let mut cleaned = Cleaned::new();
if let Some(d) = sub(doc, &oc, b"D") {
oc.set(b"D".to_vec(), Object::Dict(config_without(doc, d, &keep, &mut cleaned)?));
}
if let Some(Object::Array(configs)) = oc.get(b"Configs").map(|c| (*doc.resolve(c)).clone()) {
let mut kept = Vec::with_capacity(configs.len());
for config in configs {
kept.extend(once(doc, &config, &mut cleaned, |doc, config, cleaned| match config.as_dict() {
Some(c) => Ok(Some(Object::Dict(config_without(doc, c.clone(), &keep, cleaned)?))),
None => Ok(None),
})?);
}
oc.set(b"D".to_vec(), Object::Dict(d));
oc.set(b"Configs".to_vec(), Object::Array(kept));
}
cat.set(b"OCProperties".to_vec(), Object::Dict(oc));
}
@@ -790,7 +927,10 @@ pub fn remove_hidden(doc: &mut Document, which: &[Hidden]) -> Result<Vec<(Hidden
doc.set(r, o);
}
}
if done.is_empty() {
// Images taken off their page (Delete Image, Replace Image) are still listed in its
// resources: the full rewrite would keep them. Unlinking them is a removal of its own.
let unlinked = crate::interp::unlink_undrawn(doc)?;
if done.is_empty() && unlinked == 0 {
return Err(RedactError::NothingToApply);
}
// The rewrite drops the revisions that held the removed content, so it is a different file from
crates/redact/src/tests.rs
+225
View File
@@ -4606,3 +4606,228 @@ fn private_data_takes_opi_links_and_alternate_images() {
assert!(!has(b"ALTERNATEORIGINAL"), "the alternate image survived the save");
assert!(!has(b"OPISERVERPATH"), "the OPI link survived the save");
}
/// The full rewrite of the document made of `objs` after Remove Hidden Information, run for its
/// metadata so it has something to do.
fn hidden_information_removed(objs: Vec<Vec<u8>>) -> Vec<u8> {
use crate::sanitize::{Hidden, remove_hidden};
let mut doc = pdf(objs);
let info = doc.add(Object::Dict({
let mut d = Dict::new();
d.set(b"Title".to_vec(), pdfcraft_cos::PdfString::text("Plan"));
d
}));
doc.trailer_mut().set(b"Info".to_vec(), Object::Ref(info));
remove_hidden(&mut doc, &[Hidden::Metadata]).unwrap();
write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap()
}
fn gray_image(data: &[u8]) -> Vec<u8> {
stream(&format!("/Type /XObject /Subtype /Image /Width {} /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", data.len()), data)
}
/// An image Delete Image took off the page is still listed in its resources: Remove Hidden
/// Information unlinks it, so the full rewrite leaves it out, as it does one only an unused form
/// draws. Images a page still draws stay, directly, through a form without /Resources, or on
/// another page sharing the dictionary, its name and `Do` in two content streams.
#[test]
fn images_no_page_draws_leave_with_hidden_information() {
let bytes = hidden_information_removed(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 5 0 R /Resources 7 0 R >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents [6 0 R 15 0 R] /Resources 7 0 R >>".to_vec(),
stream("", b"q 50 0 0 50 20 20 cm /Shown Do Q /Fm Do"),
stream("", b"q 50 0 0 50 20 20 cm /Other"),
b"<< /XObject << /Shown 8 0 R /Deleted 9 0 R /Nested 10 0 R /Other 11 0 R /Fm 12 0 R /Unused 13 0 R /Orphan 14 0 R >> >>".to_vec(),
gray_image(b"SHOWNIMAGEBYTES"),
gray_image(b"DELETEDIMAGEBYTES"),
gray_image(b"NESTEDIMAGEBYTES"),
gray_image(b"OTHERPAGEIMAGE"),
stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300]", b"q 10 0 0 10 200 200 cm /Nested Do Q"),
stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300]", b"/Orphan Do"),
gray_image(b"ORPHANIMAGEBYTES"),
stream("", b"Do Q"),
]);
let has = |needle: &[u8]| bytes.windows(needle.len()).any(|w| w == needle);
assert!(!has(b"DELETEDIMAGEBYTES"), "the deleted image survived the save");
assert!(!has(b"ORPHANIMAGEBYTES"), "an image only an unused form draws survived the save");
for kept in [&b"SHOWNIMAGEBYTES"[..], b"NESTEDIMAGEBYTES", b"OTHERPAGEIMAGE"] {
assert!(has(kept), "{} is drawn and must stay", String::from_utf8_lossy(kept));
}
}
/// A page listed under two /Pages nodes draws through either one's resources, whichever a reader
/// comes to it from: neither loses the image it draws.
#[test]
fn a_page_listed_twice_keeps_both_parents_images() {
let bytes = hidden_information_removed(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 >>".to_vec(),
b"<< /Type /Pages /Parent 2 0 R /Kids [5 0 R] /Count 1 /Resources << /XObject << /Im 7 0 R >> >> >>".to_vec(),
b"<< /Type /Pages /Parent 2 0 R /Kids [5 0 R] /Count 1 /Resources << /XObject << /Im 8 0 R >> >> >>".to_vec(),
b"<< /Type /Page /Parent 3 0 R /MediaBox [0 0 300 300] /Contents 6 0 R >>".to_vec(),
stream("", b"q 50 0 0 50 20 20 cm /Im Do Q"),
gray_image(b"FIRSTPARENTIMAGE"),
gray_image(b"SECONDPARENTIMAGE"),
]);
let has = |needle: &[u8]| bytes.windows(needle.len()).any(|w| w == needle);
for kept in [&b"FIRSTPARENTIMAGE"[..], b"SECONDPARENTIMAGE"] {
assert!(has(kept), "{} is drawn and must stay", String::from_utf8_lossy(kept));
}
}
/// Unlinking an image nothing draws is a removal of its own: with nothing else to remove, it is
/// still applied, not reported as nothing to do.
#[test]
fn unlinking_alone_still_applies() {
use crate::sanitize::{Hidden, remove_hidden};
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Resources << /XObject << /Deleted 4 0 R >> >> >>".to_vec(),
gray_image(b"DELETEDIMAGEBYTES"),
]);
assert_eq!(remove_hidden(&mut doc, &[Hidden::Metadata]), Ok(Vec::new()));
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
assert!(!bytes.windows(17).any(|w| w == b"DELETEDIMAGEBYTES"), "the deleted image survived the save");
}
/// A name a pattern, a form, an appearance or a soft mask doesn't define is looked up in the
/// page's resources, /Resources of its own or not: an image drawn only that way is still shown,
/// so it stays, as one drawn by a form named through an object that holds only a reference. One
/// no content names goes.
#[test]
fn images_drawn_through_the_page_resources_stay() {
let bytes = hidden_information_removed(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources 5 0 R /Annots [6 0 R] >>".to_vec(),
stream("", b"q /Pattern cs /Pt scn 0 0 10 10 re f Q /Own Do /Loop Do q /Gs gs 0 0 5 5 re f Q /Chained Do"),
b"<< /XObject << /ByPattern 7 0 R /ByOwnForm 8 0 R /ByAppearance 9 0 R /ByMask 10 0 R /Deleted 11 0 R /Own 12 0 R /Loop 16 0 R \
/Chained 17 0 R /ByChain 19 0 R >> /Pattern << /Pt 13 0 R >> /ExtGState << /Gs << /SMask << /S /Luminosity /G 14 0 R >> >> >> >>"
.to_vec(),
b"<< /Type /Annot /Subtype /Square /Rect [0 0 50 50] /AP << /N 15 0 R >> >>".to_vec(),
gray_image(b"BYPATTERNIMAGE"),
gray_image(b"BYOWNFORMIMAGE"),
gray_image(b"BYAPPEARANCEIMAGE"),
gray_image(b"BYMASKIMAGE"),
gray_image(b"DELETEDIMAGEBYTES"),
stream("/Type /XObject /Subtype /Form /BBox [0 0 10 10] /Resources << /ProcSet [/PDF] >>", b"/ByOwnForm Do"),
stream("/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 10 10] /XStep 10 /YStep 10", b"/ByPattern Do"),
stream("/Type /XObject /Subtype /Form /BBox [0 0 10 10] /Group << /S /Transparency /CS /DeviceGray >>", b"/ByMask Do"),
stream("/Type /XObject /Subtype /Form /BBox [0 0 50 50]", b"/ByAppearance Do"),
// Draws itself: read once.
stream("/Type /XObject /Subtype /Form /BBox [0 0 10 10] /Resources << /XObject << /Loop 16 0 R >> >>", b"/Loop Do"),
b"18 0 R".to_vec(),
stream("/Type /XObject /Subtype /Form /BBox [0 0 10 10]", b"/ByChain Do"),
gray_image(b"BYCHAINIMAGE"),
]);
let has = |needle: &[u8]| bytes.windows(needle.len()).any(|w| w == needle);
assert!(!has(b"DELETEDIMAGEBYTES"), "the deleted image survived the save");
for kept in [&b"BYPATTERNIMAGE"[..], b"BYOWNFORMIMAGE", b"BYAPPEARANCEIMAGE", b"BYMASKIMAGE", b"BYCHAINIMAGE"] {
assert!(has(kept), "{} is drawn and must stay", String::from_utf8_lossy(kept));
}
}
/// A Type 3 glyph draws through the page's resources too, as does a form in the font's own
/// resources: their images stay, and the dictionary is still cleaned of what nothing draws.
#[test]
fn images_drawn_by_type3_glyphs_stay() {
let bytes = hidden_information_removed(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources 5 0 R >>".to_vec(),
stream("", b"BT /T3 12 Tf (a) Tj ET"),
b"<< /XObject << /ByGlyph 7 0 R /Deleted 8 0 R /ByFontForm 11 0 R >> /Font << /T3 6 0 R >> >>".to_vec(),
b"<< /Type /Font /Subtype /Type3 /FontBBox [0 0 10 10] /FontMatrix [0.1 0 0 0.1 0 0] /CharProcs << /a 9 0 R >> \
/Encoding << /Differences [97 /a] >> /FirstChar 97 /LastChar 97 /Widths [10] /Resources << /XObject << /Fx 10 0 R >> >> >>"
.to_vec(),
gray_image(b"BYGLYPHIMAGE"),
gray_image(b"DELETEDIMAGEBYTES"),
stream("", b"10 0 d0 /ByGlyph Do /Fx Do"),
stream("/Type /XObject /Subtype /Form /BBox [0 0 10 10]", b"/ByFontForm Do"),
gray_image(b"BYFONTFORMIMAGE"),
]);
let has = |needle: &[u8]| bytes.windows(needle.len()).any(|w| w == needle);
assert!(!has(b"DELETEDIMAGEBYTES"), "the deleted image survived the save");
assert!(has(b"BYGLYPHIMAGE"), "the glyph's image is drawn and must stay");
assert!(has(b"BYFONTFORMIMAGE"), "the image a form in the font's resources draws must stay");
}
/// Layer lists that name one array many times, through arrays that do the same, are cleaned once
/// where each array is, not copied once per path: the file stays as small as it was.
#[test]
fn a_shared_layer_list_is_cleaned_once() {
use crate::sanitize::{Hidden, remove_hidden};
let mut objs = vec![
b"<< /Type /Catalog /Pages 2 0 R /OCProperties << /OCGs [6 0 R 7 0 R] /D << /OFF [6 0 R] /Order [9 0 R 9 0 R] >> >> >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >>".to_vec(),
stream("", b"BT /F1 10 Tf 10 100 Td (Kept) Tj ET"),
FONT.replace("95 0 R", "8 0 R").into_bytes(),
b"<< /Type /OCG /Name (SECRETLAYERNAME) >>".to_vec(),
b"<< /Type /OCG /Name (Shown layer) >>".to_vec(),
widths(),
];
// Objects 9 to 22 each name the next one twice; 23 names both layers.
objs.extend((10..24).map(|n| format!("[{n} 0 R {n} 0 R]").into_bytes()));
objs.push(b"[6 0 R 7 0 R]".to_vec());
let mut doc = pdf(objs);
remove_hidden(&mut doc, &[Hidden::HiddenLayers]).unwrap();
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
let has = |needle: &[u8]| bytes.windows(needle.len()).any(|w| w == needle);
assert!(has(b"(Shown layer)") && !has(b"SECRETLAYERNAME"), "the removed layer survived the save");
assert!(bytes.len() < 32 * 1024, "the shared lists were copied once per path: {} bytes", bytes.len());
}
/// A removed off layer leaves every list that names it, or its dictionary, name included, stays in
/// the file: a sub-list of the default `/Order` (its heading too, once it names no layer), a
/// radio-button group (a group left empty goes), a usage rule (one that names it directly too), and
/// the alternate configurations in `/Configs`.
#[test]
fn a_removed_layer_leaves_every_configuration() {
use crate::sanitize::{Hidden, remove_hidden};
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R /OCProperties << /OCGs [6 0 R 7 0 R] /D << /OFF [6 0 R] /Order [7 0 R [(Group) 6 0 R]] /RBGroups [[6 0 R 7 0 R] [6 0 R]] /AS [<< /Event /Print /OCGs [6 0 R] /Category [/Print] >> << /Event /View /OCGs 6 0 R /Category [/View] >>] >> /Configs [8 0 R] >> >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> /Properties << /L1 6 0 R >> >> >>".to_vec(),
stream("", b"/OC /L1 BDC BT /F1 10 Tf 10 50 Td (Gone) Tj ET EMC BT /F1 10 Tf 10 100 Td (Kept) Tj ET"),
FONT.replace("95 0 R", "9 0 R").into_bytes(),
b"<< /Type /OCG /Name (SECRETLAYERNAME) >>".to_vec(),
b"<< /Type /OCG /Name (Shown layer) >>".to_vec(),
b"<< /Name (Other view) /OFF [7 0 R] /ON [6 0 R] /Order [6 0 R 7 0 R] >>".to_vec(),
widths(),
]);
remove_hidden(&mut doc, &[Hidden::HiddenLayers]).unwrap();
let bytes = write_full(&doc, &SaveOptions { object_streams: false, ..SaveOptions::default() }).unwrap();
let has = |needle: &[u8]| bytes.windows(needle.len()).any(|w| w == needle);
assert!(has(b"(Shown layer)") && has(b"(Other view)"), "the layer that stays and the configuration stay");
assert!(!has(b"SECRETLAYERNAME"), "the removed layer survived the save");
assert!(!has(b"(Group)"), "the heading of a sub-list left with no layer stayed");
let catalog = doc.get(doc.root().unwrap());
let config = catalog.as_dict().and_then(|c| c.get(b"OCProperties")).and_then(Object::as_dict).and_then(|p| p.get(b"D")).and_then(Object::as_dict);
let group = Object::Array(vec![Object::Ref(ObjRef::new(7, 0))]);
assert_eq!(config.and_then(|d| d.get(b"RBGroups")), Some(&Object::Array(vec![group])), "a radio-button group left empty stayed");
}
/// A layer the default configuration lists on but a View usage rule turns off is hidden on screen
/// (ISO 32000-2, 8.11.4.4): Hidden layers removes it. A Print rule leaves its layer as it is.
#[test]
fn a_layer_a_view_rule_turns_off_is_hidden() {
use crate::sanitize::{Hidden, remove_hidden, scan};
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R /OCProperties << /OCGs [6 0 R 7 0 R] /D << /AS [<< /Event /View /OCGs [6 0 R] /Category [/View] >> << /Event /Print /OCGs [7 0 R] /Category [/Print] >>] >> >> >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> /Properties << /L1 6 0 R /L2 7 0 R >> >> >>".to_vec(),
stream("", b"/OC /L1 BDC BT /F1 10 Tf 10 50 Td (ScreenOff) Tj ET EMC /OC /L2 BDC BT /F1 10 Tf 10 100 Td (PrintOff) Tj ET EMC"),
FONT.replace("95 0 R", "8 0 R").into_bytes(),
b"<< /Type /OCG /Name (Screen) /Usage << /View << /ViewState /OFF >> >> >>".to_vec(),
b"<< /Type /OCG /Name (Print) /Usage << /Print << /PrintState /OFF >> >> >>".to_vec(),
widths(),
]);
assert!(scan(&doc).into_iter().any(|(h, n)| h == Hidden::HiddenLayers && n > 0));
remove_hidden(&mut doc, &[Hidden::HiddenLayers]).unwrap();
let c = content(&doc, 0);
assert!(!c.contains("ScreenOff") && c.contains("(PrintOff)"), "{c}");
}
crates/redact/src/tests/surviving_references.rs
+261
View File
@@ -342,3 +342,264 @@ fn a_type3_glyphs_form_with_non_dictionary_resources_fails_closed() {
mark(&mut doc, 0, &[[9.0, 95.0, 14.9, 110.0]], "");
assert!(apply(&mut doc, None).is_err());
}
/// A Type 3 font with glyphs A, B and C, whose programs are objects `a`, `a + 1` and `a + 2`.
fn type3_abc(a: u32) -> Vec<u8> {
let (b, c) = (a + 1, a + 2);
format!(
"<< /Type /Font /Subtype /Type3 /FontBBox [0 -200 1000 800] /FontMatrix [0.001 0 0 0.001 0 0] /FirstChar 65 /LastChar 67 /Widths [500 500 500] /Encoding << /Differences [65 /A /B /C] >> /CharProcs << /A {a} 0 R /B {b} 0 R /C {c} 0 R >> >>"
)
.into_bytes()
}
/// An annotation's appearance shows glyph B of a Type 3 font the page lists too. The appearance
/// still draws it, so its program is kept; the redacted glyph A goes.
#[test]
fn a_type3_glyph_an_annotation_appearance_shows_is_kept() {
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources 5 0 R /Annots [7 0 R] >>".to_vec(),
stream("", b"BT /T3 10 Tf 10 100 Td (AC) Tj ET"),
b"<< /Font << /T3 6 0 R >> >>".to_vec(),
type3_abc(9),
b"<< /Type /Annot /Subtype /FreeText /Rect [150 200 250 220] /P 3 0 R /DA (/T3 10 Tf 0 g) /AP << /N 8 0 R >> >>".to_vec(),
stream("/Type /XObject /Subtype /Form /BBox [0 0 100 20] /Resources << /Font << /T3 6 0 R >> >>", b"BT /T3 10 Tf 2 5 Td (B) Tj ET"),
stream("", b"% REDACTED_GLYPH_A"),
stream("", b"% APPEARANCE_GLYPH_B"),
stream("", b"% PAGE_GLYPH_C"),
]);
mark(&mut doc, 0, &[[9.0, 95.0, 14.9, 110.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().glyphs, 1);
assert!(!full_save_holds(&doc, b"REDACTED_GLYPH_A"), "the redacted glyph program survived");
assert!(full_save_holds(&doc, b"APPEARANCE_GLYPH_B"), "the glyph the appearance shows was unlinked");
assert!(full_save_holds(&doc, b"PAGE_GLYPH_C"), "the glyph the page still shows was unlinked");
}
/// The same annotation also has an `/AP` entry no reader draws (`/X`), which shows the redacted
/// glyph A. It keeps nothing: A's program goes, and B, which the normal appearance shows, stays.
#[test]
fn a_type3_glyph_only_an_undrawn_appearance_entry_shows_is_unlinked() {
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources 5 0 R /Annots [7 0 R] >>".to_vec(),
stream("", b"BT /T3 10 Tf 10 100 Td (AC) Tj ET"),
b"<< /Font << /T3 6 0 R >> >>".to_vec(),
type3_abc(10),
b"<< /Type /Annot /Subtype /FreeText /Rect [150 200 250 220] /P 3 0 R /DA (/T3 10 Tf 0 g) /AP << /N 8 0 R /X 9 0 R >> >>".to_vec(),
stream("/Type /XObject /Subtype /Form /BBox [0 0 100 20] /Resources << /Font << /T3 6 0 R >> >>", b"BT /T3 10 Tf 2 5 Td (B) Tj ET"),
stream("/Type /XObject /Subtype /Form /BBox [0 0 100 20] /Resources << /Font << /T3 6 0 R >> >>", b"BT /T3 10 Tf 2 5 Td (A) Tj ET"),
stream("", b"% REDACTED_GLYPH_A"),
stream("", b"% APPEARANCE_GLYPH_B"),
stream("", b"% PAGE_GLYPH_C"),
]);
mark(&mut doc, 0, &[[9.0, 95.0, 14.9, 110.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().glyphs, 1);
assert!(!full_save_holds(&doc, b"REDACTED_GLYPH_A"), "an undrawn appearance entry kept the redacted glyph program");
assert!(full_save_holds(&doc, b"APPEARANCE_GLYPH_B"), "the glyph the normal appearance shows was unlinked");
assert!(full_save_holds(&doc, b"PAGE_GLYPH_C"), "the glyph the page still shows was unlinked");
}
/// An appearance without /Resources shows glyph B of a second Type 3 font, which the page lists
/// but its content never selects. The appearance draws through the page's resources, so the
/// rewritten page still lists that font and B's program stays.
#[test]
fn a_type3_font_only_an_appearance_without_resources_selects_stays_on_the_page() {
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources 5 0 R /Annots [8 0 R] >>".to_vec(),
stream("", b"BT /T3 10 Tf 10 100 Td (AC) Tj ET"),
b"<< /Font << /T3 6 0 R /T4 7 0 R >> >>".to_vec(),
type3_abc(10),
type3_abc(13),
b"<< /Type /Annot /Subtype /FreeText /Rect [150 200 250 220] /P 3 0 R /DA (/T4 10 Tf 0 g) /AP << /N 9 0 R >> >>".to_vec(),
stream("/Type /XObject /Subtype /Form /BBox [0 0 100 20]", b"BT /T4 10 Tf 2 5 Td (B) Tj ET"),
stream("", b"% REDACTED_GLYPH_A"),
stream("", b"% UNUSED_GLYPH_B"),
stream("", b"% PAGE_GLYPH_C"),
stream("", b"% UNUSED_T4_GLYPH_A"),
stream("", b"% APPEARANCE_GLYPH_B"),
stream("", b"% UNUSED_T4_GLYPH_C"),
]);
mark(&mut doc, 0, &[[9.0, 95.0, 14.9, 110.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().glyphs, 1);
assert!(!full_save_holds(&doc, b"REDACTED_GLYPH_A"), "the redacted glyph program survived");
assert!(full_save_holds(&doc, b"APPEARANCE_GLYPH_B"), "the glyph the appearance shows was unlinked");
assert!(saved_first_page_lists_font(&doc, b"T4"), "the page no longer lists the font the appearance selects");
}
/// The operators of a page's appearances count toward one total, as the content's do. Two
/// appearances under the cap alone pass it together, and the redaction is refused rather than
/// judged on part of what draws through the page's fonts.
#[test]
fn a_page_s_appearances_share_one_operator_total() {
let big = || stream("/Type /XObject /Subtype /Form /BBox [0 0 100 20]", b"n\n".repeat(1_100_000));
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources 5 0 R /Annots [7 0 R 8 0 R] >>".to_vec(),
stream("", b"BT /T3 10 Tf 10 100 Td (AC) Tj ET"),
b"<< /Font << /T3 6 0 R >> >>".to_vec(),
type3_abc(11),
b"<< /Type /Annot /Subtype /FreeText /Rect [150 200 250 220] /P 3 0 R /AP << /N 9 0 R >> >>".to_vec(),
b"<< /Type /Annot /Subtype /FreeText /Rect [150 240 250 260] /P 3 0 R /AP << /N 10 0 R >> >>".to_vec(),
big(),
big(),
stream("", b"% GLYPH_A"),
stream("", b"% GLYPH_B"),
stream("", b"% GLYPH_C"),
]);
mark(&mut doc, 0, &[[9.0, 95.0, 14.9, 110.0]], "");
let refused = apply(&mut doc, None);
assert!(refused.is_err(), "two appearances over the operator cap together were judged in part: {refused:?}");
}
/// Their decoded bytes share one page's allowance too. Two compressed appearances, each under it
/// but past it together (and blank, so far from the operator cap), refuse the redaction rather
/// than being decoded and held in full.
#[test]
fn a_page_s_appearances_share_one_decoded_byte_allowance() {
let packed = pdfcraft_filters::encode_flate(&vec![b' '; pdfcraft_cos::MAX_PAGE_DECODED / 2 + 1]);
let big = || stream("/Type /XObject /Subtype /Form /BBox [0 0 100 20] /Filter /FlateDecode", packed.as_slice());
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources 5 0 R /Annots [7 0 R 8 0 R] >>".to_vec(),
stream("", b"BT /T3 10 Tf 10 100 Td (AC) Tj ET"),
b"<< /Font << /T3 6 0 R >> >>".to_vec(),
type3_abc(11),
b"<< /Type /Annot /Subtype /FreeText /Rect [150 200 250 220] /P 3 0 R /AP << /N 9 0 R >> >>".to_vec(),
b"<< /Type /Annot /Subtype /FreeText /Rect [150 240 250 260] /P 3 0 R /AP << /N 10 0 R >> >>".to_vec(),
big(),
big(),
stream("", b"% GLYPH_A"),
stream("", b"% GLYPH_B"),
stream("", b"% GLYPH_C"),
]);
mark(&mut doc, 0, &[[9.0, 95.0, 14.9, 110.0]], "");
let refused = apply(&mut doc, None);
assert!(refused.is_err(), "two appearances past the page's decoded-byte allowance together were read in full: {refused:?}");
}
/// Whether the first page of a full save of `doc` lists the font `name` in its resources.
fn saved_first_page_lists_font(doc: &Document, name: &[u8]) -> bool {
let bytes = write_full(doc, &SaveOptions::default()).unwrap();
let saved = Document::open(Arc::new(bytes)).unwrap();
let page = pdfcraft_model::pages(&saved).swap_remove(0);
let res = saved.resolve(page.dict.get(b"Resources").unwrap()).as_dict().unwrap().clone();
res.get(b"Font").and_then(|f| saved.resolve(f).as_dict().map(|fonts| fonts.contains(name))).unwrap_or(false)
}
/// The group of a soft mask without /Resources shows glyph B of a second Type 3 font, which the
/// page lists but its content never selects. The group draws through the page's resources, so the
/// rewritten page still lists that font and B's program stays.
#[test]
fn a_type3_font_only_a_soft_mask_without_resources_selects_stays_on_the_page() {
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources 5 0 R >>".to_vec(),
stream("", b"BT /T3 10 Tf 10 100 Td (AC) Tj ET /GS1 gs 150 150 100 100 re f"),
b"<< /Font << /T3 6 0 R /T4 7 0 R >> /ExtGState << /GS1 << /Type /ExtGState /SMask << /Type /Mask /S /Luminosity /G 8 0 R >> >> >> >>"
.to_vec(),
type3_abc(9),
type3_abc(12),
stream(
"/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Group << /S /Transparency /CS /DeviceGray >>",
b"1 g BT /T4 40 Tf 160 180 Td (B) Tj ET",
),
stream("", b"% REDACTED_GLYPH_A"),
stream("", b"% UNUSED_GLYPH_B"),
stream("", b"% PAGE_GLYPH_C"),
stream("", b"% UNUSED_T4_GLYPH_A"),
stream("", b"% MASK_GLYPH_B"),
stream("", b"% UNUSED_T4_GLYPH_C"),
]);
mark(&mut doc, 0, &[[9.0, 95.0, 14.9, 110.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().glyphs, 1);
assert!(!full_save_holds(&doc, b"REDACTED_GLYPH_A"), "the redacted glyph program survived");
assert!(full_save_holds(&doc, b"MASK_GLYPH_B"), "the glyph the soft mask shows was unlinked");
assert!(saved_first_page_lists_font(&doc, b"T4"), "the page no longer lists the font the soft mask selects");
}
/// The cell of a tiling pattern another page fills with shows glyph B: its program is kept. (On
/// the redacted page itself, a pattern that shows text refuses the redaction: `pattern_guard`.)
#[test]
fn a_type3_glyph_a_tiling_pattern_on_another_page_shows_is_kept() {
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R 6 0 R] /Count 2 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources 5 0 R >>".to_vec(),
stream("", b"BT /T3 10 Tf 10 100 Td (AC) Tj ET"),
b"<< /Font << /T3 9 0 R >> >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 7 0 R /Resources 8 0 R >>".to_vec(),
stream("", b"/Pattern cs /P1 scn 200 200 50 50 re f"),
b"<< /Pattern << /P1 10 0 R >> >>".to_vec(),
type3_abc(11),
stream(
"/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 20 20] /XStep 20 /YStep 20 /Resources << /Font << /T3 9 0 R >> >>",
b"BT /T3 10 Tf 2 5 Td (B) Tj ET",
),
stream("", b"% REDACTED_GLYPH_A"),
stream("", b"% PATTERN_GLYPH_B"),
stream("", b"% PAGE_GLYPH_C"),
]);
mark(&mut doc, 0, &[[9.0, 95.0, 14.9, 110.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().glyphs, 1);
assert!(!full_save_holds(&doc, b"REDACTED_GLYPH_A"), "the redacted glyph program survived");
assert!(full_save_holds(&doc, b"PATTERN_GLYPH_B"), "the glyph the pattern shows was unlinked");
assert!(full_save_holds(&doc, b"PAGE_GLYPH_C"), "the glyph the page still shows was unlinked");
}
/// The group of a soft mask the page sets shows glyph B: its program is kept.
#[test]
fn a_type3_glyph_a_soft_mask_shows_is_kept() {
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources 5 0 R >>".to_vec(),
stream("", b"BT /T3 10 Tf 10 100 Td (AC) Tj ET /GS1 gs 150 150 100 100 re f"),
b"<< /Font << /T3 6 0 R >> /ExtGState << /GS1 << /Type /ExtGState /SMask << /Type /Mask /S /Luminosity /G 7 0 R >> >> >> >>".to_vec(),
type3_abc(8),
stream(
"/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Group << /S /Transparency /CS /DeviceGray >> /Resources << /Font << /T3 6 0 R >> >>",
b"1 g BT /T3 40 Tf 160 180 Td (B) Tj ET",
),
stream("", b"% REDACTED_GLYPH_A"),
stream("", b"% MASK_GLYPH_B"),
stream("", b"% PAGE_GLYPH_C"),
]);
mark(&mut doc, 0, &[[9.0, 95.0, 14.9, 110.0]], "");
assert_eq!(apply(&mut doc, None).unwrap().glyphs, 1);
assert!(!full_save_holds(&doc, b"REDACTED_GLYPH_A"), "the redacted glyph program survived");
assert!(full_save_holds(&doc, b"MASK_GLYPH_B"), "the glyph the soft mask shows was unlinked");
assert!(full_save_holds(&doc, b"PAGE_GLYPH_C"), "the glyph the page still shows was unlinked");
}
/// Resource cleanup read a form without /Resources of its own again for every name that could
/// still reach the removed image. Here 3 000 names of one redacted image sit in the inherited
/// /Pages resources, and the other page draws a large such form: it is read once, and every name
/// leaves with the image.
#[test]
fn many_names_of_a_redacted_image_are_unlinked_reading_inherited_forms_once() {
const ALIASES: usize = 3_000;
let aliases: String = (0..ALIASES).map(|i| format!("/A{i} 6 0 R ")).collect();
let mut doc = pdf(vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 /Resources 5 0 R >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 8 0 R >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 9 0 R >>".to_vec(),
format!("<< /XObject << /Im 6 0 R /Fm 7 0 R {aliases}>> >>").into_bytes(),
stream("/Type /XObject /Subtype /Image /Width 11 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", b"IMAGESECRET"),
stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300]", b"200 200 m 210 210 l S ".repeat(20_000)),
stream("", b"q 100 0 0 100 0 0 cm /Im Do Q"),
stream("", b"/Fm Do"),
]);
mark(&mut doc, 0, &[[0.0, 0.0, 120.0, 120.0]], "");
let started = std::time::Instant::now();
assert_eq!(apply(&mut doc, None).unwrap().images_removed, 1);
assert!(started.elapsed() < std::time::Duration::from_secs(60), "{:?}", started.elapsed());
assert!(!full_save_holds(&doc, b"IMAGESECRET"), "an alias kept the removed image reachable");
}
crates/redact/src/type3.rs
+171 -11
View File
@@ -308,6 +308,17 @@ fn scan_content_stream(
scan_xobject(ctx, res, holder_ref, name, current_font, depth)?;
}
}
b"gs" => {
apply_text_op(ctx, res, &op, &mut current_font);
if let Some(name) = op.name(0) {
scan_soft_mask(ctx, res, holder_ref, name, depth)?;
}
}
b"scn" | b"SCN" => {
if let Some(name) = op.last_name() {
scan_pattern(ctx, res, holder_ref, name, depth)?;
}
}
_ => apply_text_op(ctx, res, &op, &mut current_font),
}
}
@@ -370,20 +381,145 @@ fn scan_xobject(
) -> Result<(), CosError> {
let xo = res_dict(ctx.doc, res, b"XObject");
let Some(r) = xo.get(name).and_then(Object::as_ref) else { return Ok(()) };
let obj = ctx.doc.get(r);
let Object::Stream(s) = &*obj else { return Ok(()) };
if s.dict.name(b"Subtype") != Some(b"Form") {
if !matches!(&*ctx.doc.get(r), Object::Stream(s) if s.dict.name(b"Subtype") == Some(b"Form")) {
return Ok(());
}
let form_own_res = s.dict.get(b"Resources").and_then(|ro| ctx.doc.resolve(ro).as_dict().cloned());
// A form with resources of its own draws through them; one without draws through its caller's.
let through = if form_own_res.is_some() { r } else { holder_ref };
scan_stream_object(ctx, res, holder_ref, r, current_font, depth)
}
/// Scan the tiling pattern `name` that an `scn` or `SCN` operator selects. Its cell is content of
/// its own and can show glyphs of a font the page lists too. A shading pattern is not a stream and
/// draws no text.
fn scan_pattern(ctx: &mut ScanContext<'_>, res: &Dict, holder_ref: ObjRef, name: &[u8], depth: usize) -> Result<(), CosError> {
let patterns = res_dict(ctx.doc, res, b"Pattern");
let Some(r) = patterns.get(name).and_then(Object::as_ref) else { return Ok(()) };
scan_stream_object(ctx, res, holder_ref, r, None, depth)
}
/// Scan the transparency group of the soft mask that the ExtGState `name` sets: what it shows
/// shapes the page, so its glyphs are in use.
fn scan_soft_mask(ctx: &mut ScanContext<'_>, res: &Dict, holder_ref: ObjRef, name: &[u8], depth: usize) -> Result<(), CosError> {
let states = res_dict(ctx.doc, res, b"ExtGState");
let Some(state) = states.get(name).and_then(|s| ctx.doc.resolve(s).as_dict().cloned()) else { return Ok(()) };
let Some(mask) = state.get(b"SMask").and_then(|m| ctx.doc.resolve(m).as_dict().cloned()) else { return Ok(()) };
let Some(group) = mask.get(b"G").and_then(Object::as_ref) else { return Ok(()) };
scan_stream_object(ctx, res, holder_ref, group, None, depth)
}
/// Scan the appearance streams of a page's annotations. A comment, stamp or field can show glyphs
/// of a Type 3 font that the page lists too. An appearance without resources of its own draws
/// through the page's.
fn scan_appearances(ctx: &mut ScanContext<'_>, page: &Dict, page_ref: ObjRef, page_res: &Dict) -> Result<(), CosError> {
for annot in crate::annots_of(ctx.doc, page) {
for r in appearance_streams(ctx.doc, &annot) {
scan_stream_object(ctx, page_res, page_ref, r, None, 0)?;
}
}
Ok(())
}
/// The streams an annotation can show: its normal, rollover and down appearances (ISO 32000-2
/// §12.5.5), each one stream or a dictionary of streams by appearance state. Readers draw no
/// other `/AP` entry, so one can't keep a glyph the page no longer shows.
fn appearance_streams(doc: &Document, annot: &Object) -> Vec<ObjRef> {
let Some(ap) = doc.resolve(annot).as_dict().and_then(|a| a.get(b"AP").cloned()) else { return Vec::new() };
let Some(ap) = doc.resolve(&ap).as_dict().cloned() else { return Vec::new() };
let mut streams = Vec::new();
for key in [b"N", b"R", b"D"] {
let Some(entry) = ap.get(key) else { continue };
match &*doc.resolve(entry) {
Object::Dict(states) => streams.extend(states.iter().filter_map(|(_, s)| s.as_ref())),
_ => streams.extend(entry.as_ref()),
}
}
streams
}
/// The appearance streams of a page's annotations that have no resources of their own: they draw
/// through the page's, as [`scan_stream_object`] reads them.
fn resourceless_appearances(doc: &Document, page: &Dict) -> Vec<ObjRef> {
let mut seen = HashSet::new();
let mut out = Vec::new();
for annot in crate::annots_of(doc, page) {
for r in appearance_streams(doc, &annot) {
if !seen.insert(r) {
continue;
}
if let Object::Stream(s) = &*doc.get(r)
&& s.dict.get(b"Resources").is_none_or(|ro| doc.resolve(ro).as_dict().is_none())
{
out.push(r);
}
}
}
out
}
/// Add to `ops` those of the soft mask groups without resources of their own that `ops` set
/// (`gs`), and of the forms and further ones those draw in turn: they draw through `res`, as
/// [`scan_stream_object`] reads them. The forms that the first `known` operators draw are in `ops`
/// already. Tiling patterns are not read here: a tiling pattern must have resources of its own,
/// so its cell draws through them, and Apply Redactions refuses one without them
/// (`pattern_guard`).
fn add_soft_mask_ops(doc: &Document, res: &Dict, ops: &mut Vec<pdfcraft_content::Op>, known: usize) -> Result<(), CosError> {
let xobjects = res_dict(doc, res, b"XObject");
let states = res_dict(doc, res, b"ExtGState");
let mut seen = HashSet::new();
let mut cursor = 0;
while let Some(op) = ops.get(cursor) {
let selected = match op.op.as_slice() {
b"gs" => op
.name(0)
.and_then(|n| states.get(n))
.and_then(|s| doc.resolve(s).as_dict().and_then(|s| s.get(b"SMask").cloned()))
.and_then(|m| doc.resolve(&m).as_dict().and_then(|m| m.get(b"G").and_then(Object::as_ref))),
b"Do" if cursor >= known => op
.last_name()
.and_then(|n| xobjects.get(n))
.and_then(Object::as_ref)
.filter(|r| matches!(&*doc.get(*r), Object::Stream(s) if s.dict.name(b"Subtype") == Some(b"Form"))),
_ => None,
};
cursor += 1;
let Some(r) = selected else { continue };
if !seen.insert(r) {
continue;
}
let obj = doc.get(r);
let Object::Stream(s) = &*obj else { continue };
if s.dict.get(b"Resources").is_some_and(|ro| doc.resolve(ro).as_dict().is_some()) {
continue;
}
let parsed = pdfcraft_content::parse(&s.decoded()?);
if parsed.skipped > 0 || ops.len().saturating_add(parsed.ops.len()) > 2_000_000 {
return Err(CosError::Syntax { offset: 0, detail: "redaction could not fully inspect soft mask content".into() });
}
ops.extend(parsed.ops);
}
Ok(())
}
/// Scan a content stream object (a form, a pattern cell, a soft mask group or an appearance), once
/// per (stream, font, drawing holder). One with resources of its own draws through them; one
/// without draws through its caller's.
fn scan_stream_object(
ctx: &mut ScanContext<'_>,
res: &Dict,
holder_ref: ObjRef,
r: ObjRef,
current_font: Option<FontKey>,
depth: usize,
) -> Result<(), CosError> {
let obj = ctx.doc.get(r);
let Object::Stream(s) = &*obj else { return Ok(()) };
let own_res = s.dict.get(b"Resources").and_then(|ro| ctx.doc.resolve(ro).as_dict().cloned());
let through = if own_res.is_some() { r } else { holder_ref };
if !ctx.seen_forms.insert((r, current_font, through)) {
return Ok(());
}
let data = s.decoded()?;
match form_own_res {
Some(form_res) => scan_content_stream(ctx, std::slice::from_ref(&data), r, &form_res, current_font, depth + 1),
match own_res {
Some(own) => scan_content_stream(ctx, std::slice::from_ref(&data), r, &own, current_font, depth + 1),
None => scan_content_stream(ctx, std::slice::from_ref(&data), holder_ref, res, current_font, depth + 1),
}
}
@@ -396,8 +532,9 @@ fn gs_font_object(doc: &Document, state_obj: &Object) -> Option<Object> {
}
/// Prune Type 3 fonts from a rewritten page or form's `/Resources` dictionary if the rewritten
/// content no longer draws any text using that font name.
pub(crate) fn prune_type3_from_resources(doc: &Document, res: &mut Dict, content: &[Vec<u8>]) -> Result<(), CosError> {
/// content no longer draws any text using that font name. `page` is the page's dictionary when
/// `res` are a page's resources.
pub(crate) fn prune_type3_from_resources(doc: &Document, res: &mut Dict, content: &[Vec<u8>], page: Option<&Dict>) -> Result<(), CosError> {
let mut fonts = res_dict(doc, res, b"Font");
if fonts.is_empty() {
return Ok(());
@@ -405,7 +542,29 @@ pub(crate) fn prune_type3_from_resources(doc: &Document, res: &mut Dict, content
// The operators of forms that draw through these resources count too: dropping a font a child
// form still shows would leave its text without one.
let ops = crate::interp::resource_ops(doc, res, content)?;
let mut ops = crate::interp::resource_ops(doc, res, content)?;
// So do those of a page's annotation appearances, and of soft mask groups, without resources of
// their own. They are read only when there is a Type 3 font to keep, so other files are not
// refused over them.
if fonts.iter().any(|(_, f)| doc.resolve(f).as_dict().is_some_and(|d| d.name(b"Subtype") == Some(b"Type3"))) {
if let Some(page) = page {
// One total for all of them, as for the content: each appearance under the cap could
// still add up to far more than it. Their decoded bytes share one page's allowance
// too, and each is decoded only in its turn, so they are never all held at once.
let mut budget = pdfcraft_cos::DecodeBudget::page();
for r in resourceless_appearances(doc, page) {
let Object::Stream(s) = &*doc.get(r) else { continue };
let data = budget.decode(s)?;
let more = crate::interp::resource_ops(doc, res, std::slice::from_ref(&data))?;
if ops.len().saturating_add(more.len()) > 2_000_000 {
return Err(CosError::Syntax { offset: 0, detail: "redaction could not fully inspect annotation appearances".into() });
}
ops.extend(more);
}
}
let known = ops.len();
add_soft_mask_ops(doc, res, &mut ops, known)?;
}
let mut drawn_font_names = HashSet::new();
for op in &ops {
@@ -530,6 +689,7 @@ pub(crate) fn unlink_type3(doc: &mut Document) -> Result<(), RedactError> {
.unwrap_or_default();
scan_content_stream(&mut ctx, &content, p.obj, &page_res, None, 0)?;
scan_appearances(&mut ctx, &p.dict, p.obj, &page_res)?;
}
// Also scan any surviving form that wasn't reached by page streams directly.
crates/render/Cargo.toml
+3
View File
@@ -28,6 +28,9 @@ bytemuck = { version = "1" }
# Resolves relative URI actions against the catalog's /URI /Base, by the rules the browser uses;
# already in the tree (pdfcraft-engine checks document links with it).
url = "2.5"
# Tells combining marks (bidi class NSM: general categories Mn and Me) in extracted text; the
# version pdfcraft-fonts already uses.
unicode-bidi = "0.3"
# lopdf (bootstrap) draws random bytes for encryption; on the web they come from the browser.
[target.'cfg(target_arch = "wasm32")'.dependencies]
crates/render/src/inspect.rs
-4
View File
@@ -53,10 +53,6 @@ pub struct DocInfo {
pub keywords: Option<String>,
pub creator: Option<String>,
pub producer: Option<String>,
/// `/Info /CreationDate` and `/ModDate` for display ("2026-10-01 12:30", see [`pretty_date`]);
/// a value that isn't a PDF date is shown as written.
pub created: Option<String>,
pub modified: Option<String>,
/// Custom document properties: `/Info` entries with a key the spec does not name and a text
/// value, as (name, text), in the dictionary's order (at most 256).
pub custom: Vec<(String, String)>,
crates/render/src/inspect/annotations.rs
+66 -18
View File
@@ -464,22 +464,44 @@ impl<'a> Inspector<'a> {
let ft = self.name(d, b"FT").or(parent_ft);
let ff = d.get(b"Ff").ok().and_then(|x| x.as_i64().ok()).or(parent_ff);
// Non-terminal: has kids that are fields (they carry /T). Widgets-only kids are terminal.
let kids = match d.get(b"Kids").map(|k| self.resolve(k)) {
Ok(Object::Array(k)) => k.as_slice(),
_ => &[],
let mut kids: Vec<Object> = match d.get(b"Kids").map(|k| self.resolve(k)) {
Ok(Object::Array(k)) => k.clone(),
_ => Vec::new(),
};
let (field_kids, widget_kids): (Vec<&Object>, Vec<&Object>) = kids.iter().partition(|k| self.dict(k).is_some_and(|kd| kd.get(b"T").is_ok()));
let field_kids: Vec<_> = field_kids.into_iter().filter(|k| !self.not_a_child(walk, o, k)).collect();
let (field_kids, widget_kids): (Vec<Object>, Vec<Object>) =
kids.iter().cloned().partition(|k| self.dict(k).is_some_and(|kd| kd.get(b"T").is_ok()));
let field_kids: Vec<_> = field_kids.iter().filter(|k| !self.not_a_child(walk, o, k)).collect();
if !field_kids.is_empty() {
let id = if let Object::Reference(id) = o { Some(*id) } else { None };
walk.path.extend(id);
let here = match o {
Object::Reference(id) => Some(*id),
_ => None,
};
walk.path.extend(here);
for k in field_kids {
self.field(k, Some((&full, ft.clone(), ff)), out, walk, depth + 1);
// A fully qualified name is built up the field's own `/Parent` chain (ISO 32000-2
// §12.7.4.2): a kid that doesn't name this field as its `/Parent` is named from its
// own chain, as the form reader names it.
let parent = match self.dict(k).map(|kd| kd.get(b"Parent")) {
Some(Ok(Object::Reference(p))) => Some(*p),
_ => None,
};
if parent == here {
self.field(k, Some((&full, ft.clone(), ff)), out, walk, depth + 1);
} else {
let own = parent.map(|p| self.qualified_name(p)).unwrap_or_default();
self.field(k, Some((&own, ft.clone(), ff)), out, walk, depth + 1);
}
}
if id.is_some() {
if here.is_some() {
walk.path.pop();
}
return;
// "A field's children in the hierarchy may also include widget annotations" (ISO
// 32000-2 §12.7.4): widget annotations listed beside named kids are still this
// field's widgets, listed as a field of this name, as the form reader reads them.
kids.retain(|k| self.dict(k).is_some_and(|kd| kd.get(b"T").is_err() && self.name(kd, b"Subtype").as_deref() == Some("Widget")));
if kids.is_empty() {
return;
}
}
let flags = ff.unwrap_or(0);
let kind = match ft.as_deref() {
@@ -507,20 +529,23 @@ impl<'a> Inspector<'a> {
Ok(Object::Dictionary(_)) if kind == FieldKind::Signature => Some("signed".into()),
_ => None,
};
// The widget is either this dict (merged field/widget) or its first kid. With no widget
// among its kids, it is the first page widget naming this field as its `/Parent`: a child
// names its parent (ISO 32000-2 §12.7.4.1, Table 226), as MuPDF and PDFium read it.
// The widget is this dict (merged field/widget) or its first kid. A merged dict that also
// lists widget kids has them as its widgets (ISO 32000-2 §12.7.4), as the form reader has
// it. With no widget among its kids, it is the first page widget naming this field as its
// `/Parent`: a child names its parent (ISO 32000-2 §12.7.4.1, Table 226), as MuPDF and
// PDFium read it.
let merged = d.get(b"Rect").is_ok();
let named = if merged || !widget_kids.is_empty() { None } else { self.widget_naming(o) };
let widget = if merged {
Some(d)
} else {
widget_kids.first().and_then(|k| self.dict(k)).or_else(|| named.and_then(|id| self.doc.get_dictionary(id).ok()))
let widget_kid = widget_kids.iter().filter_map(|k| self.dict(k)).find(|kd| self.name(kd, b"Subtype").as_deref() == Some("Widget"));
let widget = match widget_kid {
Some(w) => Some(w),
None if merged => Some(d),
None => widget_kids.first().and_then(|k| self.dict(k)).or_else(|| named.and_then(|id| self.doc.get_dictionary(id).ok())),
};
let page = widget
.and_then(|w| w.get(b"P").ok())
.and_then(|p| self.page_of(p))
.or_else(|| self.find_widget_page(o, kids))
.or_else(|| self.find_widget_page(o, &kids))
.or_else(|| named.and_then(|id| self.annotation_pages.get_or_init(|| self.annotation_pages()).get(&id).copied()));
let has_actions = d.get(b"AA").is_ok() || widget.is_some_and(|w| w.get(b"AA").is_ok() || w.get(b"A").is_ok());
let rect = widget.and_then(|w| w.get(b"Rect").ok()).map(|r| rect4(self.resolve(r)));
@@ -589,6 +614,29 @@ impl<'a> Inspector<'a> {
pages
}
/// The fully qualified name of the field `id`: the partial names (`/T`) up its `/Parent`
/// chain, joined with periods; empty when none has one or the chain reaches an object that
/// isn't a dictionary, as the form reader has it. A `/Parent` loop ends the walk.
fn qualified_name(&self, id: ObjectId) -> String {
let mut parts = Vec::new();
let mut visited = HashSet::new();
let mut cur = Some(id);
while let Some(c) = cur {
if visited.len() > 64 || !visited.insert(c) {
break;
}
let o = Object::Reference(c);
let Some(d) = self.dict(&o) else { return String::new() };
parts.extend(self.text(d, b"T"));
cur = match d.get(b"Parent") {
Ok(Object::Reference(p)) => Some(*p),
_ => None,
};
}
parts.reverse();
parts.join(".")
}
/// The topmost field above widget `id` (itself without a `/Parent` that is a field), or `None` when it or an
/// ancestor is already reachable from the `/Fields` tree, or a `/Parent` loop hides the top.
pub(super) fn unlisted_field(&self, id: ObjectId, seen: &HashSet<ObjectId>) -> Option<ObjectId> {
crates/render/src/inspect/core.rs
-8
View File
@@ -188,14 +188,6 @@ impl<'a> Inspector<'a> {
info.keywords = self.text(d, b"Keywords");
info.creator = self.text(d, b"Creator");
info.producer = self.text(d, b"Producer");
// Dates are text strings only (ISO 32000-2 §7.9.4), as the engine reads them after an
// edit (`pdfcraft_organize::info`): a name value is no date.
let date = |key: &[u8]| match self.resolve(d.get(key).ok()?) {
Object::String(..) => self.text(d, key).map(|s| pretty_date(&s)),
_ => None,
};
info.created = date(b"CreationDate");
info.modified = date(b"ModDate");
const STANDARD: [&[u8]; 9] =
[b"Title", b"Author", b"Subject", b"Keywords", b"Creator", b"Producer", b"CreationDate", b"ModDate", b"Trapped"];
// The same rule as the engine's list after an edit (`pdfcraft_organize::custom_info`,
crates/render/src/inspect/tests.rs
+71 -25
View File
@@ -729,31 +729,6 @@ const CUSTOM_INFO: &[u8] = b"%PDF-1.7
trailer << /Root 1 0 R /Info 4 0 R >>
%%EOF";
/// One page whose `/Info` is `info` (the dictionary's contents).
fn with_info(info: &str) -> Vec<u8> {
format!(
"%PDF-1.7
1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj
2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 200 200] >> endobj
3 0 obj << /Type /Page /Parent 2 0 R >> endobj
4 0 obj << {info} >> endobj
trailer << /Root 1 0 R /Info 4 0 R >>
%%EOF"
)
.into_bytes()
}
#[test]
fn creation_and_modification_dates_are_read_for_document_properties() {
let info = inspect(Arc::new(with_info("/CreationDate (D:20240105093000+01'00') /ModDate (D:20261031174500Z)")), None).expect("opens");
assert_eq!((info.created.as_deref(), info.modified.as_deref()), (Some("2024-01-05 09:30"), Some("2026-10-31 17:45")));
// Not a PDF date: shown as written. Empty, a number, a name and absent: no date.
let info = inspect(Arc::new(with_info("/CreationDate (last Tuesday) /ModDate ()")), None).expect("opens");
assert_eq!((info.created.as_deref(), info.modified.as_deref()), (Some("last Tuesday"), None));
let info = inspect(Arc::new(with_info("/CreationDate 20240105 /ModDate /D:20240105093000Z")), None).expect("opens");
assert_eq!((info.created, info.modified), (None, None));
}
#[test]
fn custom_document_properties_are_listed_apart_from_the_standard_ones() {
let info = inspect(Arc::new(CUSTOM_INFO.to_vec()), None).expect("opens");
@@ -2294,6 +2269,77 @@ trailer << /Root 1 0 R >>
assert_eq!(listed, [("radio", Some(0), Some(120.0)), ("pair", Some(0), Some(90.0)), ("other", Some(0), Some(60.0))], "{:?}", info.fields);
}
/// A kid with `/T` that a field's `/Kids` lists but whose `/Parent` doesn't name that field is
/// named up its own `/Parent` chain (ISO 32000-2 §12.7.4.2), as the form reader names it.
#[test]
fn a_named_kid_without_a_parent_keeps_its_own_name() {
let pdf = |parent: &str| {
format!(
"%PDF-1.7
1 0 obj << /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [4 0 R] >> >> endobj
2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj
3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Annots [5 0 R 6 0 R] >> endobj
4 0 obj << /FT /Btn /Ff 49152 /T (Group1) /Kids [6 0 R 5 0 R] >> endobj
5 0 obj << /Type /Annot /Subtype /Widget /T (Group1){parent} /Rect [10 100 30 120] >> endobj
6 0 obj << /Type /Annot /Subtype /Widget /T (Group1){parent} /Rect [40 100 60 120] >> endobj
trailer << /Root 1 0 R >>
%%EOF"
)
.into_bytes()
};
let names = |bytes: Vec<u8>| inspect(Arc::new(bytes), None).expect("opens").fields.iter().map(|f| (f.name.clone(), f.kind)).collect::<Vec<_>>();
assert_eq!(names(pdf("")), [("Group1".into(), FieldKind::Radio), ("Group1".into(), FieldKind::Radio)]);
assert_eq!(names(pdf(" /Parent 4 0 R")), [("Group1.Group1".into(), FieldKind::Radio), ("Group1.Group1".into(), FieldKind::Radio)]);
}
/// A kid named from its own `/Parent` chain (ISO 32000-2 §12.7.4.2) when that `/Parent` is a field
/// other than the one listing it: an indirect one, and one listed as a direct dictionary in
/// `/Fields` (which no `/Parent` can name).
#[test]
fn a_named_kid_whose_parent_is_another_field_is_named_under_it() {
let pdf = |listing: &str, fields: &str| {
format!(
"%PDF-1.7
1 0 obj << /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [{fields}] >> >> endobj
2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj
3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Annots [5 0 R] >> endobj
4 0 obj {listing} endobj
5 0 obj << /Type /Annot /Subtype /Widget /T (k) /Parent 6 0 R /Rect [10 100 30 120] >> endobj
6 0 obj << /T (Other) /Kids [5 0 R] >> endobj
trailer << /Root 1 0 R >>
%%EOF"
)
.into_bytes()
};
let names = |bytes: Vec<u8>| inspect(Arc::new(bytes), None).expect("opens").fields.iter().map(|f| f.name.clone()).collect::<Vec<_>>();
let listing = "<< /FT /Tx /T (Listing) /Kids [5 0 R] >>";
assert_eq!(names(pdf(listing, "4 0 R")), ["Other.k"]);
assert_eq!(names(pdf("null", listing)), ["Other.k"], "listed as a direct dictionary");
}
/// "A field's children in the hierarchy may also include widget annotations" (ISO 32000-2
/// §12.7.4): a nameless widget beside a named kid is listed as a field of the parent's name, at
/// the widget's place, as the form reader lists it.
#[test]
fn a_widget_beside_a_named_kid_is_a_field_of_the_parents_name() {
let pdf = b"%PDF-1.7
1 0 obj << /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [4 0 R] >> >> endobj
2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj
3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Annots [5 0 R 6 0 R] >> endobj
4 0 obj << /FT /Btn /Ff 49152 /T (2) /Kids [5 0 R 6 0 R] >> endobj
5 0 obj << /Type /Annot /Subtype /Widget /T (2) /Rect [10 100 30 120] >> endobj
6 0 obj << /Type /Annot /Subtype /Widget /Parent 4 0 R /Rect [40 100 60 120] >> endobj
trailer << /Root 1 0 R >>
%%EOF"
.to_vec();
let info = inspect(Arc::new(pdf), None).expect("opens");
let got: Vec<_> = info.fields.iter().map(|f| (f.name.clone(), f.kind, f.rect)).collect();
assert_eq!(
got,
[("2".into(), FieldKind::Radio, Some([10.0, 100.0, 30.0, 120.0])), ("2".into(), FieldKind::Radio, Some([40.0, 100.0, 60.0, 120.0]))]
);
}
/// The password values (`/O`, `/U`, `/OE`, `/UE`, `/Perms`) of an AES-256 (R6) protection with a
/// user password, as a save writes them.
fn r6_password_values() -> String {
crates/render/src/text/layout.rs
+77 -9
View File
@@ -1,5 +1,7 @@
use super::device::{DIRECTION_TOLERANCE, Drawn, WORD_GAP};
use super::order::{NONE, REDRAW_CANDIDATES, Run, Scratch, Seg, fingerprint, is_script_on, place_glyphs, redraws, restarts_run, reverse_rtl};
use super::order::{
NONE, REDRAW_CANDIDATES, Run, Scratch, Seg, fingerprint, is_script_on, measure_cluster_gaps, place_glyphs, redraws, restarts_run, reverse_rtl,
};
use super::{HashMap, PageText, TextGlyph};
/// Glyphs in reading order, as indices into the page's glyphs, with their lines and word gaps.
@@ -97,11 +99,60 @@ pub(super) fn is_rtl(c: char) -> bool {
}
/// Scripts written with explicit spaces whose combining signs can sit apart from their base
/// (Indic, Thai, Lao, Myanmar, Khmer): never infer a space inside a tight cluster.
/// (Indic, Thai, Lao, Myanmar, Khmer): never infer a space next to such a sign in a tight cluster.
fn is_complex(c: char) -> bool {
matches!(c as u32, 0x0300..=0x036F | 0x0900..=0x0DFF | 0x0E00..=0x0EFF | 0x1000..=0x109F | 0x1780..=0x17FF)
}
/// Whether `text` is only combining marks (Unicode Mn and Me, which take bidi class NSM) of a
/// left-to-right script: Thai tone marks and above or below vowels, Indic signs such as virama,
/// Latin combining accents. Right-to-left marks are ordered with their script by [`reverse_rtl`].
fn is_mark(text: &str) -> bool {
!text.is_empty() && text.chars().all(|c| unicode_bidi::bidi_class(c) == unicode_bidi::BidiClass::NSM && !is_rtl(c))
}
/// How far from the glyph before it, as a fraction of its height, a combining mark may start and
/// still ride on it. A zero-advance mark starts at the pen after its base, give or take a small
/// mark-positioning offset, and its ink is drawn back over the base.
const MARK_REACH: f32 = 0.3;
/// Whether glyph `m`, drawn right after `p` in one run, is a combining mark on `p`: ISO 32000-2
/// §9.10 leaves extraction to the processor, and Unicode stores a mark after its base, as the
/// content stream draws it here. The mark then stays after `p` in reading order however far its
/// own box sits along the line; sorting by x alone moved a Thai tone mark that starts a hair past
/// its base's advance after the next letter (#6517).
fn rides_on(p: &TextGlyph, m: &TextGlyph) -> bool {
let reach = MARK_REACH * (m.rect[3] - m.rect[1]).max(0.1);
is_mark(&m.text)
&& p.text.chars().any(|c| !c.is_whitespace())
&& !p.text.chars().any(is_rtl)
&& m.rect[0] >= p.rect[0] - reach
&& m.rect[0] <= p.rect[2] + reach
}
/// How far, as a fraction of its letter's height, a combining mark's em box may sit above or below the letter's and
/// still ride on it. A tone mark lifted over an upper vowel can be drawn more than half an em higher, past where a
/// glyph of its own would start a new line.
const MARK_RISE: f32 = 1.0;
/// Whether glyph `m`, drawn right after glyph `p` (a letter, or a mark riding on letter `a`), rides on that cluster:
/// on `p` itself, or, for a mark stacked on another one, on its letter `a`.
fn rides_on_cluster(glyphs: &[&TextGlyph], p: usize, a: usize, m: &TextGlyph) -> bool {
glyphs.get(p).is_some_and(|pg| rides_on(pg, m)) || (a != p && glyphs.get(a).is_some_and(|ag| rides_on(ag, m)))
}
/// Glyph `i` continues the run of glyph `i - 1`: when it is a combining mark on that glyph's cluster, it joins it.
fn attach_mark(glyphs: &[&TextGlyph], anchor: &mut [usize], i: usize) {
let Some(p) = i.checked_sub(1) else { return };
let Some(&a) = anchor.get(p) else { return };
if let Some(mg) = glyphs.get(i)
&& rides_on_cluster(glyphs, p, a, mg)
&& let Some(slot) = anchor.get_mut(i)
{
*slot = a;
}
}
fn is_cjk(c: char) -> bool {
matches!(c as u32, 0x2E80..=0x9FFF | 0xAC00..=0xD7AF | 0xF900..=0xFAFF | 0xFF00..=0xFFEF | 0x20000..=0x3134F | 0x1100..=0x11FF)
}
@@ -173,13 +224,21 @@ fn arrange(glyphs: &[TextGlyph]) -> (Vec<usize>, Vec<u32>, Vec<bool>) {
// 1. Segments in content order. Every run is a range of consecutive glyphs.
let mut segs: Vec<Seg> = Vec::new();
// `anchor[i]`: the glyph a combining mark rides on (the first glyph of its cluster), or `i`.
let mut anchor: Vec<usize> = (0..n).collect();
let mut run_start_x = 0.0;
let mut furthest_progress = 0.0;
for i in 0..n {
let g = glyphs[i].rect;
let hi = height(i);
let cont = segs.last().is_some_and(|s| {
let Some(p) = s.first.end.checked_sub(1) else { return false };
let Some(last) = s.first.end.checked_sub(1) else { return false };
// A glyph after a combining mark continues from the mark's letter, and a mark on that letter's cluster
// stays on its line even when drawn raised or lowered by up to an em (`MARK_RISE`).
let p = anchor.get(last).copied().unwrap_or(last);
if rides_on_cluster(&glyphs, last, p, glyphs[i]) && (cy(i) - cy(p)).abs() < height(p) * MARK_RISE {
return true;
}
let ph = height(p).min(hi);
let sh = s.h.min(hi);
let gap = g[0] - glyphs[p].rect[2];
@@ -196,6 +255,7 @@ fn arrange(glyphs: &[TextGlyph]) -> (Vec<usize>, Vec<u32>, Vec<bool>) {
&& !reset
&& let Some(s) = segs.last_mut()
{
attach_mark(&glyphs, &mut anchor, i);
s.first.end = i + 1;
s.extend_box(&g, hi);
furthest_progress = furthest_progress.max((g[0] - run_start_x).abs()).max((g[2] - run_start_x).abs());
@@ -392,7 +452,7 @@ fn arrange(glyphs: &[TextGlyph]) -> (Vec<usize>, Vec<u32>, Vec<bool>) {
let mut line_of: Vec<u32> = Vec::with_capacity(n);
let mut space_before: Vec<bool> = Vec::with_capacity(n);
let mut line = 0u32;
let mut sc = Scratch::default();
let mut sc = Scratch { anchor, ..Default::default() };
// `spaces[w]`: a word gap left of the w-th glyph of the segment, by position on the line, not by drawing order
// (#1062). Reused by every segment.
let mut spaces: Vec<bool> = Vec::new();
@@ -400,12 +460,16 @@ fn arrange(glyphs: &[TextGlyph]) -> (Vec<usize>, Vec<u32>, Vec<bool>) {
for &si in &blocks[b].0 {
place_glyphs(&glyphs, &height, &segs[si], &mut sc);
let h = segs[si].h;
measure_cluster_gaps(&glyphs, &mut sc);
// The text of the cluster that glyph `i` belongs to: its base's.
let cluster_text = |i: usize| sc.anchor.get(i).and_then(|&a| glyphs.get(a)).map_or("", |g| g.text.as_str());
// Word gaps relative to this line's typical letter gap (handles tracking).
// Long runs of repeated punctuation belong to leaders, not letter tracking: a TOC leader
// must not raise the threshold above the word gaps in its title (#3086).
sc.gaps.clear();
sc.gaps.extend(sc.placed.windows(2).enumerate().filter_map(|(w, pair)| {
let &[p, c] = pair else { return None };
let gap = (*sc.cluster_gap.get(w + 1)?)?;
let (p, c) = (glyphs.get(p)?, glyphs.get(c)?);
if p.text == c.text && p.text.chars().all(|ch| !ch.is_alphanumeric() && !ch.is_whitespace()) {
// Keep short tracked punctuation, including an ellipsis with a final period.
@@ -418,7 +482,7 @@ fn arrange(glyphs: &[TextGlyph]) -> (Vec<usize>, Vec<u32>, Vec<bool>) {
return None;
}
}
Some(c.rect[0] - p.rect[2])
Some(gap)
}));
sc.gaps.sort_by(f32::total_cmp);
let typical = sc.gaps.get(sc.gaps.len() / 3).copied().unwrap_or(0.0).max(0.0);
@@ -427,10 +491,14 @@ fn arrange(glyphs: &[TextGlyph]) -> (Vec<usize>, Vec<u32>, Vec<bool>) {
spaces.resize(sc.placed.len(), false);
for (w, pair) in sc.placed.windows(2).enumerate() {
let &[p, c] = pair else { continue };
let gap = glyphs[c].rect[0] - glyphs[p].rect[2];
let (pt, ct) = (&glyphs[p].text, &glyphs[c].text);
let Some(&Some(gap)) = sc.cluster_gap.get(w + 1) else { continue };
let (pt, ct) = (cluster_text(p), cluster_text(c));
let cjk = pt.chars().any(is_cjk) && ct.chars().any(is_cjk) && gap < h * 0.5;
let tight_cluster = (pt.chars().any(is_complex) || ct.chars().any(is_complex)) && gap < h * 0.6;
// A combining sign that rides on no glyph before it (drawn apart from its base, or in a run of its own)
// may sit anywhere over its base, so it never takes a space next to it. Two base letters of these
// scripts are measured like any others: a Thai or Indic word gap is narrower than half an em.
let loose_sign = |t: &str| is_mark(t) && t.chars().any(is_complex);
let tight_cluster = (loose_sign(pt) || loose_sign(ct)) && gap < h * 0.6;
let run_start = sc.breaks.binary_search(&c).is_ok();
if let Some(space) = spaces.get_mut(w + 1) {
*space = (gap > threshold || run_start) && !cjk && !tight_cluster && !pt.trim().is_empty() && !ct.trim().is_empty();
@@ -441,7 +509,7 @@ fn arrange(glyphs: &[TextGlyph]) -> (Vec<usize>, Vec<u32>, Vec<bool>) {
if has_rtl {
sc.logical.clear();
sc.logical.extend_from_slice(&sc.placed);
reverse_rtl(&glyphs, &mut sc.logical, &mut spaces);
reverse_rtl(&glyphs, &sc.anchor, &mut sc.logical, &mut spaces);
}
let idx: &[usize] = if has_rtl { &sc.logical } else { &sc.placed };
order.extend_from_slice(idx);
crates/render/src/text/order.rs
+250 -14
View File
@@ -152,6 +152,56 @@ pub(super) struct Scratch {
pub(super) logical: Vec<usize>,
/// `placed` with the marks merged in, built before it is swapped in.
pub(super) merged: Vec<usize>,
/// For each glyph of the page, the glyph a combining mark drawn right after it rides on (the start of its cluster),
/// or the glyph itself. Placement sorts clusters, so a mark keeps its content-stream place after its base (#6517).
/// A glyph past its end is its own anchor.
pub(super) anchor: Vec<usize>,
/// The gap left of each glyph of `placed`, between clusters; `None` for the first glyph and for a riding mark.
pub(super) cluster_gap: Vec<Option<f32>>,
}
/// How close, as a fraction of its height, the glyph drawn after a combining mark must start to the mark's right edge
/// for the mark to have moved the pen by its own width ([`measure_cluster_gaps`]).
const MARK_ADVANCE_TOLERANCE: f32 = 0.05;
/// Fills `sc.cluster_gap` with the gap left of each glyph of `sc.placed`, between clusters: a combining mark riding on a
/// glyph has none (no word ever starts at it), and its cluster ends at the base's right edge. A zero-advance mark's box
/// is no measure of where the pen went (`draw_glyph` widens it to its outline or half an em), and its start is only
/// the base's advance moved by a mark-positioning offset. A mark whose font gives it an advance of its own does move
/// the pen, and the glyph drawn after it starts right at its box's right edge: the cluster then ends there, or every
/// letter after such a mark would start a word.
pub(super) fn measure_cluster_gaps(glyphs: &[&TextGlyph], sc: &mut Scratch) {
sc.cluster_gap.clear();
let mut right = f32::NEG_INFINITY;
for (w, &c) in sc.placed.iter().enumerate() {
let Some(g) = glyphs.get(c) else {
sc.cluster_gap.push(None);
continue;
};
if anchor_of(&sc.anchor, c) != c {
sc.cluster_gap.push(None);
let tolerance = MARK_ADVANCE_TOLERANCE * (g.rect[3] - g.rect[1]).max(0.1);
let pen_moved = c.checked_add(1).and_then(|n| glyphs.get(n)).is_some_and(|n| (n.rect[0] - g.rect[2]).abs() <= tolerance);
if pen_moved && g.rect[2] > right {
right = g.rect[2];
}
} else {
sc.cluster_gap.push((w > 0).then_some(g.rect[0] - right));
right = g.rect[2];
}
}
}
/// The glyph that glyph `i`'s cluster starts at ([`Scratch::anchor`]).
fn anchor_of(anchor: &[usize], i: usize) -> usize {
anchor.get(i).copied().unwrap_or(i)
}
/// Orders glyphs by the x of their cluster's first glyph, keeping each cluster together and in content order.
fn by_cluster(glyphs: &[&TextGlyph], anchor: &[usize], a: usize, b: usize) -> std::cmp::Ordering {
let (ka, kb) = (anchor_of(anchor, a), anchor_of(anchor, b));
let x = |i: usize| glyphs.get(i).map_or(0.0, |g| g.rect[0]);
x(ka).total_cmp(&x(kb)).then(ka.cmp(&kb)).then(a.cmp(&b))
}
/// One run of a colliding component.
@@ -243,7 +293,8 @@ pub(super) fn restarts_run(glyphs: &[&TextGlyph], start: usize, next: usize, pro
/// Glyphs are placed by x. Runs that collide, meaning that glyphs of different runs overprint each other, stay whole
/// and are placed by their left edge. Runs that only interleave along the line, such as symbols between letters, keep
/// their glyph-by-glyph order, so the common case is one sort. A run of zero-width glyphs (a combining mark) has no
/// extent of its own, so it goes next to the glyph it starts at.
/// extent of its own, so it goes next to the glyph it starts at. A space whose advance the next glyph takes back stays
/// before that glyph ([`keep_taken_back_blanks`]).
pub(super) fn place_glyphs(glyphs: &[&TextGlyph], height: &dyn Fn(usize) -> f32, s: &Seg, sc: &mut Scratch) {
let x0 = |i: usize| glyphs[i].rect[0];
let zero_width = |i: usize| glyphs[i].rect[2] - glyphs[i].rect[0] <= 0.05 * height(i);
@@ -261,18 +312,19 @@ pub(super) fn place_glyphs(glyphs: &[&TextGlyph], height: &dyn Fn(usize) -> f32,
if sc.text.len() <= 1 {
// A single run cannot collide with itself: its glyphs only need sorting by x.
sc.placed.extend(sc.text.iter().flat_map(|run| run.range()));
sc.placed.sort_unstable_by(|a, b| x0(*a).total_cmp(&x0(*b)).then(a.cmp(b)));
sc.placed.sort_unstable_by(|a, b| by_cluster(glyphs, &sc.anchor, *a, *b));
} else {
collide(glyphs, height, s.h, sc);
}
// Each mark goes before the first placed glyph whose x is greater than its own. Marks are visited in x order, and
// that position never moves back, so one pass over `placed` puts all of them in.
sc.marks.sort_by(|a, b| x0(*a).total_cmp(&x0(*b)));
// Each mark goes before the first placed glyph whose x is greater than its own, or its letter's when it is a
// combining mark riding on one. Marks are visited in that order, and that position never moves back, so one pass
// over `placed` puts all of them in.
sc.marks.sort_by(|a, b| by_cluster(glyphs, &sc.anchor, *a, *b));
if !sc.marks.is_empty() {
sc.merged.clear();
let mut next = 0;
for &m in &sc.marks {
while next < sc.placed.len() && x0(sc.placed[next]) <= x0(m) {
while next < sc.placed.len() && x0(anchor_of(&sc.anchor, sc.placed[next])) <= x0(anchor_of(&sc.anchor, m)) {
sc.merged.push(sc.placed[next]);
next += 1;
}
@@ -281,6 +333,123 @@ pub(super) fn place_glyphs(glyphs: &[&TextGlyph], height: &dyn Fn(usize) -> f32,
sc.merged.extend_from_slice(&sc.placed[next..]);
std::mem::swap(&mut sc.placed, &mut sc.merged);
}
// After the marks: their merge relies on `placed` being in x order, which this breaks on purpose.
keep_taken_back_blanks(glyphs, &mut sc.placed);
}
/// How many glyphs [`keep_taken_back_blanks`] looks at past a blank: marks between it and its letter, in drawing order
/// and in x order, and glyphs without a strong direction on each side of a taken-back neutral. Each scan stops there,
/// so the pass stays linear in the line's glyphs.
const TAKEN_BACK_SCAN: usize = 8;
/// Puts a blank glyph (a space) back before the glyph drawn right after it on a line with right-to-left text, when that
/// glyph's adjustment took the blank's advance back: it starts at or left of the blank's own start and covers at least
/// half of it. Right-to-left text drawn in reverse order starts a show string with the space that ends its word
/// (ISO 32000-2 §14.8.2.5.3, whose NOTE 2 has such a space realign the typography without visible effect), and a word
/// processor pulls the next letter back over it. A blank has no ink, so its box says nothing about which side of that
/// letter it belongs on, and sorting by x split "המדינה" as "המדינ ה". The space is a word break in the character
/// stream (§14.8.2.6.2), and drawing order is the only evidence of its place.
///
/// Combining marks (bidi class NSM) drawn between the blank and its letter are passed over, and the blank, those marks
/// and the letter keep drawing order, which puts each mark after its letter once the line is reversed, as Unicode
/// stores it. Only a right-to-left letter or a neutral (punctuation, a symbol) after the blank counts: left-to-right
/// text has no reverse-order show strings, and in scanned pages' invisible text layers its spaces overlap letters at
/// random. A neutral counts only inside right-to-left text: the nearest letter or number before it, and the nearest
/// after the blank, is right-to-left (numbers count as such beside neutrals), or the line's edge on a right-to-left
/// line. A parenthesis that opens Latin text then stays
/// where x order puts it, as on a line without right-to-left text. A glyph that only touches the blank's start was
/// drawn beside it, as text set letter by letter leftwards is, and x order already reads it right. A blank that a
/// glyph placed before them also reaches over sits inside a cluster (a letter and the marks drawn on it), where its
/// place is no clearer, so it stays where x order put it.
fn keep_taken_back_blanks(glyphs: &[&TextGlyph], placed: &mut [usize]) {
let glyph = |placed: &[usize], p: usize| placed.get(p).and_then(|&i| glyphs.get(i)).copied();
let blank = |g: &TextGlyph| !g.text.is_empty() && g.text.chars().all(char::is_whitespace);
let mark = |g: &TextGlyph| !g.text.is_empty() && g.text.chars().all(|c| unicode_bidi::bidi_class(c) == unicode_bidi::BidiClass::NSM);
// The direction a glyph gives the text around it: Some(false) for left-to-right letters, Some(true) for
// right-to-left letters and for numbers, which read as right-to-left beside neutrals (UAX #9 N1, as in
// `reverse_rtl`), None for a glyph without one. Nonspacing marks give none, alphabetic ones (U+0345) included: a
// mark has the direction of the letter it sits on.
let strong = |g: &TextGlyph| {
let spacing = || g.text.chars().filter(|&c| unicode_bidi::bidi_class(c) != unicode_bidi::BidiClass::NSM);
if spacing().any(|c| c.is_alphabetic() && !is_rtl(c)) {
Some(false)
} else if spacing().any(char::is_alphanumeric) {
Some(true)
} else {
None
}
};
if !placed.iter().any(|&i| glyphs.get(i).is_some_and(|g| g.text.chars().any(is_rtl))) {
return;
}
// The line's direction, as `reverse_rtl` takes it. A line's edge has that direction (UAX #9 X10: sos and eos).
let line_rtl = is_rtl_line(glyphs, placed.iter().copied());
// Whether the nearest glyph with a strong direction from position `from` on, stepping by `step`, is right-to-left,
// or the line is when there is none before its edge.
let rtl_side = |placed: &[usize], from: Option<usize>, step: fn(usize) -> Option<usize>| {
let mut p = from;
for _ in 0..TAKEN_BACK_SCAN {
let Some(g) = p.and_then(|p| glyph(placed, p)) else { return line_rtl };
if let Some(rtl) = strong(g) {
return rtl;
}
p = p.and_then(step);
}
false
};
for p in 1..placed.len() {
let Some(&space) = placed.get(p) else { continue };
let Some(s) = glyphs.get(space).copied() else { continue };
if !blank(s) {
continue;
}
// The glyph drawn after the blank, past any marks drawn between them.
let mut next = space.checked_add(1);
for _ in 0..TAKEN_BACK_SCAN {
match next.and_then(|i| glyphs.get(i)) {
Some(g) if mark(g) => next = next.and_then(|i| i.checked_add(1)),
_ => break,
}
}
let Some(ink) = next.filter(|&i| glyphs.get(i).is_some_and(|g| !mark(g))) else { continue };
// The glyphs x order put next to the blank that were drawn after it, up to and including the letter: the
// letter and the marks drawn between them, on either side of the blank.
let drawn_with = |i: usize| i > space && i <= ink;
let mut q = p;
for _ in 0..=TAKEN_BACK_SCAN {
match q.checked_sub(1).and_then(|k| placed.get(k)) {
Some(&i) if drawn_with(i) => q -= 1,
_ => break,
}
}
let mut r = p;
for _ in 0..=TAKEN_BACK_SCAN {
match r.checked_add(1).and_then(|k| placed.get(k)) {
Some(&i) if drawn_with(i) => r += 1,
_ => break,
}
}
if q == p || !placed.get(q..=r).is_some_and(|group| group.contains(&ink)) {
continue;
}
let Some(g) = glyphs.get(ink).copied() else { continue };
let width = s.rect[2] - s.rect[0];
let covered = g.rect[2].min(s.rect[2]) - s.rect[0];
let taken_back = g.rect[0] <= s.rect[0] && width > 0.0 && covered >= 0.5 * width;
let alone = q.checked_sub(1).and_then(|b| glyph(placed, b)).is_none_or(|before| before.rect[2] <= s.rect[0]);
// A left-to-right letter or a European digit after the blank never moves; a neutral moves inside right-to-left text.
let direction = match strong(g) {
_ if g.text.chars().any(|c| c.is_alphanumeric() && !is_rtl(c)) => false,
Some(rtl) => rtl,
None => rtl_side(placed, q.checked_sub(1), |p| p.checked_sub(1)) && rtl_side(placed, r.checked_add(1), |p| p.checked_add(1)),
};
if !blank(g) && direction && taken_back && alone {
// The blank, its marks and its letter in drawing order.
if let Some(group) = placed.get_mut(q..=r) {
group.sort_unstable();
}
}
}
}
/// `place_glyphs` for a segment of two or more text runs. `h` is the segment's line height. Returns whether the segment
@@ -296,17 +465,31 @@ pub(super) fn collide(glyphs: &[&TextGlyph], height: &dyn Fn(usize) -> f32, h: f
for (r, run) in sc.text.iter().enumerate() {
sc.owner.extend(run.range().map(|i| (i, r)));
}
sc.owner.sort_unstable_by(|a, b| x0(a.0).total_cmp(&x0(b.0)).then(a.0.cmp(&b.0)));
sc.owner.sort_unstable_by(|a, b| by_cluster(glyphs, &sc.anchor, a.0, b.0));
sc.parent.clear();
sc.parent.extend(0..runs);
let mut budget = COLLIDE_PAIRS_PER_GLYPH.saturating_mul(sc.owner.len());
let mut collided = false;
let mut dense = false;
// A mark riding on a glyph goes with it, so only cluster starts are swept. They are in x order in `owner`.
let rides = |i: usize| anchor_of(&sc.anchor, i) != i;
'sweep: for a in 0..sc.owner.len() {
let (i, ri) = sc.owner[a];
if rides(i) {
continue;
}
let (x1, hi) = (glyphs[i].rect[2], height(i));
for b in a + 1..sc.owner.len() {
let (j, rj) = sc.owner[b];
if rides(j) {
// Skipping counts against the budget, so a long string of marks cannot make the sweep quadratic.
if budget == 0 {
dense = true;
break 'sweep;
}
budget -= 1;
continue;
}
if x0(j) >= x1 {
break;
}
@@ -357,7 +540,7 @@ pub(super) fn collide(glyphs: &[&TextGlyph], height: &dyn Fn(usize) -> f32, h: f
for i in sc.text[head].range() {
let at = sc.flat.len();
sc.flat.push(i);
sc.units.push(Unit { left: x0(i), first: i, glyphs: at..at + 1 });
sc.units.push(Unit { left: x0(anchor_of(&sc.anchor, i)), first: i, glyphs: at..at + 1 });
}
continue;
}
@@ -368,8 +551,10 @@ pub(super) fn collide(glyphs: &[&TextGlyph], height: &dyn Fn(usize) -> f32, h: f
let run = sc.text[m];
let start = sc.group.len();
sc.group.extend(run.range());
sc.group[start..].sort_unstable_by(|a, b| x0(*a).total_cmp(&x0(*b)).then(a.cmp(b)));
let left = sc.group[start..].iter().map(|&i| x0(i)).fold(f32::INFINITY, f32::min);
sc.group[start..].sort_unstable_by(|a, b| by_cluster(glyphs, &sc.anchor, *a, *b));
// Ordered by where its clusters start: a mark riding on a glyph may reach left of it, but it has no place of
// its own, so it cannot put its run before one whose first letter stands further left.
let left = sc.group[start..].iter().map(|&i| x0(anchor_of(&sc.anchor, i))).fold(f32::INFINITY, f32::min);
let right = sc.group[start..].iter().map(|&i| glyphs[i].rect[2]).fold(f32::NEG_INFINITY, f32::max);
let lead = sc.group[start];
sc.rows.push(Row { left, right, first: run.start, lead, glyphs: start..sc.group.len() });
@@ -412,6 +597,11 @@ fn bidi_class(text: &str) -> Bidi {
Bidi::R
} else if text.chars().any(char::is_numeric) {
Bidi::Num
} else if is_nonspacing_mark(text) {
// A mark of a left-to-right script that rides on no letter (`resolve_marks`) has no direction of its own: NSM
// is a weak class (UAX #9 Table 4), so it reads with the glyphs around its drawn position, as a neutral does,
// even when it is alphabetic (U+0345, Other_Alphabetic).
Bidi::N
} else if text.chars().any(char::is_alphabetic) {
Bidi::L
} else {
@@ -419,11 +609,56 @@ fn bidi_class(text: &str) -> Bidi {
}
}
/// Whether a line of the glyphs `line` is right-to-left: it has more right-to-left letters than left-to-right ones.
/// Letters are counted one by one, so a ligature glyph ("ffi") counts each of its letters. Punctuation of a
/// right-to-left script (a maqaf, an Arabic comma) is not a letter and counts toward neither direction, nor do
/// nonspacing marks: a mark has the direction of the letter it sits on.
fn is_rtl_line(glyphs: &[&TextGlyph], line: impl Iterator<Item = usize>) -> bool {
let (mut rtl, mut ltr) = (0usize, 0usize);
for g in line.filter_map(|i| glyphs.get(i)) {
for c in g.text.chars().filter(|&c| unicode_bidi::bidi_class(c) != unicode_bidi::BidiClass::NSM) {
if is_rtl(c) && c.is_alphabetic() {
rtl = rtl.saturating_add(1);
} else if c.is_alphabetic() && !is_rtl(c) {
ltr = ltr.saturating_add(1);
}
}
}
rtl > ltr
}
/// Whether `text` is only nonspacing marks (bidi class NSM). Such a glyph counts toward neither direction of a line:
/// it has the direction of the letter it sits on.
fn is_nonspacing_mark(text: &str) -> bool {
!text.is_empty() && text.chars().all(|c| unicode_bidi::bidi_class(c) == unicode_bidi::BidiClass::NSM)
}
/// UAX #9 W1: a nonspacing mark takes the class of the letter it rides on ([`Scratch::anchor`]), which placement keeps
/// right in front of it. A mark that rides on nothing (one after a word gap, or a right-to-left mark, which placement
/// puts by its own position) keeps its own class.
fn resolve_marks(glyphs: &[&TextGlyph], anchor: &[usize], items: &mut [(Option<usize>, Bidi)]) {
let nsm = |i: usize| glyphs.get(i).is_some_and(|g| is_nonspacing_mark(&g.text));
for k in 1..items.len() {
let (prev, base) = match items.get(k - 1) {
Some(&(Some(prev), base)) => (prev, base),
_ => continue,
};
if let Some(it) = items.get_mut(k)
&& let Some(mark) = it.0
&& nsm(mark)
&& anchor_of(anchor, mark) != mark
&& anchor_of(anchor, mark) == anchor_of(anchor, prev)
{
it.1 = base;
}
}
}
/// Reorders a line of `idx` from visual to logical order with a simplified bidi level pass (#2667), carrying the
/// word gaps along: `spaces[w]` is a gap left of `idx[w]` (#1062). The paragraph is right-to-left when it has
/// more right-to-left than left-to-right glyphs; numbers keep their left-to-right order, and neutrals
/// more right-to-left than left-to-right letters ([`is_rtl_line`]); numbers keep their left-to-right order, and neutrals
/// (punctuation, gaps) take the direction of their neighbours, or the paragraph's between opposite ones.
pub(super) fn reverse_rtl(glyphs: &[&TextGlyph], idx: &mut [usize], spaces: &mut [bool]) {
pub(super) fn reverse_rtl(glyphs: &[&TextGlyph], anchor: &[usize], idx: &mut [usize], spaces: &mut [bool]) {
// Glyphs and word gaps (None) in visual order.
let mut items: Vec<(Option<usize>, Bidi)> = Vec::with_capacity(idx.len() * 2);
for (w, &i) in idx.iter().enumerate() {
@@ -434,8 +669,9 @@ pub(super) fn reverse_rtl(glyphs: &[&TextGlyph], idx: &mut [usize], spaces: &mut
}
let texts: Vec<&str> = items.iter().map(|it| it.0.map_or("", |i| glyphs[i].text.as_str())).collect();
let text = |k: usize| texts.get(k).copied().unwrap_or("");
let count = |b: Bidi| items.iter().filter(|it| it.1 == b).count();
let para = if count(Bidi::R) > count(Bidi::L) { Bidi::R } else { Bidi::L };
// The paragraph's direction, as `keep_taken_back_blanks` takes it.
let para = if is_rtl_line(glyphs, idx.iter().copied()) { Bidi::R } else { Bidi::L };
resolve_marks(glyphs, anchor, &mut items);
let n = items.len();
// W4: a single separator between two digits belongs to the number.
for k in 1..n.saturating_sub(1) {
crates/render/src/text/tests.rs
+608
View File
@@ -1658,6 +1658,257 @@ fn word_gaps_follow_position_not_drawing_order() {
assert_eq!(layout(v).plain_text(), "שלום עולם");
}
/// A right-to-left show string in reverse order starts with the space that ends its word, and the next letter is pulled
/// back over it (ISO 32000-2 §14.8.2.5.3): the space stays a word break after the word, not inside it (pdf.js #11656).
#[test]
fn a_space_taken_back_under_the_next_letter_stays_before_it() {
let rtl_line = |first: &str| {
// Drawn left to right: " הנידמה" with the space's advance taken back, then "רקבמ" after a word gap.
let mut v = vec![em_box(" ", 10.0, 0.0, 13.5, 11.0), em_box(first, 9.9, 0.0, 17.5, 11.0)];
let mut x = 17.5;
for c in ["נ", "י", "ד", "מ", "ה"] {
v.push(em_box(c, x, 0.0, x + 6.5, 11.0));
x += 6.5;
}
x += 3.5;
for c in ["ר", "ק", "ב", "מ"] {
v.push(em_box(c, x, 0.0, x + 6.5, 11.0));
x += 6.5;
}
layout(v)
};
let text = rtl_line("ה");
assert_eq!(text.plain_text().trim_end(), "מבקר המדינה");
assert_eq!(text.find("המדינה").len(), 1);
// The same with punctuation: the question mark ends the line, with no space before it.
assert_eq!(rtl_line("?").plain_text().trim_end(), "מבקר המדינ?");
// Right-to-left text drawn letter by letter leftwards: the letter after the space ends where the space starts, a
// rounding hair over it, and stays on its left.
let mut v = Vec::new();
let mut x = 60.0;
for c in ["ש", "ל", "ו", "ם"] {
v.push(em_box(c, x - 6.0, 0.0, x, 11.0));
x -= 6.0;
}
v.push(em_box(" ", x - 3.5, 0.0, x, 11.0));
x -= 3.5;
v.push(em_box("ע", x - 6.0, 0.0, x + 0.005, 11.0));
x -= 6.0;
for c in ["ו", "ל", "ם"] {
v.push(em_box(c, x - 6.0, 0.0, x, 11.0));
x -= 6.0;
}
assert_eq!(layout(v).plain_text(), "שלום עולם");
// A space inside a cluster, under a letter and the mark drawn on it, stays where x order puts it: before the mark.
let v = vec![
em_box("ש", 9.995, 0.0, 25.0, 11.0),
em_box(" ", 10.006, 0.0, 13.5, 11.0),
em_box("\u{5B4}", 10.003, 0.0, 12.8, 11.0),
em_box("ל", 25.0, 0.0, 31.0, 11.0),
];
assert_eq!(layout(v).plain_text(), "ל \u{5B4}ש");
// Left-to-right text has no reverse-order show strings: a space taken back there is still placed by position.
let v = vec![
em_box("a", 0.0, 0.0, 6.0, 11.0),
em_box("b", 6.0, 0.0, 12.0, 11.0),
em_box(" ", 12.0, 0.0, 15.5, 11.0),
em_box("c", 11.9, 0.0, 17.9, 11.0),
];
assert_eq!(layout(v).plain_text().trim_end(), "abc");
// A space whose advance stands is placed by position, as before.
let mut v = vec![em_box("a", 0.0, 0.0, 6.0, 11.0), em_box(" ", 6.0, 0.0, 9.5, 11.0), em_box("b", 9.5, 0.0, 15.5, 11.0)];
v.push(em_box("c", 15.5, 0.0, 21.5, 11.0));
assert_eq!(layout(v).plain_text(), "a bc");
}
/// A neutral taken back over a space keeps the space before it only inside right-to-left text. On a right-to-left line
/// drawn in logical order, a parenthesis that opens Latin text and is kerned back into the space before it stays
/// where its position puts it, as it would on a line without the Hebrew word.
#[test]
fn a_taken_back_neutral_beside_latin_text_keeps_its_place() {
// Drawn right to left: "שלום", the space, "(" pulled back into the space, then "AB" to its left.
let mut v = Vec::new();
let mut x = 60.0;
for c in ["ש", "ל", "ו", "ם"] {
v.push(em_box(c, x - 6.5, 0.0, x, 11.0));
x -= 6.5;
}
v.push(em_box(" ", x - 3.5, 0.0, x, 11.0));
x -= 3.5;
v.push(em_box("(", x - 2.0, 0.0, x + 2.0, 11.0));
x -= 2.0;
v.push(em_box("A", x - 13.0, 0.0, x - 6.5, 11.0));
v.push(em_box("B", x - 6.5, 0.0, x, 11.0));
assert_eq!(layout(v).plain_text(), "שלום (AB");
// A number reads as right-to-left beside a neutral: a question mark taken back over the space after "15", drawn in
// reverse order, still ends the line with no space before it.
let mut v = vec![em_box(" ", 10.0, 0.0, 13.5, 11.0), em_box("?", 9.9, 0.0, 15.0, 11.0)];
v.push(em_box("1", 15.0, 0.0, 20.5, 11.0));
v.push(em_box("5", 20.5, 0.0, 26.0, 11.0));
v.push(em_box(" ", 26.0, 0.0, 29.5, 11.0));
let mut x = 29.5;
for c in ["ף", "י", "ע", "ס"] {
v.push(em_box(c, x, 0.0, x + 6.5, 11.0));
x += 6.5;
}
assert_eq!(layout(v).plain_text().trim_end(), "סעיף 15?");
}
/// A mark drawn between a space and the letter taken back over it does not keep them apart: the space still ends the
/// word before, and the mark follows its letter, whether its box starts right or left of the space's start.
#[test]
fn a_mark_between_a_space_and_its_taken_back_letter_keeps_the_space_before_both() {
let line = |mark_x0: f32| {
let mut v = vec![em_box(" ", 10.0, 0.0, 13.5, 11.0), em_box("\u{5B8}", mark_x0, 0.0, 17.5, 11.0), em_box("ה", 9.9, 0.0, 17.5, 11.0)];
let mut x = 17.5;
for c in ["נ", "י", "ד", "מ", "ה"] {
v.push(em_box(c, x, 0.0, x + 6.5, 11.0));
x += 6.5;
}
x += 3.5;
for c in ["ר", "ק", "ב", "מ"] {
v.push(em_box(c, x, 0.0, x + 6.5, 11.0));
x += 6.5;
}
layout(v).plain_text()
};
assert_eq!(line(12.0).trim_end(), "מבקר המדינה\u{5B8}");
assert_eq!(line(9.95).trim_end(), "מבקר המדינה\u{5B8}");
// PDFBOX-5487's Arabic: a tanwin pulled back over the space, and the alef it sits on starting at the space's own
// start, so x order puts the alef after the space.
let mut v = vec![em_box(" ", 10.0, 0.0, 13.5, 11.0), em_box("\u{64B}", 9.5, 0.0, 16.3, 11.0), em_box("ا", 10.0, 0.0, 12.8, 11.0)];
let mut x = 12.8;
for c in ["ر", "ي", "ر", "ق", "ت"] {
v.push(em_box(c, x, 0.0, x + 6.5, 11.0));
x += 6.5;
}
v.push(em_box("ي", 3.0, 0.0, 6.0, 11.0));
v.push(em_box("ف", 6.0, 0.0, 9.5, 11.0));
assert_eq!(layout(v).plain_text().trim_end(), "تقريراً في");
}
/// Two taken-back spaces on one line each stay before their letter, and a second space drawn before a taken-back one
/// stays with it.
#[test]
fn two_taken_back_spaces_on_a_line_each_stay_before_their_letter() {
// Drawn left to right: " הנידמה" and " רקבמ", each space's advance taken back by its next letter, then " לע".
let word = |v: &mut Vec<TextGlyph>, x: &mut f32, first: &str, rest: &[&str]| {
v.push(em_box(" ", *x, 0.0, *x + 3.5, 11.0));
v.push(em_box(first, *x - 0.1, 0.0, *x + 6.4, 11.0));
*x += 6.4;
for c in rest {
v.push(em_box(c, *x, 0.0, *x + 6.5, 11.0));
*x += 6.5;
}
};
let mut v = Vec::new();
let mut x = 10.0;
word(&mut v, &mut x, "ה", &["נ", "י", "ד", "מ", "ה"]);
word(&mut v, &mut x, "ר", &["ק", "ב", "מ"]);
// The last word's space keeps its advance.
v.push(em_box(" ", x, 0.0, x + 3.5, 11.0));
x += 3.5;
for c in ["ל", "ע"] {
v.push(em_box(c, x, 0.0, x + 6.5, 11.0));
x += 6.5;
}
assert_eq!(layout(v).plain_text().trim(), "על מבקר המדינה");
// Two spaces in a row, the second taken back: both stay before the letter.
let mut v = vec![em_box(" ", 6.5, 0.0, 10.0, 11.0), em_box(" ", 10.0, 0.0, 13.5, 11.0), em_box("ה", 9.9, 0.0, 17.5, 11.0)];
let mut x = 17.5;
for c in ["נ", "י", "ד", "מ", "ה"] {
v.push(em_box(c, x, 0.0, x + 6.5, 11.0));
x += 6.5;
}
// Untrimmed: both spaces follow the word, in drawing order.
assert_eq!(layout(v).plain_text(), "המדינה ");
}
/// pdf.js issue 11656 (https://github.com/mozilla/pdf.js/issues/11656): its content stream rebuilt with the same
/// codes, widths, adjustments and `/ToUnicode` on an unembedded font. Each right-to-left show string starts with a
/// space taken back under the next letter or punctuation; copied text keeps whole words and punctuation after its
/// word, and the parentheses of "15(ב)(4)" read as before.
#[test]
fn issue_11656_layout_reads_whole_words() {
use lopdf::{Document, Object, Stream, dictionary};
let shows: [(f32, f32, &str); 10] = [
(
386.83,
708.96,
"[<0003>324<0022>-3<02AD>11<02AC02A5>4<02AB0003>10<02AD02A502AC>4<02B90003>8<02A402AE000302AD>9<02AC02A5>4<02B2>-3<000302AD02A502AC02B9>] TJ",
),
(380.71, 708.96, "<0003> Tj"),
(
329.69,
686.62,
"[<0003>324<001102A902BA>3<02A902AE02A0000302A302AC>4<02A9000302B8>4<02A102AB>11<000302A902B0>10<02A00003>-2<000F02AD02A902A8>4<02A502A7>13<02A4000302AF02AE000302A302B8>6<02A502A9>5<000302A902B002A0>] TJ",
),
(323.57, 686.62, "<0003> Tj"),
(446.35, 664.3, "[<0011>10<02AD02A902A3>6<02AC02A9>4<02A4000302AC02AB000302A502AE02AB>] TJ"),
(
378.19,
641.98,
"[<000302B3>-2<02A902B202B1>11<000302A902B402AC0003>10<02A402AC02A0000302BA02A5>3<02AC02A5>4<02B2>7<02B4000302A1>-3<02A502A9>5<02A702AC>] TJ",
),
(366.19, 641.98, "-0.0278 Tc [<00140018>] TJ 0 Tc"),
(346.51, 641.98, "[<000C000B>5<02A1000C>] TJ"),
(340.49, 641.98, "[<0017>] TJ"),
(236.33, 641.98, "[<0003>324<02A402B002A902A3>4<02AE02A4000302B802B7>16<02A102AE0003>-2<02B7>3<02A502A7>13<02AC0003000B>] TJ"),
];
let ops: String = shows.iter().map(|(x, y, show)| format!("BT /F1 11.04 Tf 1 0 0 1 {x} {y} Tm {show} ET\n")).collect();
let cmap = "/CIDInit /ProcSet findresource begin 12 dict begin begincmap /CIDSystemInfo << /Registry (Adobe) /Ordering (UCS) /Supplement 0 >> def /CMapName /Adobe-Identity-UCS def /CMapType 2 def 1 begincodespacerange <0000> <FFFF> endcodespacerange 1 beginbfchar <0003> <0020> endbfchar 1 beginbfrange <000B> <000C> [<0029> <0028>] endbfrange 3 beginbfchar <000F> <002C> <0011> <002E> <0014> <0031> endbfchar 1 beginbfrange <0017> <0018> <0034> endbfrange 1 beginbfchar <0022> <003F> endbfchar 5 beginbfrange <02A0> <02A1> <05D0> <02A3> <02A5> <05D3> <02A7> <02A9> <05D7> <02AB> <02B4> <05DB> <02B7> <02BA> <05E7> endbfrange endcmap CMapName currentdict /CMap defineresource pop end end";
let ranges: [(i64, &[i64]); 13] = [
(0, &[1000]),
(3, &[313]),
(11, &[383, 383]),
(15, &[303]),
(17, &[303]),
(20, &[546]),
(23, &[546, 546]),
(34, &[474]),
(672, &[673, 640]),
(675, &[601, 685, 274]),
(679, &[685, 699, 274]),
(683, &[621, 621, 685, 685, 362, 401, 730, 648, 662, 652]),
(695, &[677, 569, 815, 709]),
];
let w: Vec<Object> = ranges
.iter()
.flat_map(|(first, widths)| [Object::Integer(*first), Object::Array(widths.iter().map(|&n| Object::Integer(n)).collect())])
.collect();
let mut doc = Document::with_version("1.7");
let unicode = doc.add_object(Stream::new(dictionary! {}, cmap.as_bytes().to_vec()));
let cid = doc.add_object(dictionary! {
"Type" => "Font", "Subtype" => "CIDFontType2", "BaseFont" => "TestHebrew",
"CIDSystemInfo" => dictionary! { "Registry" => Object::string_literal("Adobe"), "Ordering" => Object::string_literal("Identity"), "Supplement" => 0 },
"DW" => 1000, "W" => w, "CIDToGIDMap" => "Identity"
});
let font = doc.add_object(dictionary! { "Type" => "Font", "Subtype" => "Type0", "BaseFont" => "TestHebrew", "Encoding" => "Identity-H", "DescendantFonts" => vec![Object::Reference(cid)], "ToUnicode" => unicode });
let content = doc.add_object(Stream::new(dictionary! {}, ops.into_bytes()));
let pages = doc.new_object_id();
let page = doc.add_object(dictionary! { "Type" => "Page", "Parent" => pages, "MediaBox" => vec![Object::Integer(0), Object::Integer(0), Object::Integer(612), Object::Integer(792)], "Resources" => dictionary! {"Font" => dictionary! {"F1" => font}}, "Contents" => content });
doc.objects.insert(pages, dictionary! {"Type" => "Pages", "Kids" => vec![Object::Reference(page)], "Count" => 1}.into());
let catalog = doc.add_object(dictionary! {"Type" => "Catalog", "Pages" => pages});
doc.trailer.set("Root", catalog);
let mut bytes = Vec::new();
doc.save_to(&mut bytes).unwrap();
let text = extract_page(&Pdf::new(std::sync::Arc::new(bytes)).unwrap(), 0, &InterpreterSettings::default()).unwrap();
let plain = text.plain_text();
let lines: Vec<&str> = plain.lines().map(str::trim).collect();
assert_eq!(
lines,
[
"שלום עולם מה שלום כולם?",
"אני יורד מן החוטים, אני כבר ילד אמיתי.",
"כמו כל הילדים.",
"לחיוב פעולות אלה לפי סעיף 15(ב)(4) לחוק מבקר המדינה"
]
);
for word in ["המדינה", "כולם?", "אמיתי.", "15(ב)(4)", "(ב)"] {
assert_eq!(text.find(word).len(), 1, "{word:?} in {lines:?}");
}
}
/// #2667: a number or punctuation on a right-to-left line must not split it into separately reversed words.
#[test]
fn rtl_line_with_number_and_punctuation_reads_in_logical_order() {
@@ -2447,6 +2698,190 @@ fn text_outside_the_crop_box_is_clipped() {
}
}
/// One glyph per text object, as a CAD or word-processor PDF writer draws shaped text: `text` in Helvetica
/// remapped by `/ToUnicode`, base letters 600 units wide, combining marks 0 wide and placed at the pen after their
/// base shifted by `mark_dx` points (a mark-positioning offset), words 260 units apart, with a space glyph or
/// without one.
fn one_glyph_per_show(text: &str, mark_dx: f32, space_glyphs: bool) -> Pdf {
use lopdf::{Document, Object, Stream, dictionary};
let is_mark = |c: char| unicode_bidi::bidi_class(c) == unicode_bidi::BidiClass::NSM;
let mut chars: Vec<char> = Vec::new();
for c in text.chars() {
if !chars.contains(&c) {
chars.push(c);
}
}
let code = |c: char| 0x21 + chars.iter().position(|&k| k == c).unwrap();
let (size, base, space) = (20.0_f32, 600.0_f32, 260.0_f32);
let mut ops = String::new();
let mut x = 50.0_f32;
for c in text.chars() {
let at = if is_mark(c) { x + mark_dx } else { x };
if c != ' ' || space_glyphs {
ops.push_str(&format!("BT /F1 {size} Tf 1 0 0 1 {at:.3} 300 Tm <{:02X}> Tj ET\n", code(c)));
}
if c == ' ' {
x += space * size / 1000.0;
} else if !is_mark(c) {
x += base * size / 1000.0;
}
}
let width = |c: char| {
if is_mark(c) {
0
} else if c == ' ' {
260
} else {
600
}
};
let widths: Vec<Object> = chars.iter().map(|&c| Object::Integer(width(c))).collect();
let pairs: String = chars.iter().map(|&c| format!("<{:02X}> <{:04X}>\n", code(c), u32::from(c))).collect();
let cmap = format!(
"/CIDInit /ProcSet findresource begin 12 dict begin begincmap /CIDSystemInfo << /Registry (Adobe) /Ordering (UCS) /Supplement 0 >> def /CMapName /T def /CMapType 2 def 1 begincodespacerange <00> <FF> endcodespacerange {} beginbfchar\n{pairs}endbfchar endcmap CMapName currentdict /CMap defineresource pop end end",
chars.len()
);
let mut doc = Document::with_version("1.7");
let unicode = doc.add_object(Stream::new(dictionary! {}, cmap.into_bytes()));
let mut differences = vec![Object::Integer(0x21)];
differences.extend(chars.iter().map(|_| Object::Name(b"a".to_vec())));
let font = doc.add_object(dictionary! {
"Type" => "Font", "Subtype" => "Type1", "BaseFont" => "Helvetica", "FirstChar" => 0x21, "LastChar" => 0x20 + chars.len() as i64,
"Widths" => widths, "Encoding" => dictionary! { "Type" => "Encoding", "Differences" => differences }, "ToUnicode" => unicode
});
let content = doc.add_object(Stream::new(dictionary! {}, ops.into_bytes()));
let pages = doc.new_object_id();
let page = doc.add_object(dictionary! { "Type" => "Page", "Parent" => pages, "MediaBox" => vec![Object::Integer(0), Object::Integer(0), Object::Integer(600), Object::Integer(400)], "Resources" => dictionary! {"Font" => dictionary! {"F1" => font}}, "Contents" => content });
doc.objects.insert(pages, dictionary! {"Type" => "Pages", "Kids" => vec![Object::Reference(page)], "Count" => 1}.into());
let catalog = doc.add_object(dictionary! {"Type" => "Catalog", "Pages" => pages});
doc.trailer.set("Root", catalog);
let mut bytes = Vec::new();
doc.save_to(&mut bytes).unwrap();
Pdf::new(std::sync::Arc::new(bytes)).unwrap()
}
/// #6517: Thai drawn one glyph per show operation, each zero-advance tone mark or above vowel starting a hair past
/// its base's advance (or exactly at it, or back over the base), reads in content order: no mark moves after the
/// next letter, and the word gaps read as spaces whether a space glyph is drawn or not. Copy and Find use this order.
#[test]
fn thai_marks_drawn_one_glyph_at_a_time_keep_their_place_and_words() {
let thai = "ห้อง ก๋วยเตี๋ยว น้ำ ที่นี่";
for mark_dx in [0.4, 0.0, -6.0] {
for space_glyphs in [false, true] {
let text = extract_page(&one_glyph_per_show(thai, mark_dx, space_glyphs), 0, &InterpreterSettings::default()).unwrap();
assert_eq!(text.plain_text(), thai, "marks {mark_dx} pt past the pen, space glyphs {space_glyphs}");
for word in ["ห้อง", "ก๋วยเตี๋ยว", "น้ำ", "ที่นี่"] {
assert_eq!(text.find_opts(word, true, true).len(), 1, "{word:?} found as a whole word in {:?}", text.plain_text());
}
}
}
}
/// Combining marks of other left-to-right scripts ride on their base the same way: Devanagari signs (virama,
/// anusvara, a below vowel; the spacing vowel sign ि is a letter of its own), and Latin combining accents.
#[test]
fn combining_marks_of_other_scripts_keep_their_place() {
for text in ["कुछ हिंदी स्कूल", "e\u{301}te\u{301} na\u{303}o"] {
let page = extract_page(&one_glyph_per_show(text, 0.4, false), 0, &InterpreterSettings::default()).unwrap();
assert_eq!(page.plain_text(), text);
}
}
/// A line drawn right to left glyph by glyph still reads by x, and a mark drawn right after its letter moves with
/// it. A mark that starts far from the glyph before it is not taken for a mark on that glyph: it goes by its own x.
#[test]
fn a_mark_moves_with_its_letter_in_a_line_drawn_backwards() {
let v = vec![g("c", 24.0, 10.0, 30.0), g("b", 18.0, 10.0, 24.0), g("\u{301}", 24.3, 10.0, 29.3), g("a", 12.0, 10.0, 18.0)];
assert_eq!(layout(v).plain_text(), "ab\u{301}c");
let v = vec![g("a", 12.0, 10.0, 18.0), g("b", 18.0, 10.0, 24.0), g("c", 24.0, 10.0, 30.0), g("\u{301}", 13.0, 10.0, 18.0)];
assert_eq!(layout(v).plain_text(), "a\u{301}bc");
}
/// A second mark stacked on a letter rides on the letter, not only on the mark before it: a mark drawn back over its
/// base, then a tone mark a hair past the base's advance, both stay before the next letter.
#[test]
fn a_stacked_mark_rides_on_its_base_letter() {
let v = vec![g("ก", 12.0, 10.0, 18.0), g("\u{e34}", 10.0, 10.0, 15.0), g("\u{e48}", 19.5, 10.0, 24.5), g("ข", 18.0, 10.0, 24.0)];
assert_eq!(layout(v).plain_text(), "ก\u{e34}\u{e48}ข");
}
/// A mark drawn raised by more than half an em over its letter (a tone mark lifted above an upper vowel) stays on the
/// letter's line and before the next letter.
#[test]
fn a_raised_mark_stays_with_its_letter() {
let v = vec![g("ก", 12.0, 10.0, 18.0), g("\u{e48}", 18.4, 16.0, 23.4), g("ข", 18.0, 10.0, 24.0), g("ค", 24.0, 10.0, 30.0)];
assert_eq!(layout(v).plain_text(), "ก\u{e48}ขค");
}
/// A word gap after a mark is measured from its letter's advance: a mark drawn a hair past the pen does not narrow it.
#[test]
fn a_word_gap_after_a_mark_is_measured_from_its_letter() {
let v = vec![g("a", 0.0, 10.0, 6.0), g("b", 6.0, 10.0, 12.0), g("\u{301}", 12.4, 10.0, 17.4), g("c", 13.8, 10.0, 19.8), g("d", 19.8, 10.0, 25.8)];
assert_eq!(layout(v).plain_text(), "ab\u{301} cd");
}
/// In a run squeezed to no width, a mark still follows its letter, even when it is drawn past the next letter.
#[test]
fn a_mark_in_a_zero_width_run_follows_its_letter() {
let v = vec![g("a", 10.0, 10.0, 10.0), g("\u{301}", 10.6, 10.0, 10.6), g("b", 10.3, 10.0, 10.3)];
assert_eq!(layout(v).plain_text(), "a\u{301}b");
}
/// One show string of `text` in Helvetica remapped by `/ToUnicode`, every glyph `width` units wide, combining marks
/// and spaces included, as a font whose `/Widths` give a mark an advance of its own draws it: each mark moves the pen,
/// and the next letter starts right at the mark's box.
fn one_show_string(text: &str, width: i64) -> Pdf {
use lopdf::{Document, Object, Stream, dictionary};
let mut chars: Vec<char> = Vec::new();
for c in text.chars() {
if !chars.contains(&c) {
chars.push(c);
}
}
let code = |c: char| 0x21 + chars.iter().position(|&k| k == c).unwrap();
let codes: String = text.chars().map(|c| format!("{:02X}", code(c))).collect();
let ops = format!("BT /F1 14 Tf 20 220 Td <{codes}> Tj ET");
let widths: Vec<Object> = chars.iter().map(|_| Object::Integer(width)).collect();
let pairs: String = chars.iter().map(|&c| format!("<{:02X}> <{:04X}>\n", code(c), u32::from(c))).collect();
let cmap = format!(
"/CIDInit /ProcSet findresource begin 12 dict begin begincmap /CIDSystemInfo << /Registry (Adobe) /Ordering (UCS) /Supplement 0 >> def /CMapName /T def /CMapType 2 def 1 begincodespacerange <00> <FF> endcodespacerange {} beginbfchar\n{pairs}endbfchar endcmap CMapName currentdict /CMap defineresource pop end end",
chars.len()
);
let mut doc = Document::with_version("1.7");
let unicode = doc.add_object(Stream::new(dictionary! {}, cmap.into_bytes()));
let mut differences = vec![Object::Integer(0x21)];
differences.extend(chars.iter().map(|_| Object::Name(b"a".to_vec())));
let font = doc.add_object(dictionary! {
"Type" => "Font", "Subtype" => "Type1", "BaseFont" => "Helvetica", "FirstChar" => 0x21, "LastChar" => 0x20 + chars.len() as i64,
"Widths" => widths, "Encoding" => dictionary! { "Type" => "Encoding", "Differences" => differences }, "ToUnicode" => unicode
});
let content = doc.add_object(Stream::new(dictionary! {}, ops.into_bytes()));
let pages = doc.new_object_id();
let page = doc.add_object(dictionary! { "Type" => "Page", "Parent" => pages, "MediaBox" => vec![Object::Integer(0), Object::Integer(0), Object::Integer(600), Object::Integer(400)], "Resources" => dictionary! {"Font" => dictionary! {"F1" => font}}, "Contents" => content });
doc.objects.insert(pages, dictionary! {"Type" => "Pages", "Kids" => vec![Object::Reference(page)], "Count" => 1}.into());
let catalog = doc.add_object(dictionary! {"Type" => "Catalog", "Pages" => pages});
doc.trailer.set("Root", catalog);
let mut bytes = Vec::new();
doc.save_to(&mut bytes).unwrap();
Pdf::new(std::sync::Arc::new(bytes)).unwrap()
}
/// A Thai tone mark or above or below vowel whose font gives it an advance of its own (here 300 units, as every
/// glyph) moves the pen, so the next letter of the word starts a mark's width past its base. The cluster ends at the
/// mark: no space opens inside "ข้อความ", "เดิม" or "ผู้ใช้", and the space glyphs between the words still read as
/// spaces.
#[test]
fn a_thai_mark_with_its_own_advance_does_not_open_a_word_gap() {
let thai = "ข้อความเดิม น้ำ ผู้ใช้";
for width in [300, 600] {
let text = extract_page(&one_show_string(thai, width), 0, &InterpreterSettings::default()).unwrap();
assert_eq!(text.plain_text(), thai, "every glyph {width} units wide");
for word in ["ข้อความเดิม", "น้ำ", "ผู้ใช้"] {
assert_eq!(text.find_opts(word, true, true).len(), 1, "{word:?} found as a whole word in {:?}", text.plain_text());
}
}
}
/// A one-page document showing `codes` in a Type 0 font with `encoding` over an embedded
/// `descendant` (`CIDFontType0` takes the bundled CFF program, `CIDFontType2` a TrueType one)
/// of the Adobe-Japan1 collection, with `to_unicode` as its `/ToUnicode` CMap when given.
@@ -2706,3 +3141,176 @@ fn the_glyph_at_a_point_is_the_first_whose_box_holds_it() {
assert_eq!(text.glyph_at(25.0, 5.0), None);
assert_eq!(PageText::default().glyph_at(0.0, 0.0), None);
}
/// A combining mark takes the direction of the letter it rides on (UAX #9 W1, nonspacing marks): a Latin accent at the
/// end of a Latin word on a right-to-left line stays after its letter, not on the far side of it.
#[test]
fn a_latin_mark_at_a_word_end_on_a_right_to_left_line_stays_after_its_letter() {
// Drawn left to right: "abé" (e and a combining acute riding on it), a space, then "שלום" right to left.
let mut v = vec![em_box("a", 0.0, 0.0, 6.0, 11.0), em_box("b", 6.0, 0.0, 12.0, 11.0), em_box("e", 12.0, 0.0, 18.0, 11.0)];
v.push(em_box("\u{301}", 14.0, 0.0, 14.0, 11.0));
v.push(em_box(" ", 18.0, 0.0, 21.5, 11.0));
let mut x = 21.5;
for c in ["ם", "ו", "ל", "ש", "ם", "ו", "ל", "ש"] {
v.push(em_box(c, x, 0.0, x + 6.5, 11.0));
x += 6.5;
}
assert_eq!(layout(v).plain_text().trim_end(), "שלוםשלום abe\u{301}");
}
/// A line's edge has the line's direction (UAX #9 X10): on a line of mostly left-to-right text, a neutral taken back
/// over a space at the line's left edge is not inside right-to-left text, so it stays where its position puts it,
/// however close a Hebrew word is on its other side.
#[test]
fn a_taken_back_neutral_at_the_edge_of_a_left_to_right_line_keeps_its_place() {
let line = |words: &[&str]| {
let mut v = vec![em_box(" ", 10.0, 0.0, 13.5, 11.0), em_box("(", 9.9, 0.0, 15.0, 11.0)];
let mut x = 15.0;
for word in words {
for c in word.chars() {
v.push(em_box(&c.to_string(), x, 0.0, x + 6.5, 11.0));
x += 6.5;
}
x += 3.5;
}
layout(v).plain_text()
};
// Mostly Latin: x order, "(" first.
assert_eq!(line(&["םולש", "Hello", "world", "again"]).trim(), "( שלום Hello world again");
// Mostly Hebrew: the line's edge is right-to-left, so "(" is taken back with the space, as "?" is in
// `a_space_taken_back_under_the_next_letter_stays_before_it`.
assert_eq!(line(&["םולש", "םלוע", "AB"]).trim_end(), "AB עולם שלום(");
}
/// Runs that overprint each other are placed by where their clusters start: a combining mark riding on the second
/// run's first letter and reaching left of the first run's start does not put the second run first.
#[test]
fn a_mark_reaching_left_of_its_letter_does_not_reorder_overprinting_runs() {
// A first run "ABCDEF", then a second run drawn back over it, its "c" starting 1 pt right of "A" and the acute on
// that "c" reaching 1.5 pt left of "A".
let mut v: Vec<TextGlyph> =
"ABCDEF".chars().zip(0u8..).map(|(c, k)| em_box(&c.to_string(), 100.0 + 6.0 * f32::from(k), 0.0, 106.0 + 6.0 * f32::from(k), 11.0)).collect();
v.push(em_box("c", 101.0, 0.0, 107.0, 11.0));
v.push(em_box("\u{301}", 98.5, 0.0, 102.5, 11.0));
v.push(em_box("d", 107.0, 0.0, 113.0, 11.0));
let text = layout(v).plain_text();
assert!(text.starts_with("ABCDEF"), "{text:?}");
assert!(text.contains("c\u{301}d"), "{text:?}");
}
/// A right-to-left mark rides on no glyph, so it keeps its own class: one placed after a Latin
/// letter, left of the Hebrew letter it belongs to, still reads with the Hebrew word, after its letter.
#[test]
fn a_hebrew_mark_after_a_latin_letter_stays_with_the_hebrew_word() {
let mut v = vec![em_box("a", 0.0, 0.0, 6.0, 11.0), em_box("b", 6.0, 0.0, 12.0, 11.0)];
v.push(em_box("\u{5B8}", 11.9, 0.0, 11.9, 11.0));
let mut x = 12.0;
for c in ["ם", "ו", "ל", "ש"] {
v.push(em_box(c, x, 0.0, x + 6.5, 11.0));
x += 6.5;
}
let text = layout(v).plain_text();
// The mark follows its letter, the last of "שלום", before the Latin text.
assert_eq!(text, "שלום\u{5B8}ab");
}
/// Nonspacing marks count toward neither direction of a line: a line of four Latin letters and a
/// three-letter Hebrew word whose letters each carry two points reads in the same order as without
/// the points.
#[test]
fn marks_do_not_decide_a_lines_direction() {
let line = |marks: bool| {
let mut v: Vec<TextGlyph> =
["a", "b", "c", "d"].iter().zip(0u8..).map(|(c, k)| em_box(c, 6.0 * f32::from(k), 0.0, 6.0 * f32::from(k) + 6.0, 11.0)).collect();
let mut x = 27.5;
for c in ["ו", "ל", "ש"] {
v.push(em_box(c, x, 0.0, x + 6.5, 11.0));
if marks {
v.push(em_box("\u{5B8}", x + 1.0, 0.0, x + 1.0, 11.0));
v.push(em_box("\u{5B4}", x + 2.0, 0.0, x + 2.0, 11.0));
}
x += 6.5;
}
layout(v).plain_text()
};
// Compared on the letters alone: the points' own word gaps are not what this is about.
let letters = |s: String| s.chars().filter(|&c| c != '\u{5B8}' && c != '\u{5B4}' && c != ' ').collect::<String>();
let marked = line(true);
for point in ['\u{5B8}', '\u{5B4}'] {
assert_eq!(marked.chars().filter(|&c| c == point).count(), 3, "every point is kept: {marked:?}");
}
assert_eq!(letters(marked), letters(line(false)));
assert!(letters(line(true)).starts_with("abcd"), "a mostly Latin line reads Latin first");
}
/// A ligature glyph counts each of its letters toward a line's direction: a line of an "ffi" ligature, a word gap and
/// a two-letter Hebrew word reads as the same line with "f", "f" and "i" drawn one by one, Latin first.
#[test]
fn a_ligature_counts_each_of_its_letters_toward_a_lines_direction() {
let line = |latin: &[&str]| {
let mut v: Vec<TextGlyph> = Vec::new();
let step = 12.0 / latin.len() as f32;
for (k, t) in latin.iter().enumerate() {
let x = step * k as f32;
v.push(em_box(t, x, 0.0, x + step, 11.0));
}
v.push(em_box("ל", 16.0, 0.0, 22.5, 11.0));
v.push(em_box("ש", 22.5, 0.0, 29.0, 11.0));
layout(v).plain_text()
};
let split = line(&["f", "f", "i"]);
assert_eq!(split, "ffi של");
assert_eq!(line(&["ffi"]), split);
}
/// Punctuation of a right-to-left script is not a letter and doesn't decide a line's direction: two Latin letters,
/// a Hebrew letter and a maqaf and a sof pasuq after it read Latin first.
#[test]
fn right_to_left_punctuation_does_not_decide_a_lines_direction() {
let mut v = vec![em_box("a", 0.0, 0.0, 6.0, 11.0), em_box("b", 6.0, 0.0, 12.0, 11.0)];
let mut x = 16.0;
for c in ["\u{5C3}", "\u{5BE}", "ש"] {
v.push(em_box(c, x, 0.0, x + 6.5, 11.0));
x += 6.5;
}
let text = layout(v).plain_text();
assert!(text.starts_with("ab"), "{text:?}");
}
/// An alphabetic nonspacing mark (U+0345) gives the text around a taken-back neutral no direction: a question mark
/// taken back over the space that starts a reverse-order show string reads as on the same line without the mark.
#[test]
fn an_alphabetic_mark_beside_a_taken_back_neutral_gives_it_no_direction() {
let line = |mark: bool| {
let mut v = vec![em_box(" ", 10.0, 0.0, 13.5, 11.0), em_box("?", 9.9, 0.0, 15.0, 11.0)];
let mut x = 15.0;
for c in ["ף", "י", "ע", "ס"] {
v.push(em_box(c, x, 0.0, x + 6.5, 11.0));
x += 6.5;
}
if mark {
v.push(em_box("\u{345}", 14.0, 0.0, 14.0, 11.0));
}
layout(v).plain_text()
};
let without = line(false);
assert_eq!(line(true).replace('\u{345}', ""), without);
}
/// A mark of a left-to-right script that rides on no letter has no direction of its own: U+0345 (alphabetic, bidi
/// class NSM) drawn on the last letter of a Hebrew word, left of it and right after a Latin word, reads with the
/// Hebrew word, after that letter, as a Hebrew point drawn there does.
#[test]
fn an_unattached_alphabetic_mark_reads_with_the_glyphs_at_its_place() {
let mut v = vec![em_box("a", 0.0, 0.0, 6.0, 11.0), em_box("b", 6.0, 0.0, 12.0, 11.0)];
let mut x = 12.0;
for c in ["ם", "ו", "ל", "ש"] {
v.push(em_box(c, x, 0.0, x + 6.5, 11.0));
if c == "ם" {
v.push(em_box("\u{345}", 11.9, 0.0, 11.9, 11.0));
}
x += 6.5;
}
let text = layout(v).plain_text();
assert_eq!(text, "שלום\u{345}ab");
}
crates/ui-egui/src/canvas/zoom_snap.rs
+48 -3
View File
@@ -120,13 +120,13 @@ impl PdfCraftApp {
Ok(())
}
/// Render `view_rect` (page view points) of `page` at the current zoom and copy it to the
/// clipboard (and keep it in `last_snapshot`).
/// Render `view_rect` (page view points) of `page` and copy it to the clipboard (and keep it
/// in `last_snapshot`), at the current zoom or [`SNAPSHOT_MIN_SCALE`] if that is finer.
pub fn snapshot(&mut self, index: usize, page: usize, view_rect: [f32; 4]) -> Result<(u32, u32), String> {
let view = &self.views[index];
let doc = self.session.get(view.id).ok_or("no document")?;
let ppp = self.ctx.as_ref().map_or(2.0, |c| c.pixels_per_point());
let scale = (view.zoom * ppp).clamp(0.5, 8.0);
let scale = snapshot_scale(view.render_scale(ppp), view_rect[2] - view_rect[0], view_rect[3] - view_rect[1]);
let tile = Tile {
x: (view_rect[0] * scale).floor().max(0.0) as u32,
y: (view_rect[1] * scale).floor().max(0.0) as u32,
@@ -153,6 +153,20 @@ impl PdfCraftApp {
}
}
/// The fewest pixels per point a snapshot is drawn at: 300 ppi, so what it copies stays readable
/// however far the page is zoomed out (#5716).
const SNAPSHOT_MIN_SCALE: f32 = 300.0 / 72.0;
/// The longest side a snapshot has, in pixels: the renderer draws at most 4096 a side in one tile
/// and cuts a larger area short, and this leaves room for rounding up.
const SNAPSHOT_MAX_SIDE: f32 = 4095.0;
/// The pixels per point to draw a `width` × `height` point area at, given the `on_screen` scale:
/// the finer of that and the minimum, or as fine as the renderer can draw the whole area.
fn snapshot_scale(on_screen: f32, width: f32, height: f32) -> f32 {
on_screen.max(SNAPSHOT_MIN_SCALE).min(SNAPSHOT_MAX_SIDE / width.max(height).max(1.0)).max(0.01)
}
/// The normalised bounds [u0, v0, u1, v1] of the pixels that aren't (near) white, if any.
pub(crate) fn ink_bounds(w: usize, h: usize, rgba: &[u8]) -> Option<[f32; 4]> {
if w == 0 || h == 0 || rgba.len() < w * h * 4 {
@@ -175,6 +189,37 @@ pub(crate) fn ink_bounds(w: usize, h: usize, rgba: &[u8]) -> Option<[f32; 4]> {
#[cfg(test)]
mod tests {
use super::{SNAPSHOT_MAX_SIDE, SNAPSHOT_MIN_SCALE, snapshot_scale};
#[test]
fn a_snapshot_is_drawn_at_the_finer_of_the_zoom_and_the_minimum() {
assert_eq!(snapshot_scale(0.5, 100.0, 100.0), SNAPSHOT_MIN_SCALE, "zoomed out");
assert_eq!(snapshot_scale(SNAPSHOT_MIN_SCALE, 100.0, 100.0), SNAPSHOT_MIN_SCALE);
assert_eq!(snapshot_scale(8.0, 100.0, 100.0), 8.0, "zoomed in further, it keeps the detail on screen");
}
#[test]
fn a_snapshot_stays_within_what_the_renderer_draws_at_once() {
// Past 983 pt a side, 300 ppi is more than one tile holds.
for (on_screen, w, h) in [(1.0, 2384.0, 3370.0), (1.0, 3370.0, 100.0), (8.0, 1000.0, 1000.0), (8.0, 14_400.0, 14_400.0)] {
let scale = snapshot_scale(on_screen, w, h);
let side = (w * scale).ceil().max((h * scale).ceil());
assert!(side <= SNAPSHOT_MAX_SIDE + 1.0, "{on_screen} on {w} × {h}: {side} pixels");
}
}
#[test]
fn a_snapshot_of_no_area_is_drawn_at_the_minimum() {
assert_eq!(snapshot_scale(1.0, 0.0, 0.0), SNAPSHOT_MIN_SCALE);
}
#[test]
fn a_snapshot_scale_survives_numbers_that_are_not() {
for scale in [snapshot_scale(f32::NAN, 5000.0, 5000.0), snapshot_scale(1.0, f32::NAN, 100.0), snapshot_scale(1.0, f32::INFINITY, 100.0)] {
assert!(scale.is_finite() && scale > 0.0, "{scale}");
}
}
#[test]
fn ink_bounds_finds_the_marked_area() {
let (w, h) = (10, 20);
crates/ui-egui/src/dialogs.rs
-2
View File
@@ -561,8 +561,6 @@ fn dialog_properties(ui: &mut egui::Ui, app: &mut PdfCraftApp, req: &mut Pending
}
row(ui, "Application", i.creator.clone().unwrap_or_default());
row(ui, "PDF producer", i.producer.clone().unwrap_or_default());
row(ui, "Created", i.created.clone().unwrap_or_default());
row(ui, "Modified", i.modified.clone().unwrap_or_default());
}
PropsTab::InitialView => {
use pdfcraft_engine::{InitialLayout as L, Magnification as M, Navigation as N};
crates/ui-egui/src/platform/updates.rs
+3
View File
@@ -113,6 +113,9 @@ pub(crate) fn dialog(app: &mut PdfCraftApp, ctx: &egui::Context) {
app.updates.updater.dismiss_status();
}
CheckResultResponse::CheckAgain => app.updates.check_requested = true,
CheckResultResponse::RestartToUpdate => {
app.updates.updater.restart_to_update();
}
}
}
}
crates/ui-egui/src/tools/measure_ui.rs
+233 -5
View File
@@ -41,6 +41,9 @@ impl Tool {
}
}
}
/// Where a thread reading snapping geometry sends it.
type PathsReceiver = std::sync::mpsc::Receiver<Result<measure::snap::Geometry, String>>;
pub struct MeasureView {
pub points: Vec<Point>,
pub page: Option<usize>,
@@ -70,6 +73,9 @@ pub struct MeasureView {
/// Snapping geometry (or why it failed) per (page, edit generation): extraction is not
/// repeated on every hover frame.
paths: Option<(usize, u64, Result<measure::snap::Geometry, String>)>,
/// The snapping geometry being read on a thread, for (page, edit generation): a heavy
/// drawing takes seconds, which would freeze the window.
pending_paths: Option<(usize, u64, PathsReceiver)>,
/// Saved measurements per edit generation, for the panel.
listing: Option<(u64, Result<measure::Listing, String>)>,
}
@@ -98,6 +104,7 @@ impl Default for MeasureView {
label: String::new(),
seeded_page: None,
paths: None,
pending_paths: None,
listing: None,
}
}
@@ -171,6 +178,7 @@ impl MeasureView {
/// Snaps `point` to the page's drawn geometry (endpoints, midpoints, intersections, paths),
/// recording in `state` why it couldn't when the content is too complex or unreadable.
fn snap_to_content(
ctx: &egui::Context,
state: &mut MeasureView,
doc: &Document,
page: usize,
@@ -178,8 +186,13 @@ fn snap_to_content(
pixels_per_unit: f32,
point: Point,
) -> Option<(Point, measure::snap::SnapKind)> {
if state.paths.as_ref().is_none_or(|(p, g, _)| *p != page || *g != doc.edit_generation()) {
state.paths = Some((page, doc.edit_generation(), doc.measurement_paths(page)));
let reading = tl!("Reading the drawing for snapping…");
if !load_paths(ctx, state, doc, page) {
state.error = Some(reading.into());
return None;
}
if state.error.as_deref() == Some(reading) {
state.error = None;
}
if let Some((_, _, Err(e))) = &state.paths {
state.error = Some(e.clone());
@@ -204,6 +217,83 @@ fn snap_to_content(
None
}
/// A file smaller than this may have its snapping geometry read on the spot.
#[cfg(not(target_arch = "wasm32"))]
const SNAP_SMALL_FILE: usize = 8 * 1024 * 1024;
/// Whether the snapping geometry of `page` as it is now is at hand, starting (on a thread) or
/// collecting the read when it isn't.
fn load_paths(ctx: &egui::Context, state: &mut MeasureView, doc: &Document, page: usize) -> bool {
use std::sync::mpsc::TryRecvError;
let generation = doc.edit_generation();
let current = |p: usize, g: u64| p == page && g == generation;
if state.paths.as_ref().is_some_and(|(p, g, _)| current(*p, *g)) {
return true;
}
// One read at a time, each being a whole page: one of another page or edit is let finish, and
// its answer dropped.
if let Some((p, g, receiver)) = &state.pending_paths {
let answer = match receiver.try_recv() {
Ok(result) => result,
Err(TryRecvError::Empty) => return false,
Err(TryRecvError::Disconnected) => Err("the snapping read stopped unexpectedly".into()),
};
let wanted = current(*p, *g);
state.pending_paths = None;
if wanted {
state.paths = Some((page, generation, answer));
return true;
}
}
start_read(ctx, state, doc, page, generation)
}
/// What the window itself reads of `page`: a small file's page, if it is within what a frame can
/// afford. A small file can still hold millions of segments.
#[cfg(not(target_arch = "wasm32"))]
fn read_on_the_spot(doc: &Document, page: usize) -> Option<Result<measure::snap::Geometry, String>> {
if doc.bytes.len() >= SNAP_SMALL_FILE {
return None;
}
let read = doc.measurement_paths_within(page, measure::snap::Limits::WITHIN_A_FRAME);
(!read.as_ref().is_ok_and(|g| g.truncated)).then_some(read)
}
/// Read `page`'s geometry, on a thread unless it is little. Whether it is at hand already.
#[cfg(not(target_arch = "wasm32"))]
fn start_read(ctx: &egui::Context, state: &mut MeasureView, doc: &Document, page: usize, generation: u64) -> bool {
if let Some(read) = read_on_the_spot(doc, page) {
state.paths = Some((page, generation, read));
return true;
}
let (sender, receiver) = std::sync::mpsc::channel();
let job = doc.measurement_paths_job(page);
let repaint = ctx.clone();
let work = move || {
let _ = sender.send(job());
repaint.request_repaint();
};
if let Err(e) = std::thread::Builder::new().name("pdfcraft-snap".into()).spawn(work) {
state.paths = Some((page, generation, Err(e.to_string())));
return true;
}
state.pending_paths = Some((page, generation, receiver));
false
}
/// With no threads here, what a frame can afford; past it the drawing says it exceeds the limit.
#[cfg(target_arch = "wasm32")]
fn start_read(_ctx: &egui::Context, state: &mut MeasureView, doc: &Document, page: usize, generation: u64) -> bool {
state.paths = Some((page, generation, doc.measurement_paths_within(page, measure::snap::Limits::WITHIN_A_FRAME)));
true
}
/// Whether a click may place a point: with snapping on, not while the drawing is being read, as
/// the point would land unsnapped.
fn takes_clicks(state: &MeasureView) -> bool {
!state.snap_enabled || state.pending_paths.is_none()
}
/// The Measure tools' own clicks this frame: those of a press nobody holds.
#[derive(Clone, Copy)]
pub(crate) struct Clicks {
@@ -258,7 +348,7 @@ pub(crate) fn page_input(
let mut point = [f64::from(x), f64::from(y)];
let mut snapped = None;
if state.snap_enabled {
snapped = snap_to_content(state, doc, page, [vx, vy], xf.zoom().max(1e-6), point);
snapped = snap_to_content(ui.ctx(), state, doc, page, [vx, vy], xf.zoom().max(1e-6), point);
}
if let Some((snapped_point, kind)) = snapped {
point = snapped_point;
@@ -270,7 +360,7 @@ pub(crate) fn page_input(
point = measure::snap::constrain(*last, point);
}
hover = Some(point);
if clicks.single {
if clicks.single && takes_clicks(state) {
if state.page != Some(page) {
state.cancel();
state.page = Some(page);
@@ -300,7 +390,8 @@ pub(crate) fn page_input(
}
}
if state.page == Some(page) && !state.points.is_empty() {
if (clicks.double || !typing && ui.input(|i| i.key_pressed(egui::Key::Enter))) && matches!(tool, Tool::Perimeter | Tool::Area) {
let double = clicks.double && takes_clicks(state);
if (double || !typing && ui.input(|i| i.key_pressed(egui::Key::Enter))) && matches!(tool, Tool::Perimeter | Tool::Area) {
finish(state, &mut view.pending_edit, doc, page, tool, author, units);
}
if !typing && ui.input(|i| i.key_pressed(egui::Key::Backspace)) {
@@ -653,6 +744,143 @@ mod tests {
measure::reading(Kind::Distance, &[from, to], &scale).expect("reading").value
}
/// A one-page PDF of `lines` short lines, written out uncompressed.
fn lined_pdf(lines: usize) -> Vec<u8> {
padded_lined_pdf(lines, 0)
}
/// [`lined_pdf`], with `padding` more bytes in the file that aren't on the page.
fn padded_lined_pdf(lines: usize, padding: usize) -> Vec<u8> {
let content = "10 10 m 200 200 l S\n".repeat(lines);
let objects = [
"<< /Type /Catalog /Pages 2 0 R >>".to_string(),
"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_string(),
"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R >>".to_string(),
format!("<< /Length {} >>\nstream\n{content}\nendstream", content.len()),
];
let mut pdf = b"%PDF-1.7\n".to_vec();
pdf.extend_from_slice(format!("%{}\n", "x".repeat(padding)).as_bytes());
let mut offsets = Vec::new();
for (i, object) in objects.iter().enumerate() {
offsets.push(pdf.len());
pdf.extend_from_slice(format!("{} 0 obj\n{object}\nendobj\n", i + 1).as_bytes());
}
let xref = pdf.len();
pdf.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objects.len() + 1).as_bytes());
for offset in offsets {
pdf.extend_from_slice(format!("{offset:010} 00000 n \n").as_bytes());
}
pdf.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objects.len() + 1).as_bytes());
pdf
}
/// A big file's geometry is read on a thread, which the window doesn't wait for; a small
/// file's is there at once.
#[test]
fn only_a_small_file_is_read_for_snapping_on_the_window_thread() {
let ctx = egui::Context::default();
let mut session = pdfcraft_engine::Session::new();
let small = session.open_new("small.pdf", std::sync::Arc::new(lined_pdf(10))).unwrap();
let mut state = MeasureView::default();
assert!(load_paths(&ctx, &mut state, session.get(small).unwrap(), 0), "there at once");
assert!(state.pending_paths.is_none());
// However little is on its page.
let big = padded_lined_pdf(10, SNAP_SMALL_FILE);
assert!(big.len() >= SNAP_SMALL_FILE);
let big = session.open_new("big.pdf", std::sync::Arc::new(big)).unwrap();
let doc = session.get(big).unwrap();
let mut state = MeasureView::default();
assert!(!load_paths(&ctx, &mut state, doc, 0), "started, and not waited for");
assert!(state.pending_paths.is_some());
let start = std::time::Instant::now();
while !load_paths(&ctx, &mut state, doc, 0) {
assert!(start.elapsed() < std::time::Duration::from_secs(60), "it arrives");
std::thread::sleep(std::time::Duration::from_millis(5));
}
assert!(state.pending_paths.is_none());
let geometry = state.paths.as_ref().and_then(|(_, _, g)| g.as_ref().ok()).expect("read");
assert!(geometry.segment_count() > 0);
}
/// A small file can hold more than a frame can read: it goes to the thread, and comes whole.
#[test]
fn a_small_file_past_a_frames_limits_is_read_on_a_thread() {
let lines = measure::snap::Limits::WITHIN_A_FRAME.segments + 1;
let pdf = lined_pdf(lines);
assert!(pdf.len() < SNAP_SMALL_FILE);
let ctx = egui::Context::default();
let mut session = pdfcraft_engine::Session::new();
let id = session.open_new("small.pdf", std::sync::Arc::new(pdf)).unwrap();
let doc = session.get(id).unwrap();
let mut state = MeasureView::default();
assert!(!load_paths(&ctx, &mut state, doc, 0), "not read on the window's thread");
let start = std::time::Instant::now();
while !load_paths(&ctx, &mut state, doc, 0) {
assert!(start.elapsed() < std::time::Duration::from_secs(60), "it arrives");
std::thread::sleep(std::time::Duration::from_millis(5));
}
let geometry = state.paths.as_ref().and_then(|(_, _, g)| g.as_ref().ok()).expect("read");
assert_eq!(geometry.segment_count(), lines);
assert!(!geometry.truncated);
}
/// A read of another page is let finish before the next starts, and only then is it dropped.
#[test]
fn only_one_snapping_read_runs_at_a_time() {
let ctx = egui::Context::default();
let mut session = pdfcraft_engine::Session::new();
let id = session.open_new("big.pdf", std::sync::Arc::new(lined_pdf(SNAP_SMALL_FILE / 20 + 1))).unwrap();
let doc = session.get(id).unwrap();
let (sender, receiver) = std::sync::mpsc::channel();
let mut state = MeasureView { pending_paths: Some((9, 9, receiver)), ..MeasureView::default() };
assert!(!load_paths(&ctx, &mut state, doc, 0));
assert_eq!(state.pending_paths.as_ref().map(|(p, _, _)| *p), Some(9), "waits for it");
sender.send(Ok(measure::snap::Geometry::default())).unwrap();
assert!(!load_paths(&ctx, &mut state, doc, 0));
assert_eq!(state.pending_paths.as_ref().map(|(p, _, _)| *p), Some(0), "then reads this page");
assert!(state.paths.is_none(), "the other page's answer is dropped");
}
/// A read that dies says so, instead of reading on for ever.
#[test]
fn a_snapping_read_that_stops_is_an_error() {
let ctx = egui::Context::default();
let mut session = pdfcraft_engine::Session::new();
let id = session.open_new("small.pdf", std::sync::Arc::new(lined_pdf(10))).unwrap();
let doc = session.get(id).unwrap();
let (sender, receiver) = std::sync::mpsc::channel();
drop(sender);
let mut state = MeasureView { pending_paths: Some((0, doc.edit_generation(), receiver)), ..MeasureView::default() };
assert!(load_paths(&ctx, &mut state, doc, 0));
assert!(matches!(&state.paths, Some((_, _, Err(_)))));
assert!(state.pending_paths.is_none());
}
#[test]
fn the_reading_message_goes_when_the_drawing_is_read() {
let ctx = egui::Context::default();
let mut session = pdfcraft_engine::Session::new();
let id = session.open_new("small.pdf", std::sync::Arc::new(lined_pdf(10))).unwrap();
let doc = session.get(id).unwrap();
let mut state = MeasureView { error: Some(tl!("Reading the drawing for snapping…").into()), ..MeasureView::default() };
let _ = snap_to_content(&ctx, &mut state, doc, 0, [5.0, 5.0], 1.0, [5.0, 5.0]);
assert_eq!(state.error, None);
}
#[test]
fn clicks_wait_for_the_drawing_only_with_snapping_on() {
let (_sender, receiver) = std::sync::mpsc::channel();
let mut state = MeasureView { pending_paths: Some((0, 0, receiver)), ..MeasureView::default() };
assert!(!takes_clicks(&state));
state.snap_enabled = false;
assert!(takes_clicks(&state));
state.snap_enabled = true;
state.pending_paths = None;
assert!(takes_clicks(&state));
}
#[test]
fn typed_ratios_are_read_leniently() {
assert_eq!(parse_ratio("1:200"), Some(200.0));
crates/ui-egui/tests/it/viewer.rs
+63
View File
@@ -1057,6 +1057,69 @@ fn marquee_zoom_and_snapshot() {
assert!(after > before * 2.0, "{before} → {after}");
}
/// #5716: what a snapshot holds doesn't depend on how far the page is zoomed out.
#[test]
fn a_snapshot_keeps_its_resolution_when_zoomed_out() {
let mut h = harness();
let i = h.state().active.unwrap();
h.state_mut().system_clipboard = false;
let mut sizes = Vec::new();
for zoom in ["25", "100"] {
h.state_mut().set_option("zoom", zoom).unwrap();
h.run_steps(2);
// The text line of the page, as a marquee would give it.
sizes.push(h.state_mut().snapshot(i, 0, [15.0, 125.0, 180.0, 160.0]).expect("a snapshot"));
}
assert_eq!(sizes[0], sizes[1], "the same pixels at either zoom");
assert_eq!(sizes[0], (688, 146), "165 × 35 pt at 300 ppi");
let (_, _, px) = h.state().last_snapshot.clone().expect("a snapshot");
assert!(px.as_chunks::<4>().0.iter().any(|p| p[0] < 100), "the text is in it");
// Zoomed in past the minimum, a snapshot has every pixel the screen shows, however far.
let ppp = h.ctx.pixels_per_point();
for zoom in [400.0_f32, 1600.0] {
h.state_mut().set_option("zoom", &zoom.to_string()).unwrap();
h.run_steps(2);
let (w, _) = h.state_mut().snapshot(i, 0, [15.0, 125.0, 180.0, 160.0]).expect("a snapshot");
let on_screen = (165.0 * zoom / 100.0 * pdfcraft_ui_egui::canvas::PT * ppp).ceil() as u32;
assert!(w.abs_diff(on_screen) <= 1, "{w} pixels at {zoom}%, {on_screen} on screen");
}
}
/// #5716: at 300 ppi this page is more than the renderer draws in one tile, which would cut off
/// the side the snapshot was taken for. A 2000 pt square with a black mark in its far corner.
#[test]
fn a_snapshot_of_a_large_page_reaches_its_far_corner() {
let stream = "0 0 0 rg 1900 0 100 100 re f";
let pdf = format!(
"%PDF-1.7
1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj
2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj
3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 2000 2000] /Contents 4 0 R >> endobj
4 0 obj << /Length {} >> stream
{stream}
endstream endobj
trailer << /Root 1 0 R >>
%%EOF",
stream.len()
);
let mut h = Harness::builder().with_size(egui::vec2(1200.0, 800.0)).build_eframe(move |_cc| {
let mut app = PdfCraftApp::new();
app.set_option("language", "en").unwrap();
app.open_bytes("large.pdf", None, pdf.into_bytes()).unwrap();
app.system_clipboard = false;
app
});
h.run_steps(4);
let i = h.state().active.unwrap();
let (w, hgt) = h.state_mut().snapshot(i, 0, [0.0, 0.0, 2000.0, 2000.0]).expect("a snapshot");
assert!(w <= 4096 && hgt <= 4096 && w == hgt, "{w} × {hgt}");
let (_, _, px) = h.state().last_snapshot.clone().expect("a snapshot");
let at = |x: u32, y: u32| px[((y * w + x) * 4) as usize];
assert!(at(w * 98 / 100, hgt * 98 / 100) < 100, "the mark in the far corner is in it");
assert!(at(w / 2, hgt / 2) > 200, "and the page around it is white");
}
#[test]
fn fit_visible_zooms_to_the_content_width() {
let mut h = harness();
crates/xfa/src/layout.rs
+12
View File
@@ -43,6 +43,15 @@ const MAX_DEPTH: usize = 64;
/// Most size measurements of one layout. Measurements of containers are memoized, so a real
/// form stays far below this; it bounds the work a hostile template can cause.
const MAX_MEASURES: usize = 2_000_000;
/// Most characters the draw phase lays out in one layout, each pass counted. A draw re-measures
/// overflowing text up to six times to shrink it, and a repeating subform draws its text once
/// per instance, so large text drawn many times would otherwise cost seconds; a real form's
/// draws stay far below this. Reduced under test so the guard is exercised without laying out
/// hundreds of megabytes of text.
#[cfg(not(test))]
const MAX_DRAW_CHARS: usize = 256 << 20;
#[cfg(test)]
const MAX_DRAW_CHARS: usize = 1 << 20;
/// Most initial instances of one repeating subform.
const MAX_INSTANCES: usize = 50;
/// Largest picture taken from the data (decoded bytes), as for template pictures.
@@ -149,6 +158,8 @@ struct Layouter<'t> {
measured: RefCell<HashMap<MeasureKey, Measured>>,
/// Measurements made so far (see [`MAX_MEASURES`]).
measures: Cell<usize>,
/// Characters the draw phase has laid out, each pass counted (see [`MAX_DRAW_CHARS`]).
drawn_chars: Cell<usize>,
/// The fields and draws of the block being placed: (first item, one past the last, top,
/// bottom) on the last page. A page break keeps each whole (see `split_block`).
leaves: Vec<(usize, usize, f64, f64)>,
@@ -198,6 +209,7 @@ pub fn layout(tpl: &Template, data: Option<&DataNode>) -> Result<Form, XfaError>
image_bytes: 0,
measured: RefCell::new(HashMap::new()),
measures: Cell::new(0),
drawn_chars: Cell::new(0),
leaves: Vec::new(),
frames: Vec::new(),
};
crates/xfa/src/layout/draw.rs
+32 -3
View File
@@ -230,13 +230,42 @@ impl Layouter<'_> {
_ => return Ok(()),
};
let face = Face::of(font);
// Each pass over the text — a re-measure that shrinks it, or the final layout — counts
// its characters against the draw budget, so large text drawn in many instances stops.
let cost = rich.plain().chars().count().max(1);
// The values of embedded fields are laid out with it, so each pass counts those too: a
// short text embedding a long value is as much work as the value written out.
let embed = |id: &str, max: usize| {
let v = self.embed_in(id, max, scope);
self.drawn_chars.set(self.drawn_chars.get().saturating_add(v.chars().count()));
v
};
// Text that doesn't fit the height it was given (the template's face is narrower than
// the standard one drawn in its place, so it wraps to more lines) is drawn smaller,
// down to 70%, rather than over what lies below.
let embed = |id: &str, max: usize| self.embed_in(id, max, scope);
let fits = |k: f64| text::measure_height(&rich, rect.w / k, para, &face, &embed) <= rect.h / k + 1.0;
let scale = if rect.h < 4.0 || fits(1.0) { 1.0 } else { [0.95, 0.9, 0.85, 0.8, 0.75].into_iter().find(|k| fits(*k)).unwrap_or(0.7) };
// A pass is not started once the budget is spent, and one that spends it stops the rest.
let fits = |k: f64| -> Result<bool, XfaError> {
self.check_drawn_chars()?;
self.drawn_chars.set(self.drawn_chars.get().saturating_add(cost));
let fit = text::measure_height(&rich, rect.w / k, para, &face, &embed) <= rect.h / k + 1.0;
self.check_drawn_chars()?;
Ok(fit)
};
let mut scale = 1.0;
if rect.h >= 4.0 && !fits(1.0)? {
scale = 0.7;
for k in [0.95, 0.9, 0.85, 0.8, 0.75] {
if fits(k)? {
scale = k;
break;
}
}
}
self.drawn_chars.set(self.drawn_chars.get().saturating_add(cost));
self.check_drawn_chars()?;
let mut block = text::layout(&rich, Rect::new(rect.x, rect.y, rect.w / scale, rect.h / scale), para, &face, &embed);
// The layout itself expands the embedded values once more.
self.check_drawn_chars()?;
if scale < 1.0 {
for s in &mut block.spans {
s.x = rect.x + (s.x - rect.x) * scale;
crates/xfa/src/layout/pages.rs
+10 -2
View File
@@ -2,8 +2,8 @@
use super::EMBED_COPIED;
use super::items::{Item, Page, UNDERLINE_DROP};
use super::{
Block, Border, Ctx, Cursor, FlowMark, Font, Frame, Layouter, MAX_ITEMS, MAX_MEASURES, MAX_PAGES, PAGE_COUNT_MARK, PageRole, Para, Rc, Rect,
SomIndices, XfaError, data_path, text,
Block, Border, Ctx, Cursor, FlowMark, Font, Frame, Layouter, MAX_DRAW_CHARS, MAX_ITEMS, MAX_MEASURES, MAX_PAGES, PAGE_COUNT_MARK, PageRole, Para,
Rc, Rect, SomIndices, XfaError, data_path, text,
};
impl Layouter<'_> {
@@ -57,6 +57,14 @@ impl Layouter<'_> {
Ok(())
}
/// `TooLarge` once the draw phase has laid out far more text than a real form does.
pub(super) fn check_drawn_chars(&self) -> Result<(), XfaError> {
if self.drawn_chars.get() > MAX_DRAW_CHARS {
return Err(XfaError::TooLarge("the form draws far more text than the reader lays out".into()));
}
Ok(())
}
pub(super) fn push(&mut self, item: Item) -> Result<(), XfaError> {
self.check_measures()?;
self.items += 1;
crates/xfa/src/tests.rs
+75 -1
View File
@@ -906,6 +906,75 @@ fn a_block_split_across_very_many_pages_spends_the_layout_budget() {
}
}
#[test]
fn large_text_drawn_in_many_instances_spends_the_draw_budget() {
// A draw whose text overflows its fixed height is re-measured up to six times to shrink it,
// and a repeating subform draws it once per instance. Large text drawn in many instances
// used to cost one full text layout per instance per pass, for seconds of work on a small
// file; each pass now counts its characters against the draw budget.
let words = "word ".repeat(4000); // 20,000 characters
let draw = format!("<draw w=\"1in\" h=\"0.5in\"><value><text>{words}</text></value></draw>");
let many =
format!("<template><subform layout=\"tb\"><subform layout=\"tb\"><occur initial=\"50\" max=\"-1\"/>{draw}</subform></subform></template>");
assert!(matches!(layout_xml(&many), Err(XfaError::TooLarge(_))), "large text in many instances must be refused");
// One copy of the same draw lays out: the budget does not reject an ordinary form.
let one = format!("<template><subform layout=\"tb\">{draw}</subform></template>");
assert!(layout_xml(&one).is_ok(), "a single large draw must lay out");
}
/// The draw budget counts the values of embedded fields as well as the text around them: a short
/// text embedding a long value, drawn in many instances, is as much work as the value written out
/// each time (CodeRabbit review on #5852).
#[test]
fn embedded_values_drawn_in_many_instances_spend_the_draw_budget() {
let value = "word ".repeat(4000); // 20,000 characters
let span = r##"<span xfa:embed="#long" xmlns:xfa="http://www.xfa.org/schema/xfa-data/1.0/"/>"##;
let draw = format!(
r#"<draw w="1in" h="0.5in"><value><exData contentType="text/html"><body xmlns="http://www.w3.org/1999/xhtml"><p>x {span}</p></body></exData></value></draw>"#
);
let field = format!(r#"<field id="long" presence="hidden" w="1in" h="0.2in"><ui><textEdit/></ui><value><text>{value}</text></value></field>"#);
let many =
format!(r#"<template><subform layout="tb">{field}<subform layout="tb"><occur initial="50" max="-1"/>{draw}</subform></subform></template>"#);
assert!(matches!(layout_xml(&many), Err(XfaError::TooLarge(_))), "a long embedded value in many instances must be refused");
// One copy lays out: the budget does not reject an ordinary form.
let one = format!(r#"<template><subform layout="tb">{field}{draw}</subform></template>"#);
assert!(layout_xml(&one).is_ok(), "a single draw embedding a long value must lay out");
}
/// The last layout pass of a draw counts against the budget too: a draw that fits is measured
/// once and laid out once, each pass expanding its embedded value, and when the form's last draw
/// crosses the budget in that final pass it is still refused (CodeRabbit review on #5852).
#[test]
fn the_draw_that_crosses_the_budget_in_its_last_pass_is_refused() {
// 15,000 characters: with the 1 MiB test budget, the tenth such draw crosses it during its
// final layout pass, after every check before that pass has passed.
let value = "word ".repeat(3000);
let span = r##"<span xfa:embed="#long" xmlns:xfa="http://www.xfa.org/schema/xfa-data/1.0/"/>"##;
let form = |n: usize| {
format!(
r#"<template><subform layout="tb"><field id="long" presence="hidden" w="1in" h="0.2in"><ui><textEdit/></ui><value><text>{value}</text></value></field><subform layout="tb"><occur initial="{n}" max="-1"/><draw w="7in" h="9in"><value><exData contentType="text/html"><body xmlns="http://www.w3.org/1999/xhtml"><p>x {span}</p></body></exData></value></draw></subform></subform></template>"#
)
};
assert!(layout_xml(&form(9)).is_ok());
assert!(matches!(layout_xml(&form(10)), Err(XfaError::TooLarge(_))), "the draw crossing the budget in its last pass must be refused");
}
/// Once a fit pass spends the budget, no more passes are made: a draw that doesn't fit and embeds
/// a long value is measured once, not seven times, before it is refused (CodeRabbit review on
/// #5852).
#[test]
fn fit_passes_stop_once_the_draw_budget_is_spent() {
let value = "word ".repeat(text::MAX_EMBEDDED / 5 + 1000);
let span = r##"<span xfa:embed="#long" xmlns:xfa="http://www.xfa.org/schema/xfa-data/1.0/"/>"##;
let tpl = format!(
r#"<template><subform layout="tb"><field id="long" presence="hidden" w="1in" h="0.2in"><ui><textEdit/></ui><value><text>{value}</text></value></field><draw w="1in" h="0.5in"><value><exData contentType="text/html"><body xmlns="http://www.w3.org/1999/xhtml"><p>x {span}</p></body></exData></value></draw></subform></template>"#
);
crate::layout::EMBED_COPIED.with(|c| c.set(0));
assert!(matches!(layout_xml(&tpl), Err(XfaError::TooLarge(_))));
let copied = crate::layout::EMBED_COPIED.with(|c| c.get());
assert!(copied <= 2 * text::MAX_EMBEDDED as u64, "{copied} bytes copied: more passes after the budget was spent");
}
#[test]
fn a_packet_stream_listed_many_times_is_read_once() {
let mut doc = Document::open(Arc::new(shell("<template><subform/></template>"))).unwrap();
@@ -4199,7 +4268,12 @@ fn embedded_values_are_copied_out_only_as_far_as_the_budget() {
</subform></template>"#
);
crate::layout::EMBED_COPIED.with(|c| c.set(0));
layout_xml(&tpl).unwrap();
// The draw budget counts embedded values too (1 MiB in tests), so it may stop the draw
// before its last pass; either way no more than this is copied out.
match layout_xml(&tpl) {
Ok(_) | Err(XfaError::TooLarge(_)) => {}
Err(e) => panic!("{e:?}"),
}
let copied = crate::layout::EMBED_COPIED.with(|c| c.get());
// The draw is measured at most seven times before it is laid out once.
assert!(copied <= 8 * text::MAX_EMBEDDED as u64 + 8 * 4 * 201, "{copied} bytes of embedded values copied");
crates/xfdf/Cargo.toml
+4
View File
@@ -14,4 +14,8 @@ pdfcraft-annot = { workspace = true }
pdfcraft-forms = { workspace = true }
pdfcraft-model = { workspace = true }
roxmltree = "0.21"
base64 = "0.23"
thiserror = { workspace = true }
[dev-dependencies]
proptest = { workspace = true }
crates/xfdf/README.md
+28 -5
View File
@@ -14,9 +14,30 @@ let report = import(&mut doc, &bytes)?; // XFDF, FDF, XML, CSV o
name, author, subject, dates, flags, colours, opacity, width, icon, quads, line ends, ink,
default appearance, callouts (`callout`, `fringe`, `intent`, `head`), pop-ups and reply threads
(`inreplyto`); field values nested by name, with text preserved across XML comments and processing instructions.
Formatting travels both ways as XFDF 3.0 names it: rich text (`<contents-richtext>`, the `/RC`
XHTML as markup; plain text in it is the comment's text), default style (`<defaultstyle>`),
border width, style and dash pattern (`width`, `style`, `dashes`; also from an older `/Border`
array), cloudy borders (`style="cloudy"`, `intensity`), `rotation`, `justification` (free text
`left`/`centered`/`right`, redaction `0`–`2`), `fringe` (`/RD`) on every type, line leaders and
captions (`leaderLength`, `leaderExtend`, `leader-offset`, `caption`, `caption-style`,
`caption-offset-h`/`-v`), the caret `symbol` and redaction `overlay-text`/`overlay-text-repeat`.
The intent (`/IT`) is written as both `intent` and `IT`, and either is read.
- **Appearances** (`<appearance>`, `appearance.rs`): a stamp's `/AP`, as XFDF 3.0 defines it for
stamps, base 64 of the dictionary written as `DICT`/`STREAM`/`ARRAY`/`NAME`/`INT`/`FIXED`/`BOOL`/
`STRING`/`NULL` elements with stream data in `DATA` (XFDF 3.0 "Stream encoding": `filtered`/`ascii`
or `raw`/`hex`), the shape XFDF files from other tools carry. Copy and paste carries every
comment's appearance but a redaction mark's, which PdfCraft always draws and import never
reads. Import keeps an appearance it can read and otherwise draws the comment and records the reason in
`Report::skipped` (the engine puts it in the repair log); it refuses one over 16 MiB, 64 MiB per
import, 64 nesting levels or 100,000 values. Export leaves out one past those limits or whose
rect has no area, and stops at the 64 MiB an import reads; where `/AS` selects one of several
states, the selected stream is written (and import, with no `/AS` in XFDF, refuses a normal
appearance of several states). Import and FDF import are all or nothing: on an error the
document is left as it was.
- **FDF**: fields (with `/Kids` hierarchies) and indirect comments. Nested annotation values
are copied under FDF object numbers; shared references and cycles remain shared. Document
links and appearances are excluded. Source `/Page` and `/Pages` dictionaries become null
are copied under FDF object numbers; shared references and cycles remain shared. A stamp's appearance
(`/AP`, with the forms and images it draws) is copied, as in XFDF, and an `/AP` in the file comes
back on import as objects of the document (a stream shared by several comments once); document links are excluded. Source `/Page` and `/Pages` dictionaries become null
at every copy depth, so nested destination actions cannot carry the source page tree or content.
Type names may be indirect; lookup checks each reference generation, is iterative, stops at
cycles and refuses chains beyond
@@ -28,7 +49,9 @@ let report = import(&mut doc, &bytes)?; // XFDF, FDF, XML, CSV o
levels or a conservative 64 MiB serialization budget instead of exporting partial data.
These are copied-graph limits: source parsing and page discovery retain their existing
COS/parser limits, with no total source-memory bound. Unknown indirect streams retain their
encoded bytes. FDF import semantics are unchanged.
encoded bytes. FDF import copies a comment's streams only when the comment lands on a page, and
refuses the file (nothing imported) when its comments expand past 1,000,000 values or 64 MiB, or
nest more than 256 levels, rather than import part of an appearance.
- **Form data**: Acrobat's XML (`xfdf:original` keeps names that aren't XML names), CSV and
tab-delimited text (a row of names, a row of values).
@@ -44,7 +67,7 @@ tables remains heuristic. Both write a cell that a spreadsheet would run as a fo
or `+` or `-` when it isn't a plain number) with a leading `'`; CSV import takes off the `'` that
export added, so exported data imports back unchanged.
Import replaces comments with the same `/NM` (and drops the replaced comment's pop-up), draws
appearances PdfCraft can draw, and sets values through the form's own checks (formats,
Import replaces comments with the same `/NM` (and drops the replaced comment's pop-up), keeps the
appearance the file carries or draws one where PdfCraft can, and sets values through the form's own checks (formats,
validation, recalculation); read-only fields take imported values, as in Acrobat. XML is read
with `roxmltree`.
crates/xfdf/src/appearance.rs
+377
View File
@@ -0,0 +1,377 @@
//! A comment's appearance dictionary (`/AP`) in XFDF.
//!
//! ISO 19444-1 (XFDF 3.0, "appearance") carries a stamp's `/AP` as a base 64 encoded string. What
//! the string holds is the dictionary written as XML, one element per PDF object: `DICT`, `STREAM`,
//! `ARRAY`, `NAME`, `INT`, `FIXED` (a real number), `BOOL`, `STRING` and `NULL`, each with its
//! dictionary key in `KEY` and a simple value in `VAL`; a stream's data is a `DATA` element whose
//! `MODE` and `ENCODING` follow XFDF 3.0 "Stream encoding" (`filtered`/`ascii` for short printable
//! data, `raw`/`hex` otherwise). That is what the XFDF files in the wild hold (for example the
//! stamps attached to PDFBOX-4437 and PDFBOX-4478). Indirect objects are written in place, so the
//! XML is self-contained.
//!
//! Both directions are bounded: the decoded XML, the stream data, the nesting and the number of
//! values. Anything unreadable makes the whole appearance unusable; the caller then records it and
//! draws the comment from its other entries.
use base64::Engine as _;
use super::common::{check_form_xml_depth, esc, parse_form_xml};
use super::{Dict, Document, ObjRef, Object, PdfString};
/// At most this much decoded XML in one appearance (and at most this much XML is written for one).
pub(crate) const MAX_APPEARANCE_BYTES: usize = 16 << 20;
/// At most this much appearance data (decoded XML) in one import, over all of its comments.
pub(crate) const MAX_IMPORT_APPEARANCE_BYTES: usize = 64 << 20;
/// How deeply objects may nest inside one appearance.
const MAX_NESTING: usize = 64;
/// How many PDF values one appearance may hold.
const MAX_VALUES: usize = 100_000;
/// XFDF 3.0 "Stream encoding": data shorter than this that is all printable ASCII is written as it
/// is (`filtered`/`ascii`); everything else as hexadecimal (`raw`/`hex`).
const ASCII_DATA_LIMIT: usize = 4096;
/// The base 64 `<appearance>` text for annotation `d`'s appearance dictionary, or `None` when it
/// has no normal appearance stream or is past the limits (the comment is then exported without
/// one).
///
/// XFDF has no appearance state (`/AS`), so where `/N`, `/R` or `/D` holds one stream per state
/// only the selected state's stream is written (ISO 32000-2 §12.5.5: `/AS` picks it); a normal
/// appearance with no selected stream has nothing to draw and nothing is written.
///
/// `budget` is what the export may still write (decoded XML), shared by all of its comments, as
/// import reads at most [`MAX_IMPORT_APPEARANCE_BYTES`]: an appearance past it is left out rather
/// than written for import to skip. It is charged for what is written.
pub(crate) fn encode(doc: &Document, d: &Dict, budget: &mut usize) -> Option<String> {
if !has_appearance(doc, d) {
return None;
}
let ap = d.get(b"AP").map(|a| doc.resolve(a))?;
let mut ap = ap.as_dict()?.clone();
let state = d.get(b"AS").and_then(|s| doc.resolve(s).as_name().map(<[u8]>::to_vec));
for key in [b"N".as_slice(), b"R", b"D"] {
let Some(entry) = ap.get(key).map(|e| doc.resolve(e)) else { continue };
let Object::Dict(states) = entry.as_ref() else { continue };
match state.as_deref().and_then(|s| states.get(s)).cloned() {
Some(selected) => ap.set(key.to_vec(), selected),
None if key == b"N" => return None,
None => {
ap.remove(key);
}
}
}
let limit = MAX_APPEARANCE_BYTES.min(*budget);
let mut writer = Writer { doc, out: String::new(), values: MAX_VALUES, path: Vec::new(), limit };
writer.dict(Some(b"AP"), &ap, 0)?;
// The checks while writing leave room for the closing tags and line breaks: the finished XML
// must fit what the reader accepts, or the comment would arrive without its appearance.
if writer.out.len() > limit {
return None;
}
*budget = budget.saturating_sub(writer.out.len());
let b64 = base64::engine::general_purpose::STANDARD.encode(writer.out.as_bytes());
// Line breaks every 76 characters, as base 64 text usually has them; the reader skips them.
let mut wrapped = String::with_capacity(b64.len() + b64.len() / 76 + 1);
for (i, c) in b64.chars().enumerate() {
if i > 0 && i % 76 == 0 {
wrapped.push('\n');
}
wrapped.push(c);
}
Some(wrapped)
}
struct Writer<'a> {
doc: &'a Document,
out: String,
values: usize,
/// The references being written, so a reference back to one of them ends the cycle.
path: Vec<ObjRef>,
/// At most this much XML.
limit: usize,
}
impl Writer<'_> {
fn key_attr(key: Option<&[u8]>) -> String {
key.map(|k| format!(" KEY=\"{}\"", esc(&pdfcraft_forms::name_text(k)))).unwrap_or_default()
}
fn spend(&mut self, depth: usize) -> Option<()> {
if depth > MAX_NESTING || self.out.len() > self.limit {
return None;
}
self.values = self.values.checked_sub(1)?;
Some(())
}
fn dict(&mut self, key: Option<&[u8]>, d: &Dict, depth: usize) -> Option<()> {
self.spend(depth)?;
self.out.push_str(&format!("<DICT{}>\n", Self::key_attr(key)));
for (k, v) in d.iter() {
self.value(Some(k), v, depth + 1)?;
}
self.out.push_str("</DICT>\n");
Some(())
}
fn value(&mut self, key: Option<&[u8]>, o: &Object, depth: usize) -> Option<()> {
self.spend(depth)?;
let k = Self::key_attr(key);
match o {
Object::Ref(r) => {
if self.path.contains(r) {
// A cycle: the object is already being written further up.
self.out.push_str(&format!("<NULL{k}/>\n"));
return Some(());
}
self.path.push(*r);
let target = self.doc.get(*r);
let written = self.value(key, &target, depth + 1);
self.path.pop();
written?;
}
Object::Null => self.out.push_str(&format!("<NULL{k}/>\n")),
Object::Bool(b) => self.out.push_str(&format!("<BOOL{k} VAL=\"{b}\"/>\n")),
Object::Int(i) => self.out.push_str(&format!("<INT{k} VAL=\"{i}\"/>\n")),
// Rust prints the shortest decimal that reads back as the same number, never with an
// exponent; a non-finite number (which PDF can't hold) is written as 0.
Object::Real(r) => self.out.push_str(&format!("<FIXED{k} VAL=\"{}\"/>\n", if r.is_finite() { *r } else { 0.0 })),
Object::Name(n) => self.out.push_str(&format!("<NAME{k} VAL=\"{}\"/>\n", esc(&pdfcraft_forms::name_text(n)))),
Object::String(s) => match std::str::from_utf8(&s.bytes) {
// Text XML can hold as it is (`esc` would drop other control characters).
Ok(t) if t.chars().all(|c| !(c < ' ' && !matches!(c, '\t' | '\n' | '\r')) && c != '\u{FFFE}' && c != '\u{FFFF}') => {
self.out.push_str(&format!("<STRING{k} VAL=\"{}\"/>\n", esc(t)));
}
_ => self.out.push_str(&format!("<STRING{k} ENCODING=\"HEX\" VAL=\"{}\"/>\n", hex_of(&s.bytes, false))),
},
Object::Array(a) => {
self.out.push_str(&format!("<ARRAY{k}>\n"));
for v in a {
self.value(None, v, depth + 1)?;
}
self.out.push_str("</ARRAY>\n");
}
// A page (or the page tree) reached from the appearance is the source document's, not
// the comment's: written in place it would carry every page along.
// `/Type` may itself be a reference (`resolve` follows a bounded number of them).
Object::Dict(d) if d.get(b"Type").is_some_and(|t| matches!(self.doc.resolve(t).as_name(), Some(b"Page" | b"Pages"))) => {
self.out.push_str(&format!("<NULL{k}/>\n"));
}
Object::Dict(d) => self.dict(key, d, depth)?,
Object::Stream(s) => {
// XFDF 3.0 counts a carriage return as printable, but XML reads one, alone or before
// a line feed, as a line feed (XML 1.0 §2.11): data holding one is written as
// hexadecimal so it comes back byte for byte (the 4 KiB rule "is not a rule").
let printable = s.raw.iter().all(|&b| (b > 32 && b < 127) || b == b'\n');
let ascii = s.raw.len() < ASCII_DATA_LIMIT && printable && !s.dict.contains(b"Filter");
// The data, in the encoding it is written in, must fit what is left of the budget
// (as text it is at least its own length; the finished XML is checked again).
let size = if ascii { s.raw.len() } else { s.raw.len().saturating_mul(2) };
if size.saturating_add(self.out.len()) > self.limit {
return None;
}
self.out.push_str(&format!("<STREAM{k}>\n"));
// `Length` follows from the data.
for (dk, dv) in s.dict.iter().filter(|(dk, _)| dk.as_slice() != b"Length") {
self.value(Some(dk), dv, depth + 1)?;
}
if ascii {
// Printable ASCII is valid UTF-8.
let text = String::from_utf8_lossy(&s.raw);
self.out.push_str(&format!("<DATA MODE=\"FILTERED\" ENCODING=\"ASCII\">{}</DATA>\n", esc_data(&text)));
} else {
self.out.push_str(&format!("<DATA MODE=\"RAW\" ENCODING=\"HEX\">{}</DATA>\n", hex_of(&s.raw, true)));
}
self.out.push_str("</STREAM>\n");
}
}
Some(())
}
}
/// XFDF 3.0 "Stream encoding", method 1: the XML control characters as entities; line breaks stay.
fn esc_data(s: &str) -> String {
s.replace('&', "&amp;").replace('<', "&lt;").replace('>', "&gt;").replace('"', "&quot;")
}
/// Bytes as upper-case hexadecimal, with a line feed after every 80 characters when `wrap`.
fn hex_of(bytes: &[u8], wrap: bool) -> String {
const DIGITS: &[u8; 16] = b"0123456789ABCDEF";
let mut out = String::with_capacity(bytes.len().saturating_mul(2).saturating_add(bytes.len() / 40));
for (i, b) in bytes.iter().enumerate() {
if wrap && i > 0 && i % 40 == 0 {
out.push('\n');
}
out.push(char::from(DIGITS[usize::from(b >> 4)]));
out.push(char::from(DIGITS[usize::from(b & 15)]));
}
out
}
/// Hexadecimal digits back to bytes, ignoring white space; `None` for any other character or an
/// odd number of digits.
fn bytes_of_hex(text: &str) -> Option<Vec<u8>> {
let digits: Vec<u8> = text.bytes().filter(|b| !b.is_ascii_whitespace()).collect();
if !digits.len().is_multiple_of(2) {
return None;
}
let value = |c: u8| char::from(c).to_digit(16).and_then(|d| u8::try_from(d).ok());
digits.as_chunks::<2>().0.iter().map(|[h, l]| Some(value(*h)? << 4 | value(*l)?)).collect()
}
/// An appearance read from XFDF, before it is put into a document.
#[derive(Debug, PartialEq)]
pub(crate) enum Node {
Value(Object),
Array(Vec<Node>),
Dict(Vec<(Vec<u8>, Node)>),
Stream(Vec<(Vec<u8>, Node)>, Vec<u8>),
}
/// Read the text of an `<appearance>` element: base 64, then the XML it holds. `budget` is what
/// the import may still read (decoded bytes) and is charged. The result is the `/AP` dictionary,
/// with a normal appearance (`/N`) that is one stream: XFDF has no appearance state (`/AS`), so a
/// dictionary of states would leave nothing selected to draw (ISO 32000-2 §12.5.5).
pub(crate) fn decode(text: &str, budget: &mut usize) -> Result<Node, String> {
let limit = MAX_APPEARANCE_BYTES.min(*budget);
// Base 64 is 4 characters for 3 bytes: refuse a text that decodes past the limit before
// decoding it.
let mut compact = String::new();
for c in text.chars().filter(|c| !c.is_whitespace()) {
if compact.len() / 4 * 3 > limit {
return Err(format!("the appearance is larger than {} MiB", limit >> 20));
}
compact.push(c);
}
let engine = base64::engine::GeneralPurpose::new(
&base64::alphabet::STANDARD,
base64::engine::GeneralPurposeConfig::new().with_decode_padding_mode(base64::engine::DecodePaddingMode::Indifferent),
);
let bytes = engine.decode(compact.as_bytes()).map_err(|e| format!("the appearance is not base 64: {e}"))?;
if bytes.len() > limit {
return Err(format!("the appearance is larger than {} MiB", limit >> 20));
}
*budget = budget.saturating_sub(bytes.len());
let xml = std::str::from_utf8(&bytes).map_err(|_| "the appearance is not UTF-8 text".to_string())?;
let xml = xml.trim_start_matches('\u{feff}');
check_form_xml_depth(xml).map_err(|e| e.to_string())?;
let parsed = parse_form_xml(xml).map_err(|e| format!("the appearance is not XML: {e}"))?;
let root = parsed.root_element();
if !root.tag_name().name().eq_ignore_ascii_case("DICT") {
return Err("the appearance is not a dictionary".into());
}
let mut values = MAX_VALUES;
let node = read(root, 0, &mut values)?;
let usable = match &node {
Node::Dict(entries) => entries.iter().any(|(k, v)| k.as_slice() == b"N" && matches!(v, Node::Stream(..))),
_ => false,
};
if !usable {
return Err("the appearance has no normal appearance stream".into());
}
Ok(node)
}
fn attr<'a>(node: roxmltree::Node<'a, '_>, name: &str) -> Option<&'a str> {
node.attributes().find(|a| a.name().eq_ignore_ascii_case(name)).map(|a| a.value())
}
fn read(node: roxmltree::Node, depth: usize, values: &mut usize) -> Result<Node, String> {
if depth > MAX_NESTING {
return Err(format!("the appearance nests objects more than {MAX_NESTING} deep"));
}
*values = values.checked_sub(1).ok_or_else(|| format!("the appearance holds more than {MAX_VALUES} values"))?;
let tag = node.tag_name().name().to_ascii_uppercase();
let val = || attr(node, "VAL").ok_or_else(|| format!("{tag} has no VAL"));
let entries = |values: &mut usize| -> Result<Vec<(Vec<u8>, Node)>, String> {
let mut out = Vec::new();
for child in node.children().filter(|c| c.is_element() && !c.tag_name().name().eq_ignore_ascii_case("DATA")) {
let key = attr(child, "KEY").ok_or_else(|| format!("an entry of {tag} has no KEY"))?;
out.push((pdfcraft_forms::name_bytes(key), read(child, depth + 1, values)?));
}
Ok(out)
};
Ok(match tag.as_str() {
"DICT" => Node::Dict(entries(values)?),
"ARRAY" => {
let mut items = Vec::new();
for child in node.children().filter(|c| c.is_element()) {
items.push(read(child, depth + 1, values)?);
}
Node::Array(items)
}
"STREAM" => {
let mut dict = entries(values)?;
let data = node.children().find(|c| c.is_element() && c.tag_name().name().eq_ignore_ascii_case("DATA")).ok_or("a stream has no DATA")?;
let text: String = data.children().filter(|c| c.is_text()).filter_map(|c| c.text()).collect();
let bytes = match attr(data, "ENCODING").map(str::to_ascii_lowercase).as_deref() {
Some("hex") => bytes_of_hex(&text).ok_or("stream data is not hexadecimal")?,
Some("ascii") | None => text.into_bytes(),
Some(other) => return Err(format!("unknown stream encoding {other}")),
};
// `filtered` data is already decoded: the filters no longer apply to it.
if attr(data, "MODE").is_some_and(|m| m.eq_ignore_ascii_case("filtered")) {
dict.retain(|(k, _)| !matches!(k.as_slice(), b"Filter" | b"DecodeParms" | b"FFilter" | b"FDecodeParms"));
}
dict.retain(|(k, _)| k.as_slice() != b"Length");
Node::Stream(dict, bytes)
}
"NAME" => Node::Value(Object::Name(pdfcraft_forms::name_bytes(val()?))),
"INT" => Node::Value(Object::Int(val()?.trim().parse::<i64>().map_err(|_| "INT is not an integer")?)),
"FIXED" => {
let v = val()?.trim().parse::<f64>().ok().filter(|v| v.is_finite()).ok_or("FIXED is not a number")?;
Node::Value(Object::Real(v))
}
"BOOL" => Node::Value(Object::Bool(match val()?.trim() {
"true" => true,
"false" => false,
_ => return Err("BOOL is neither true nor false".into()),
})),
"STRING" => {
let v = val()?;
let bytes = if attr(node, "ENCODING").is_some_and(|e| e.eq_ignore_ascii_case("hex")) {
bytes_of_hex(v).ok_or("STRING is not hexadecimal")?
} else {
v.as_bytes().to_vec()
};
Node::Value(Object::String(PdfString::literal(bytes)))
}
"NULL" => Node::Value(Object::Null),
other => return Err(format!("unknown appearance element {other}")),
})
}
/// Put a read appearance into `doc`: each stream becomes an object of its own. Returns the `/AP`
/// value. Null dictionary entries are left out (ISO 32000-2 §7.3.7: they mean the entry is absent).
pub(crate) fn materialize(doc: &mut Document, node: Node) -> Object {
let dict = |doc: &mut Document, entries: Vec<(Vec<u8>, Node)>| {
let mut d = Dict::new();
for (k, v) in entries {
let v = materialize(doc, v);
if !matches!(v, Object::Null) {
d.set(k, v);
}
}
d
};
match node {
Node::Value(o) => o,
Node::Array(items) => Object::Array(items.into_iter().map(|n| materialize(doc, n)).collect()),
Node::Dict(entries) => Object::Dict(dict(doc, entries)),
Node::Stream(entries, raw) => {
let d = dict(doc, entries);
Object::Ref(doc.add(Object::Stream(pdfcraft_cos::Stream { dict: d, raw: raw.into() })))
}
}
}
/// Whether an annotation dictionary carries an appearance the document can draw: `/AP` with a
/// normal appearance that is a stream, or a dictionary of streams (one per state).
pub(crate) fn has_appearance(doc: &Document, d: &Dict) -> bool {
let Some(ap) = d.get(b"AP").map(|a| doc.resolve(a)) else { return false };
let Some(normal) = ap.as_dict().and_then(|ap| ap.get(b"N")).map(|n| doc.resolve(n)) else { return false };
match normal.as_ref() {
Object::Stream(_) => true,
Object::Dict(states) => !states.is_empty() && states.iter().all(|(_, s)| matches!(doc.resolve(s).as_ref(), Object::Stream(_))),
_ => false,
}
}
crates/xfdf/src/fdf.rs
+77 -12
View File
@@ -86,13 +86,16 @@ impl<'a> FdfCopy<'a> {
Ok(())
}
/// Skip source-document links and appearances without cloning their values. The
/// dictionary and every retained entry are charged before any owned copy is made.
/// Skip source-document links without cloning their values. A stamp's appearance (`/AP`) is
/// copied with its streams and resources, so the stamp looks the same where it is imported;
/// other comments carry their formatting entries and leave the appearance out, as XFDF does.
/// The dictionary and every retained entry are charged before any owned copy is made.
fn annotation_dict(&mut self, dict: &Dict) -> Result<Dict, DataError> {
self.charge_value(0)?;
let stamp = dict.name(b"Subtype") == Some(b"Stamp".as_slice());
let mut out = Dict::new();
for (key, value) in dict.iter() {
if matches!(key.as_slice(), b"P" | b"AP" | b"Popup" | b"IRT" | b"Parent" | b"FS" | b"Sound") {
if matches!(key.as_slice(), b"P" | b"Popup" | b"IRT" | b"Parent" | b"FS" | b"Sound") || (key.as_slice() == b"AP" && !stamp) {
continue;
}
self.dict_entry(&mut out, key, value, 1)?;
@@ -202,7 +205,7 @@ impl<'a> FdfCopy<'a> {
}
}
/// FDF with field values and/or comments (appearances are not carried). Each comment is an
/// FDF with field values and/or comments, a stamp with its appearance. Each comment is an
/// indirect object of the FDF, so a reply's `/IRT` can point at its parent's object there
/// (§12.7.8.3.1). Nested values are copied and renumbered, including shared or cyclic graphs.
/// Source `/Page` and `/Pages` dictionaries become null at every copy depth.
@@ -356,27 +359,69 @@ impl ImportBudget {
}
}
/// Where [`deep`] copies to: the document, and the streams already copied into it (by their
/// reference in the FDF), so a stream two comments share is copied once.
struct DeepTarget<'a> {
doc: &'a mut Document,
streams: HashMap<ObjRef, ObjRef>,
}
/// How deeply [`deep`] follows a comment's objects, references included. A form that draws a form
/// costs four levels (the reference, the stream's dictionary, `/Resources`, `/XObject`), and
/// export writes a chain of them however long, so this leaves room for one sixty forms deep.
const MAX_COPY_DEPTH: usize = 4 * MAX_DEPTH;
/// Copy an object out of another document, resolving references. A reference back to an object
/// being copied becomes null.
fn deep(src: &Document, o: &Object, depth: usize, path: &mut Vec<ObjRef>, budget: &mut ImportBudget) -> Result<Object, DataError> {
/// being copied becomes null. A stream (an appearance and the images and forms it draws) becomes
/// an object of `dst`, its data charged to the budget. Objects nested past [`MAX_COPY_DEPTH`] are
/// an error, not a partial copy: the comment would arrive with part of its appearance missing.
fn deep(
src: &Document,
dst: &mut DeepTarget,
o: &Object,
depth: usize,
path: &mut Vec<ObjRef>,
budget: &mut ImportBudget,
) -> Result<Object, DataError> {
budget.spend(1, 0)?;
if depth > 16 {
return Ok(Object::Null);
if depth > MAX_COPY_DEPTH {
return Err(DataError::Malformed(format!("a comment's objects nest more than {MAX_COPY_DEPTH} levels deep")));
}
Ok(match o {
Object::Ref(r) => {
if path.contains(r) {
return Ok(Object::Null);
}
if let Some(copied) = dst.streams.get(r) {
return Ok(Object::Ref(*copied));
}
let target = src.get(*r);
if let Object::Stream(stream) = target.as_ref() {
budget.spend(0, stream.raw.len())?;
path.push(*r);
let dict = deep(src, dst, &Object::Dict(stream.dict.clone()), depth + 1, path, budget);
path.pop();
let Object::Dict(mut dict) = dict? else { return Ok(Object::Null) };
dict.remove(b"Length");
let copied = dst.doc.add(Object::Stream(pdfcraft_cos::Stream { dict, raw: stream.raw.clone() }));
dst.streams.insert(*r, copied);
return Ok(Object::Ref(copied));
}
path.push(*r);
let copy = deep(src, &src.get(*r), depth + 1, path, budget);
let copy = deep(src, dst, &target, depth + 1, path, budget);
path.pop();
copy?
}
Object::Array(a) => Object::Array(a.iter().map(|x| deep(src, x, depth + 1, path, budget)).collect::<Result<_, _>>()?),
Object::Array(a) => Object::Array(a.iter().map(|x| deep(src, dst, x, depth + 1, path, budget)).collect::<Result<_, _>>()?),
// A page or page tree the comment reaches is the file's, not the comment's: copied in, it
// would bring its contents along (as export, which nulls them; `resolve` bounds the hops
// of an indirect /Type).
Object::Dict(d) if d.get(b"Type").is_some_and(|t| matches!(src.resolve(t).as_name(), Some(b"Page" | b"Pages"))) => Object::Null,
Object::Dict(d) => {
Object::Dict(d.iter().map(|(k, v)| Ok((k.clone(), deep(src, v, depth + 1, path, budget)?))).collect::<Result<_, DataError>>()?)
Object::Dict(d.iter().map(|(k, v)| Ok((k.clone(), deep(src, dst, v, depth + 1, path, budget)?))).collect::<Result<_, DataError>>()?)
}
// A stream is always an indirect object (ISO 32000-2 §7.3.8); one held directly is not
// read.
Object::Stream(_) => Object::Null,
Object::String(s) => {
budget.spend(0, s.bytes.len())?;
@@ -469,8 +514,17 @@ pub(crate) fn import_fdf(doc: &mut Document, bytes: &[u8]) -> Result<Report, Dat
let pages: Vec<ObjRef> = pdfcraft_model::pages(doc).iter().map(|p| p.obj).collect();
let mut placed: HashMap<ObjRef, ObjRef> = HashMap::new();
let mut later: HashMap<ObjRef, (Option<ObjRef>, Option<String>)> = HashMap::new();
let mut copied_streams: HashMap<ObjRef, ObjRef> = HashMap::new();
for a in body.get(b"Annots").map(|a| fdf.resolve(a)).and_then(|a| a.as_array().cloned()).unwrap_or_default() {
let Some(mut src) = fdf.resolve(&a).as_dict().cloned() else { continue };
// Only a comment of a known type on a page of this document is copied in: its streams
// become objects of the document.
let on_page = src.get(b"Page").and_then(|p| fdf.resolve(p).as_int()).and_then(|p| usize::try_from(p).ok()).is_some_and(|p| p < pages.len());
let known =
src.get(b"Subtype").map(|s| fdf.resolve(s)).is_some_and(|s| s.as_name().is_some_and(|s| SUBTYPES.iter().any(|(x, _)| x.as_bytes() == s)));
if !on_page || !known {
continue;
}
// The parent is another comment: link it below rather than copying it in.
let irt = src.get(b"IRT").cloned();
src.remove(b"IRT");
@@ -479,8 +533,19 @@ pub(crate) fn import_fdf(doc: &mut Document, bytes: &[u8]) -> Result<Report, Dat
// placed on the page below, as the XFDF import does.
let popup = src.get(b"Popup").map(|p| fdf.resolve(p)).and_then(|p| p.as_dict().cloned());
src.remove(b"Popup");
// The comment's page is `/Page` here (§12.7.8.3.4, Table 254); a `/P` the file holds
// anyway points into the file and is replaced below, so it is not copied (nor what it
// reaches).
src.remove(b"P");
// A redaction mark is always drawn by PdfCraft (see `place`): its appearance is not copied.
if src.get(b"Subtype").is_some_and(|s| fdf.resolve(s).as_name() == Some(b"Redact".as_slice())) {
src.remove(b"AP");
}
let mut path: Vec<ObjRef> = a.as_ref().into_iter().collect();
let Object::Dict(mut d) = deep(&fdf, &Object::Dict(src), 0, &mut path, &mut budget)? else { continue };
let mut target = DeepTarget { doc: &mut *doc, streams: std::mem::take(&mut copied_streams) };
let copy = deep(&fdf, &mut target, &Object::Dict(src), 0, &mut path, &mut budget);
copied_streams = target.streams;
let Object::Dict(mut d) = copy? else { continue };
let Some(page) = d.get(b"Page").and_then(Object::as_int).and_then(|p| usize::try_from(p).ok()).and_then(|p| pages.get(p).copied()) else {
continue;
};
crates/xfdf/src/lib.rs
+18 -3
View File
@@ -2,8 +2,10 @@
//!
//! - **XFDF** (ISO 19444-1, Acrobat's "Export all to data file" / "Import comments", and form
//! data): comments with their geometry, colours, flags, authors, dates, replies, pop-ups,
//! ink, quadrilaterals, line ends and callouts; field values, nested by name.
//! - **FDF** (ISO 32000-2 §12.7.8): fields and comments in PDF syntax.
//! ink, quadrilaterals, line ends, callouts, formatting (rich text, default style, border
//! style and dashes, rotation, justification, fringe, line leaders) and a stamp's appearance;
//! field values, nested by name.
//! - **FDF** (ISO 32000-2 §12.7.8): fields and comments in PDF syntax, with their appearances.
//! - Form data as **XML**, **CSV** and **tab-delimited text** (Acrobat's Export data formats).
//!
//! Import merges: a comment whose name (`/NM`) already exists on its page replaces it; field
@@ -18,6 +20,7 @@ use std::io::Write as _;
use pdfcraft_cos::{Dict, Document, ObjRef, Object, PdfString};
use pdfcraft_forms::{FieldKind, FieldValue};
mod appearance;
mod common;
mod fdf;
mod xfdf;
@@ -36,6 +39,8 @@ use xfdf::{apply_values, import_xfdf};
#[cfg(test)]
mod tests;
#[cfg(test)]
mod tests_format;
#[derive(Debug, thiserror::Error, PartialEq)]
pub enum DataError {
@@ -85,6 +90,9 @@ pub struct Report {
pub fields: usize,
/// Field values the form refused (format, validation, unknown options).
pub rejected: Vec<String>,
/// What the import left out of a comment it did import, each as "name: why" (an appearance
/// it could not read: the comment is drawn from its other entries instead).
pub skipped: Vec<String>,
}
/// Form data as XML, CSV or tab-delimited text (Acrobat's Export data formats).
@@ -330,8 +338,15 @@ fn tab_delimited(t: &str) -> Result<Vec<(String, Vec<String>)>, DataError> {
}
/// Import comments and/or field values from XFDF, FDF, XML, CSV or tab-delimited text
/// (detected from the content).
/// (detected from the content). All or nothing: on an error `doc` is left as it was.
pub fn import(doc: &mut Document, bytes: &[u8]) -> Result<Report, DataError> {
let mut work = doc.clone();
let report = import_into(&mut work, bytes)?;
*doc = work;
Ok(report)
}
fn import_into(doc: &mut Document, bytes: &[u8]) -> Result<Report, DataError> {
let report = if bytes.starts_with(b"%FDF") {
import_fdf(doc, bytes)?
} else {
crates/xfdf/src/tests.rs
+17 -2
View File
@@ -567,8 +567,10 @@ fn fdf_keeps_replies_and_review_status_on_their_note() {
// Every cross-reference entry points at its object (other readers trust the table).
let tail = std::str::from_utf8(&fdf[fdf.windows(5).position(|w| w == b"xref\n").unwrap()..]).unwrap();
let xref: usize = tail.lines().skip_while(|l| *l != "startxref").nth(1).unwrap().parse().unwrap();
assert!(fdf[xref..].starts_with(b"xref\n0 6\n"), "{tail}");
for (num, line) in tail.lines().skip(3).take(5).enumerate() {
// The root, the four comments and their appearance streams.
let count: usize = tail.lines().nth(1).and_then(|l| l.strip_prefix("0 ")).unwrap().parse().unwrap();
assert!(fdf[xref..].starts_with(b"xref\n") && count >= 6, "{tail}");
for (num, line) in tail.lines().skip(3).take(count - 1).enumerate() {
let at: usize = line[..10].parse().unwrap();
assert!(fdf[at..].starts_with(format!("{} 0 obj\n", num + 1).as_bytes()), "object {}: {line}", num + 1);
}
@@ -2284,6 +2286,19 @@ fn importing_comments_as_copies_adds_them_on_the_page_with_fresh_names() {
assert_eq!(summaries(&other).len(), 4, "the named import replaced its own earlier copy");
}
/// Comment import never takes the file's field values, whatever the placement (a non-finite
/// offset, which is ignored, and no page or prefix made it the full import).
#[test]
fn comment_import_leaves_field_values_alone() {
let xfdf = r#"<?xml version="1.0"?><xfdf xmlns="http://ns.adobe.com/xfdf/"><annots><text page="0" rect="10,10,30,30" name="n"/></annots><fields><field name="Name"><value>from the file</value></field></fields></xfdf>"#;
for offset in [Some([f64::NAN, 0.0]), None] {
let mut doc = one_field_named("Name", "mine");
let r = import_comments(&mut doc, xfdf, None, None, offset).unwrap();
assert_eq!((r.comments, r.fields), (1, 0), "{offset:?}: {r:?}");
assert_eq!(values(&doc), [("Name".to_string(), vec!["mine".to_string()])], "{offset:?}");
}
}
/// Other tools write comments straight into the page's `/Annots` array, without an object of
/// their own: they copy like any other, and two source comments sharing a name give two copies.
#[test]
crates/xfdf/src/tests_format.rs
+1216
View File
File diff suppressed because it is too large. Load diff
crates/xfdf/src/xfdf.rs
+456 -48
View File
@@ -1,6 +1,6 @@
use super::common::{
FLAGS, MAX_DEPTH, SUBTYPES, arr, check_form_xml_depth, csv, esc, hex, n, nums_of, parse_form_xml, parse_hex, quad_has_area, rect_strict, text_of,
walk_xfdf,
walk_xfdf, xml_nesting_depth,
};
use super::*;
@@ -99,6 +99,226 @@ fn read_own_keys(node: roxmltree::Node, d: &mut Dict) {
}
}
/// Rich text deeper than this is written as text rather than as markup, so the XFDF it goes into
/// stays well inside the nesting the reader accepts (`MAX_XML_DEPTH`).
const MAX_RICH_DEPTH: usize = 512;
/// Rich text (`/RC`) as `<contents-richtext>` holds it: its root element alone (no XML
/// declaration), when the text is well-formed XML; `None` otherwise.
fn rich_text_element(rc: &str) -> Option<&str> {
let t = rc.trim_start_matches('\u{feff}');
if xml_nesting_depth(t) > MAX_RICH_DEPTH {
return None;
}
let parsed = parse_form_xml(t).ok()?;
t.get(parsed.root_element().range())
}
/// The formatting attributes of a comment: XFDF 3.0 "Border style attributes" (`width`, `style`,
/// `dashes`), "Border effect attributes" (`style="cloudy"`, `intensity`), and the free text
/// (`rotation`, `justification`), stamp (`rotation`), circle/square/caret (`fringe`), line
/// (`leaderLength`, `leaderExtend`, `leader-offset`, `caption`, `caption-style`,
/// `caption-offset-h`/`-v`), caret (`symbol`) and redaction (`overlay-text`,
/// `overlay-text-repeat`, `justification`) attributes.
fn format_attrs(doc: &Document, d: &Dict, sub: &[u8], attrs: &mut String) {
let num = |k: &[u8]| d.get(k).and_then(|o| doc.resolve(o).as_f64()).filter(|v| v.is_finite());
let name = |k: &[u8]| d.get(k).and_then(|o| doc.resolve(o).as_name().map(<[u8]>::to_vec));
let list = |v: &[f64]| v.iter().map(|x| n(*x)).collect::<Vec<_>>().join(",");
// The border: /BS, or (without one) the older /Border array [h v width dashes].
let (width, style, dashes) = match d.get(b"BS").map(|b| doc.resolve(b)).and_then(|b| b.as_dict().cloned()) {
Some(bs) => (
bs.get(b"W").and_then(|w| doc.resolve(w).as_f64()),
bs.get(b"S").and_then(|s| doc.resolve(s).as_name().map(<[u8]>::to_vec)),
nums_of(doc, bs.get(b"D")),
),
None => {
let border = d.get(b"Border").map(|b| doc.resolve(b)).and_then(|b| b.as_array().cloned()).unwrap_or_default();
let dash = border.get(3).map(|x| nums_of(doc, Some(x))).unwrap_or_default();
(border.get(2).and_then(|w| doc.resolve(w).as_f64()), (!dash.is_empty()).then(|| b"D".to_vec()), dash)
}
};
if let Some(w) = width.filter(|w| w.is_finite()) {
let _ = write!(attrs, " width=\"{}\"", n(w));
}
// One `style` attribute names both: a cloudy border effect (/BE /S /C) takes it.
let cloudy = d.get(b"BE").map(|o| doc.resolve(o)).and_then(|o| o.as_dict().cloned()).filter(|b| b.name(b"S") == Some(b"C".as_slice()));
if let Some(be) = cloudy {
attrs.push_str(" style=\"cloudy\"");
if let Some(i) = be.get(b"I").and_then(|i| doc.resolve(i).as_f64()).filter(|i| i.is_finite()) {
let _ = write!(attrs, " intensity=\"{}\"", n(i));
}
} else if let Some(s) = match style.as_deref() {
Some(b"D") => Some("dash"),
Some(b"B") => Some("bevelled"),
Some(b"I") => Some("inset"),
Some(b"U") => Some("underline"),
_ => None,
} {
let _ = write!(attrs, " style=\"{s}\"");
}
// The dash array (/D) whatever the style, as `dashes` maps it; only `style="dash"` draws it.
if !dashes.is_empty() && dashes.iter().all(|v| v.is_finite()) {
let _ = write!(attrs, " dashes=\"{}\"", list(&dashes));
}
if let Some(r) = d.get(b"Rotate").and_then(|o| doc.resolve(o).as_int()) {
let _ = write!(attrs, " rotation=\"{r}\"");
}
if let Some(q) = d.get(b"Q").and_then(|o| doc.resolve(o).as_int()) {
// Free text names its justification; a redaction mark gives the number (XFDF 3.0).
match (sub, q) {
(b"FreeText", 0) => attrs.push_str(" justification=\"left\""),
(b"FreeText", 1) => attrs.push_str(" justification=\"centered\""),
(b"FreeText", 2) => attrs.push_str(" justification=\"right\""),
(b"FreeText", _) => {}
(_, q) => {
let _ = write!(attrs, " justification=\"{q}\"");
}
}
}
let rd = nums_of(doc, d.get(b"RD"));
if rd.len() == 4 && rd.iter().all(|v| v.is_finite()) {
let _ = write!(attrs, " fringe=\"{}\"", list(&rd));
}
if let Some(ll) = num(b"LL") {
let _ = write!(attrs, " leaderLength=\"{}\"", n(ll));
}
if let Some(lle) = num(b"LLE") {
let _ = write!(attrs, " leaderExtend=\"{}\"", n(lle));
}
if let Some(llo) = num(b"LLO") {
let _ = write!(attrs, " leader-offset=\"{}\"", n(llo));
}
if let Some(Object::Bool(cap)) = d.get(b"Cap").map(|o| doc.resolve(o)).as_deref() {
let _ = write!(attrs, " caption=\"{}\"", if *cap { "yes" } else { "no" });
}
if let Some(cp) = name(b"CP") {
let _ = write!(attrs, " caption-style=\"{}\"", esc(&String::from_utf8_lossy(&cp)));
}
if let [h, v] = nums_of(doc, d.get(b"CO")).as_slice()
&& h.is_finite()
&& v.is_finite()
{
let _ = write!(attrs, " caption-offset-h=\"{}\" caption-offset-v=\"{}\"", n(*h), n(*v));
}
match name(b"Sy").as_deref() {
Some(b"P") => attrs.push_str(" symbol=\"paragraph\""),
Some(b"None") => attrs.push_str(" symbol=\"none\""),
_ => {}
}
if let Some(t) = text_of(doc, d, b"OverlayText") {
let _ = write!(attrs, " overlay-text=\"{}\"", esc(&t));
}
if let Some(Object::Bool(repeat)) = d.get(b"Repeat").map(|o| doc.resolve(o)).as_deref() {
let _ = write!(attrs, " overlay-text-repeat=\"{repeat}\"");
}
}
/// The formatting attributes of an XFDF comment element back as dictionary entries (see
/// [`format_attrs`]). Values that are not numbers, or out of range, are left out.
fn read_format(node: roxmltree::Node, d: &mut Dict) {
let num = |a: &str| node.attribute(a).and_then(|v| v.trim().parse::<f64>().ok()).filter(|v| v.is_finite());
let style = node.attribute("style").map(str::trim);
let dashes = node.attribute("dashes").map(csv).filter(|v| !v.is_empty() && v.iter().all(|x| *x >= 0.0) && v.iter().any(|x| *x > 0.0));
let bs_style = match style {
Some("dash") => Some("D"),
Some("bevelled" | "beveled") => Some("B"),
Some("inset") => Some("I"),
Some("underline") => Some("U"),
Some("solid") => Some("S"),
_ => None,
};
let width = num("width");
if width.is_some() || bs_style.is_some() || dashes.is_some() {
let mut bs = Dict::new();
// XFDF 3.0: the width is 1 and the style solid unless given.
bs.set(b"W".to_vec(), Object::Real(width.unwrap_or(1.0).max(0.0)));
bs.set(b"S".to_vec(), Object::name(bs_style.unwrap_or("S")));
if let Some(dv) = &dashes {
bs.set(b"D".to_vec(), arr(dv));
}
d.set(b"BS".to_vec(), Object::Dict(bs));
}
if style == Some("cloudy") {
let mut be = Dict::new();
be.set(b"S".to_vec(), Object::name("C"));
let intensity = num("intensity").unwrap_or(1.0);
be.set(b"I".to_vec(), Object::Real(intensity.clamp(0.0, 2.0)));
d.set(b"BE".to_vec(), Object::Dict(be));
}
if let Some(r) = node.attribute("rotation").and_then(|r| r.trim().parse::<i32>().ok()) {
d.set(b"Rotate".to_vec(), Object::Int(i64::from(r)));
}
let q = match node.attribute("justification").map(str::trim) {
Some("left" | "0") => Some(0),
Some("centered" | "center" | "1") => Some(1),
Some("right" | "2") => Some(2),
_ => None,
};
if let Some(q) = q {
d.set(b"Q".to_vec(), Object::Int(q));
}
if let Some(rd) = node.attribute("fringe").map(csv).filter(|r| r.len() == 4) {
d.set(b"RD".to_vec(), arr(&rd));
}
if let Some(ll) = num("leaderLength") {
d.set(b"LL".to_vec(), Object::Real(ll));
}
if let Some(lle) = num("leaderExtend").filter(|v| *v >= 0.0) {
d.set(b"LLE".to_vec(), Object::Real(lle));
}
if let Some(llo) = num("leader-offset").filter(|v| *v >= 0.0) {
d.set(b"LLO".to_vec(), Object::Real(llo));
}
match node.attribute("caption").map(str::trim) {
Some("yes" | "true") => d.set(b"Cap".to_vec(), Object::Bool(true)),
Some("no" | "false") => d.set(b"Cap".to_vec(), Object::Bool(false)),
_ => {}
}
if let Some(cp) = node.attribute("caption-style").map(str::trim).filter(|c| matches!(*c, "Inline" | "Top")) {
d.set(b"CP".to_vec(), Object::name(cp));
}
let (h, v) = (num("caption-offset-h"), num("caption-offset-v"));
if h.is_some() || v.is_some() {
d.set(b"CO".to_vec(), arr(&[h.unwrap_or(0.0), v.unwrap_or(0.0)]));
}
match node.attribute("symbol").map(str::trim) {
Some("paragraph" | "P") => d.set(b"Sy".to_vec(), Object::name("P")),
Some("none" | "None") => d.set(b"Sy".to_vec(), Object::name("None")),
_ => {}
}
if let Some(t) = node.attribute("overlay-text") {
d.set(b"OverlayText".to_vec(), PdfString::text(t));
}
match node.attribute("overlay-text-repeat").map(str::trim) {
Some("true" | "yes") => d.set(b"Repeat".to_vec(), Object::Bool(true)),
Some("false" | "no") => d.set(b"Repeat".to_vec(), Object::Bool(false)),
_ => {}
}
}
/// A `<contents-richtext>` element's plain text, a line per paragraph: what `/Contents` gets
/// when the element comes without `<contents>`.
fn rich_plain(node: roxmltree::Node) -> String {
let mut out = String::new();
for n in node.descendants() {
// A line break is a line of its own, so two in a row leave a blank line; a paragraph
// starts on a new line.
if n.is_element() && n.tag_name().name() == "br" {
out.push('\r');
} else if n.is_element() && matches!(n.tag_name().name(), "p" | "div" | "li") {
if !out.is_empty() && !out.ends_with('\r') {
out.push('\r');
}
} else if let Some(t) = n.text().filter(|_| n.is_text()) {
// Indentation between elements is not text.
if !(t.trim().is_empty() && t.contains('\n')) {
out.push_str(&t.replace("\r\n", "\r").replace('\n', "\r"));
}
}
}
out.trim_end_matches('\r').to_string()
}
/// The name of every annotation, for `inreplyto`: its /NM, or, for one without one (other tools
/// leave /NM out) that is in `only` or that another annotation replies to or is grouped under
/// (`/IRT`, ISO 32000-2 §12.5.6.2; a Replace Text pair is such a group), a name new to the
@@ -143,15 +363,42 @@ fn annotation_names(doc: &Document, pages: &[pdfcraft_model::Page], only: Option
names
}
/// The elements inside a comment's XFDF element: its text, default appearance, ink, vertices
/// and pop-up (on page `pi`).
fn xfdf_annot_children(doc: &Document, out: &mut String, d: &Dict, pi: usize) {
if let Some(c) = text_of(doc, d, b"Contents") {
let _ = write!(out, "<contents>{}</contents>", esc(&c));
/// The elements inside a comment's XFDF element: its text (plain and rich), default appearance
/// and style, ink, vertices, appearance (with `ap`) and pop-up (on page `pi`).
fn xfdf_annot_children(doc: &Document, out: &mut String, d: &Dict, pi: usize, ap: Option<&mut usize>) {
let rc = text_of(doc, d, b"RC");
match text_of(doc, d, b"Contents") {
Some(c) => {
let _ = write!(out, "<contents>{}</contents>", esc(&c));
}
// Rich text written as text (below) is read back as rich text only beside `<contents>`;
// alone it would become the comment's plain text.
None if rc.as_deref().is_some_and(|rc| rich_text_element(rc).is_none()) => out.push_str("<contents></contents>"),
None => {}
}
if let Some(rc) = rc {
match rich_text_element(&rc) {
// Rich text is an XML (XHTML) document (ISO 32000-2 Table 172, RC): its element goes in as markup.
Some(markup) => {
let _ = write!(out, "<contents-richtext>{markup}</contents-richtext>");
}
// Text that is not well-formed XML goes in as text, so nothing is lost.
None => {
let _ = write!(out, "<contents-richtext>{}</contents-richtext>", esc(&rc));
}
}
}
if let Some(da) = text_of(doc, d, b"DA") {
let _ = write!(out, "<defaultappearance>{}</defaultappearance>", esc(&da));
}
if let Some(ds) = text_of(doc, d, b"DS") {
let _ = write!(out, "<defaultstyle>{}</defaultstyle>", esc(&ds));
}
if let Some(budget) = ap
&& let Some(b64) = super::appearance::encode(doc, d, budget)
{
let _ = write!(out, "<appearance>{b64}</appearance>");
}
let ink = d.get(b"InkList").map(|o| doc.resolve(o)).and_then(|o| o.as_array().cloned()).unwrap_or_default();
if !ink.is_empty() {
out.push_str("<inklist>");
@@ -191,6 +438,8 @@ fn xfdf_annots(doc: &Document, out: &mut String, only: Option<&HashSet<(usize, u
// NM of every annotation, for inreplyto. A selected or replied-to annotation without one
// (other tools leave /NM out) gets a name for this export, so its replies still find it.
let names = annotation_names(doc, &pages, only);
// What the import reads of appearances, over all of the comments.
let mut appearance_budget = super::appearance::MAX_IMPORT_APPEARANCE_BYTES;
for (pi, p) in pages.iter().enumerate() {
for (ai, a) in p.dict.get(b"Annots").map(|a| doc.resolve(a)).and_then(|a| a.as_array().cloned()).unwrap_or_default().into_iter().enumerate() {
if only.is_some_and(|only| !only.contains(&(pi, ai))) {
@@ -241,9 +490,7 @@ fn xfdf_annots(doc: &Document, out: &mut String, only: Option<&HashSet<(usize, u
if let Some(o) = d.get(b"CA").and_then(|o| doc.resolve(o).as_f64()) {
let _ = write!(attrs, " opacity=\"{}\"", n(o));
}
if let Some(w) = d.get(b"BS").and_then(|b| doc.resolve(b).as_dict().and_then(|b| b.get(b"W").and_then(Object::as_f64))) {
let _ = write!(attrs, " width=\"{}\"", n(w));
}
format_attrs(doc, &d, sub, &mut attrs);
let quads = nums_of(doc, d.get(b"QuadPoints"));
if !quads.is_empty() {
let _ = write!(attrs, " coords=\"{}\"", quads.iter().map(|v| n(*v)).collect::<Vec<_>>().join(","));
@@ -262,26 +509,14 @@ fn xfdf_annots(doc: &Document, out: &mut String, only: Option<&HashSet<(usize, u
let cl = nums_of(doc, d.get(b"CL"));
if cl.len() == 4 || cl.len() == 6 {
let _ = write!(attrs, " callout=\"{}\"", cl.iter().map(|v| n(*v)).collect::<Vec<_>>().join(","));
let rd = nums_of(doc, d.get(b"RD"));
if rd.len() == 4 {
let _ = write!(attrs, " fringe=\"{}\"", rd.iter().map(|v| n(*v)).collect::<Vec<_>>().join(","));
}
if let Some(le) = d.get(b"LE").map(|o| doc.resolve(o)).and_then(|o| o.as_name().map(|x| String::from_utf8_lossy(x).into_owned())) {
let _ = write!(attrs, " head=\"{}\"", esc(&le));
}
}
// What the comment is meant as (/IT): typewriter text, a callout, a cloud, an arrow.
// Written under both spellings XFDF files use for it, `intent` and `IT`.
if let Some(it) = d.get(b"IT").map(|o| doc.resolve(o)).and_then(|o| o.as_name().map(|x| String::from_utf8_lossy(x).into_owned())) {
let _ = write!(attrs, " intent=\"{}\"", esc(&it));
}
// A cloudy border (/BE /S /C) and how far its arcs bulge.
if let Some(be) =
d.get(b"BE").map(|o| doc.resolve(o)).and_then(|o| o.as_dict().cloned()).filter(|b| b.name(b"S") == Some(b"C".as_slice()))
{
attrs.push_str(" style=\"cloudy\"");
if let Some(i) = be.get(b"I").and_then(Object::as_f64) {
let _ = write!(attrs, " intensity=\"{}\"", n(i));
}
let _ = write!(attrs, " intent=\"{0}\" IT=\"{0}\"", esc(&it));
}
if let Some(irt) = d.get(b"IRT").and_then(Object::as_ref).and_then(|r| names.get(&r)) {
let _ = write!(attrs, " inreplyto=\"{}\"", esc(irt));
@@ -290,7 +525,13 @@ fn xfdf_annots(doc: &Document, out: &mut String, only: Option<&HashSet<(usize, u
let _ = write!(attrs, " pdfcraft:keys=\"{}\"", esc(&keys));
}
let _ = write!(out, "<{tag}{attrs}>");
xfdf_annot_children(doc, out, &d, pi);
// XFDF 3.0 carries a stamp's appearance, and none for a rect with no area; a copy and
// paste (`own`) carries every comment's, so the copy looks the same, except a
// redaction mark's, which import never reads (PdfCraft draws the mark) and which would
// only use up the budget the others share.
let has_area = rect.len() == 4 && rect[0] != rect[2] && rect[1] != rect[3];
let carried = has_area && sub != b"Redact" && (own || sub == b"Stamp");
xfdf_annot_children(doc, out, &d, pi, carried.then_some(&mut appearance_budget));
let _ = writeln!(out, "</{tag}>");
}
}
@@ -667,12 +908,7 @@ fn annot_from_xml(node: roxmltree::Node, subtype: &str, page_ref: ObjRef) -> Opt
if let Some(o) = node.attribute("opacity").and_then(|o| o.parse::<f64>().ok()) {
d.set(b"CA".to_vec(), Object::Real(o.clamp(0.0, 1.0)));
}
if let Some(w) = node.attribute("width").and_then(|w| w.parse::<f64>().ok()) {
let mut bs = Dict::new();
bs.set(b"W".to_vec(), Object::Real(w.max(0.0)));
bs.set(b"S".to_vec(), Object::name("S"));
d.set(b"BS".to_vec(), Object::Dict(bs));
}
read_format(node, &mut d);
if let Some(q) = node.attribute("coords") {
d.set(b"QuadPoints".to_vec(), arr(&csv(q)));
}
@@ -688,22 +924,14 @@ fn annot_from_xml(node: roxmltree::Node, subtype: &str, page_ref: ObjRef) -> Opt
Object::Array(vec![Object::name(node.attribute("head").unwrap_or("None")), Object::name(node.attribute("tail").unwrap_or("None"))]),
);
}
if let Some(it) = node.attribute("intent").filter(|it| !it.is_empty()) {
// The intent (/IT), spelt `intent` or `IT`.
let intent = node.attribute("intent").filter(|it| !it.is_empty()).or_else(|| node.attribute("IT").filter(|it| !it.is_empty()));
if let Some(it) = intent {
d.set(b"IT".to_vec(), Object::name(it));
}
if node.attribute("style") == Some("cloudy") {
let mut be = Dict::new();
be.set(b"S".to_vec(), Object::name("C"));
let intensity = node.attribute("intensity").and_then(|i| i.parse::<f64>().ok()).filter(|i| i.is_finite()).unwrap_or(1.0);
be.set(b"I".to_vec(), Object::Real(intensity.clamp(0.0, 2.0)));
d.set(b"BE".to_vec(), Object::Dict(be));
}
if let Some(cl) = node.attribute("callout").map(csv).filter(|c| c.len() == 4 || c.len() == 6) {
d.set(b"CL".to_vec(), arr(&cl));
if let Some(rd) = node.attribute("fringe").map(csv).filter(|r| r.len() == 4) {
d.set(b"RD".to_vec(), arr(&rd));
}
d.set(b"IT".to_vec(), Object::name(node.attribute("intent").unwrap_or("FreeTextCallout")));
d.set(b"IT".to_vec(), Object::name(intent.unwrap_or("FreeTextCallout")));
// A callout's /LE is one name, not the head/tail pair of a line.
match node.attribute("head") {
Some(h) => d.set(b"LE".to_vec(), Object::name(h)),
@@ -720,6 +948,9 @@ fn annot_from_xml(node: roxmltree::Node, subtype: &str, page_ref: ObjRef) -> Opt
"defaultappearance" => {
d.set(b"DA".to_vec(), PdfString::literal(child.text().unwrap_or("").as_bytes().to_vec()));
}
"defaultstyle" => {
d.set(b"DS".to_vec(), PdfString::text(&child.children().filter(|c| c.is_text()).filter_map(|c| c.text()).collect::<String>()));
}
"inklist" => {
let list: Vec<Object> = child
.children()
@@ -739,10 +970,120 @@ fn annot_from_xml(node: roxmltree::Node, subtype: &str, page_ref: ObjRef) -> Opt
_ => {}
}
}
if let Some(rich) = node.children().find(|c| c.has_tag_name("contents-richtext")) {
read_rich_text(rich, node.children().any(|c| c.has_tag_name("contents")), &mut d);
}
read_own_keys(node, &mut d);
Some(d)
}
/// `<contents-richtext>` back as `/RC` (XFDF 3.0 "The contents and contents-richtext elements in
/// annotations"): its markup, as written in the file, after an XML declaration. Plain text in it is the comment's text
/// (`/Contents`) when the element comes alone; beside `<contents>` it is kept as `/RC` (rich text
/// PdfCraft exported as text because it was not well-formed XML). Markup without `<contents>`
/// gives `/Contents` its plain text too, so the comment's text shows where rich text is not read.
fn read_rich_text(rich: roxmltree::Node, has_contents: bool, d: &mut Dict) {
let elements: Vec<roxmltree::Node> = rich.children().filter(|c| c.is_element()).collect();
let (Some(first), Some(last)) = (elements.first(), elements.last()) else {
let text: String = rich.children().filter(|c| c.is_text()).filter_map(|c| c.text()).collect();
if has_contents {
d.set(b"RC".to_vec(), PdfString::text(&text));
} else {
d.set(b"Contents".to_vec(), PdfString::text(&text.replace("\r\n", "\r").replace('\n', "\r")));
}
return;
};
let source = rich.document().input_text();
let Some(markup) = source.get(first.range().start..last.range().end) else { return };
let markup = match elements.as_slice() {
[_] => standalone(rich, markup),
_ => markup.to_string(),
};
// Rich text is an XML document of its own; export left its declaration out.
d.set(b"RC".to_vec(), PdfString::text(&format!("<?xml version=\"1.0\"?>{markup}")));
if !has_contents {
d.set(b"Contents".to_vec(), PdfString::text(&rich_plain(rich)));
}
}
/// Whether start tag `tag` (from `<` up to its closing `>`) declares attribute `name` (white
/// space may come before the `=`). Attribute values are skipped: `title=" xmlns:b='x'"` declares
/// nothing.
fn declares(tag: &str, name: &str) -> bool {
let mut quote = None;
let mut after_space = false;
for (i, c) in tag.char_indices() {
if let Some(q) = quote {
if c == q {
quote = None;
}
continue;
}
if c == '"' || c == '\'' {
quote = Some(c);
} else if after_space && tag.get(i..).and_then(|rest| rest.strip_prefix(name)).is_some_and(|a| a.trim_start().starts_with('=')) {
return true;
}
after_space = c.is_whitespace();
}
false
}
/// Where the start tag at the beginning of `markup` ends: its first `>` outside an attribute
/// value (the end of `markup` when there is none).
fn start_tag_end(markup: &str) -> usize {
let mut quote = None;
for (i, c) in markup.char_indices() {
match quote {
Some(q) if c == q => quote = None,
Some(_) => {}
None if c == '"' || c == '\'' => quote = Some(c),
None if c == '>' => return i,
None => {}
}
}
markup.len()
}
/// The one element `markup` (cut out of `rich`, a `<contents-richtext>`) as an XML
/// document of its own: the namespaces it inherits from the XFDF around it are declared on it,
/// the default one (XHTML declared on `<contents-richtext>`, say; not XFDF's own, which a
/// standalone document has no use for) and the prefixes it uses. Unchanged when nothing is
/// inherited, or when the declarations would not make it parse.
fn standalone(rich: roxmltree::Node, markup: &str) -> String {
// The start tag's name ends at white space, `/` or `>`.
let Some(name_end) = markup.get(1..).and_then(|t| t.find(|c: char| c.is_whitespace() || c == '/' || c == '>')).map(|i| i + 1) else {
return markup.to_string();
};
let start_tag = markup.get(..start_tag_end(markup)).unwrap_or(markup);
let mut declared = String::new();
// The default namespace in scope where the element sits: its own when it has no prefix, and
// its unprefixed descendants' when it has one.
if let Some(uri) = rich.namespaces().find(|ns| ns.name().is_none()).map(|ns| ns.uri())
&& !uri.is_empty()
&& uri != XFDF_NS
&& !declares(start_tag, "xmlns")
{
let _ = write!(declared, " xmlns=\"{}\"", esc(uri));
}
for ns in rich.namespaces() {
let Some(prefix) = ns.name() else { continue };
if prefix == "xml" || !markup.contains(&format!("{prefix}:")) || declares(start_tag, &format!("xmlns:{prefix}")) {
continue;
}
let _ = write!(declared, " xmlns:{prefix}=\"{}\"", esc(ns.uri()));
}
if declared.is_empty() {
return markup.to_string();
}
let (Some(head), Some(tail)) = (markup.get(..name_end), markup.get(name_end..)) else { return markup.to_string() };
let fixed = format!("{head}{declared}{tail}");
if parse_form_xml(&fixed).is_ok() { fixed } else { markup.to_string() }
}
/// XFDF's own namespace (ISO 19444-1).
const XFDF_NS: &str = "http://ns.adobe.com/xfdf/";
/// Put an annotation dictionary on a page, replacing one with the same `/NM`. Returns its
/// reference.
/// `/F` bits an imported file may not set on a redaction mark: Invisible, Hidden, NoView — so a
@@ -772,6 +1113,11 @@ pub(crate) fn place(doc: &mut Document, page_ref: ObjRef, mut d: Dict) -> Result
// without usable /QuadPoints removes its /Rect, which the outline is then drawn from.
d.remove(b"CA");
d.remove(b"ca");
// The outline is PdfCraft's own (an appearance the file carries could hide the mark), and
// a redaction has no border effect (ISO 32000-2 Table 195 lists none; §12.5.4 gives
// /BE to squares, circles, polygons and free text), which the outline could not draw.
d.remove(b"AP");
d.remove(b"BE");
let quads_drawable = d.get(b"QuadPoints").and_then(Object::as_array).is_some_and(|q| {
let v: Vec<f64> = q.iter().filter_map(Object::as_f64).filter(|v| v.is_finite()).collect();
let (quads, rest) = v.as_chunks::<8>();
@@ -791,7 +1137,32 @@ pub(crate) fn place(doc: &mut Document, page_ref: ObjRef, mut d: Dict) -> Result
}
}
}
// The outline is drawn inside /Rect: it is grown to hold every quad, so the whole area
// the mark removes shows (a /Rect with no area would show none of it).
let quads: Vec<f64> =
d.get(b"QuadPoints").and_then(Object::as_array).map(|q| q.iter().filter_map(Object::as_f64).collect()).unwrap_or_default();
if !quads.is_empty() {
let rect = rect_strict(doc, d.get(b"Rect"));
let start = rect.map(|[x0, y0, x1, y1]| [x0.min(x1), y0.min(y1), x0.max(x1), y0.max(y1)]);
let bounds = quads
.as_chunks::<2>()
.0
.iter()
.fold(start, |b, [x, y]| Some(b.map_or([*x, *y, *x, *y], |[l, bo, r, t]| [l.min(*x), bo.min(*y), r.max(*x), t.max(*y)])));
if let Some(b) = bounds
&& Some(b) != rect
{
d.set(b"Rect".to_vec(), arr(&b));
}
}
}
// An appearance the file carries is how the comment looked where it came from, and is kept
// (XFDF 3.0 "stamp": it takes precedence over the icon). A redaction mark is always drawn by
// PdfCraft, so what it marks for removal is shown.
let redact = d.name(b"Subtype") == Some(b"Redact".as_slice());
// A redaction mark with an area removes something when applied.
let marks = redact && d.contains(b"QuadPoints");
let keep_appearance = !redact && super::appearance::has_appearance(doc, &d);
let nm = d.get(b"NM").and_then(|n| n.as_string().map(|s| s.to_text()));
let annots: Vec<Object> = doc
.get(page_ref)
@@ -835,8 +1206,17 @@ pub(crate) fn place(doc: &mut Document, page_ref: ObjRef, mut d: Dict) -> Result
}
None => doc.add(Object::Dict(d)),
};
// Our own appearance where we can draw one; others keep none (viewers draw a default).
let _ = pdfcraft_annot::set_appearance(doc, r);
// Otherwise our own appearance where we can draw one; others keep none (viewers draw a default).
if !keep_appearance {
let drawn = pdfcraft_annot::set_appearance(doc, r);
// A redaction mark that removes something but shows nothing is refused: applying it
// would remove content the reviewer never saw marked.
if let Err(why) = drawn
&& marks
{
return Err(DataError::Malformed(format!("a redaction mark could not be drawn, so what it removes would not show ({why})")));
}
}
kept.push(Object::Ref(r));
doc.update_dict(page_ref, |p| p.set(b"Annots".to_vec(), Object::Array(kept)))?;
Ok(r)
@@ -872,19 +1252,35 @@ pub fn import_comments(
fresh: Option<&str>,
offset: Option<[f64; 2]>,
) -> Result<Report, DataError> {
import_xfdf_with(doc, text, page, fresh, offset.filter(|o| o.iter().all(|v| v.is_finite())))
// Into a copy, kept only when the whole import succeeds.
let mut work = doc.clone();
let report = import_xfdf_with(
&mut work,
text,
page,
fresh,
offset.filter(|o| o.iter().all(|v| v.is_finite())),
super::appearance::MAX_IMPORT_APPEARANCE_BYTES,
true,
)?;
*doc = work;
Ok(report)
}
pub(crate) fn import_xfdf(doc: &mut Document, text: &str) -> Result<Report, DataError> {
import_xfdf_with(doc, text, None, None, None)
import_xfdf_with(doc, text, None, None, None, super::appearance::MAX_IMPORT_APPEARANCE_BYTES, false)
}
fn import_xfdf_with(
/// The import, with `appearance_budget` decoded bytes of appearance for all of its comments;
/// `comments_only` leaves the file's field values out (comments placed as copies).
pub(crate) fn import_xfdf_with(
doc: &mut Document,
text: &str,
target_page: Option<usize>,
fresh: Option<&str>,
offset: Option<[f64; 2]>,
mut appearance_budget: usize,
comments_only: bool,
) -> Result<Report, DataError> {
check_form_xml_depth(text)?;
let xml = parse_form_xml(text).map_err(|e| DataError::Malformed(e.to_string()))?;
@@ -920,6 +1316,18 @@ fn import_xfdf_with(
let own_page = node.attribute("page").and_then(|p| p.parse::<usize>().ok());
let Some(page) = target_page.or(own_page).and_then(|p| pages.get(p).copied()) else { continue };
let Some(mut d) = annot_from_xml(node, subtype, page) else { continue };
// The appearance the file carries (XFDF 3.0 "appearance"), when it can be read.
// A redaction mark is always drawn by PdfCraft (see `place`): its appearance is not read.
if let Some(ap) = node.children().find(|c| c.has_tag_name("appearance")).filter(|_| *subtype != "Redact") {
let text: String = ap.children().filter(|c| c.is_text()).filter_map(|c| c.text()).collect();
match super::appearance::decode(&text, &mut appearance_budget) {
Ok(tree) => {
let ap = super::appearance::materialize(doc, tree);
d.set(b"AP".to_vec(), ap);
}
Err(why) => report.skipped.push(format!("{}: {why}", node.attribute("name").unwrap_or(&tag))),
}
}
let assigned = fresh.map(|prefix| format!("{prefix}-{}", n + 1));
if let Some(name) = &assigned {
// A copy under a name of its own (one even where the original had none).
@@ -964,7 +1372,7 @@ fn import_xfdf_with(
}
}
// Comments placed as copies: the file's field values are not part of the paste.
if fresh.is_some() || target_page.is_some() || offset.is_some() {
if comments_only {
return Ok(report);
}
// Fields.
docs/editing-existing-text.md
+8 -2
View File
@@ -160,8 +160,14 @@ scanned line kept apart. Applying an edit draws the new text visibly in the scan
the one chosen) and first covers the old words with the paper colour sampled around them. Each
word keeps its position and width, and a longer phrase grows rightwards rather than wrapping.
OCR sizes each word from its own box, so a phrase's words can come out several points apart: the
editor and the rewrite set the whole phrase in one size, the median of its words' sizes. The
cover is one flat colour, and the text uses the layer's Helvetica, not the scanned typeface.
editor and the rewrite set the whole phrase in one size, the median of its words' sizes.
A text edit covers and redraws only the words it changes: the rest of the line stays the scan, and
a deleted word leaves its gap, because the words after it are pixels. A replacement a little too
wide is squeezed across to fit its place (by up to a quarter); wider than that it takes over the
word it would run into, and new words with nothing to replace take a neighbour.
The whole phrase is redrawn for a change of formatting or a new line, and for a layer whose words
would move without the ones before them. While the phrase is open, the editor still shows all of
it in the layer's Helvetica. The cover is one flat colour, and the text uses the layer's Helvetica, not the scanned typeface.
This applies to any invisible text (rendering mode 3), not just a PdfCraft OCR layer: editing
invisible text, such as another OCR tool's layer, makes the rewritten paragraph visible over a
painted cover in the sampled paper colour. Edit text also groups invisible text differently: its lines never
docs/roadmap.md
+1
View File
@@ -325,6 +325,7 @@ Estimates unchanged.
| 2026-11-03 | patch | M5 #5850: Enter adds Fill & Sign line breaks; outside clicks commit, Escape cancels, and saved multiline appearances retain their text and height; estimates unchanged. |
| 2026-11-02 | patch | M10 #5743: macOS print jobs carry the job name and owner some vendors' filters need; not yet checked on a physical printer |
| 2026-11-02 | patch | M3 #5708: pages shorter than the window sit in its middle, on a whole physical pixel |
| 2026-11-02 | patch | M3 #5716: a snapshot keeps its resolution when zoomed out; a choice of resolution remains |
| 2026-11-02 | patch | M14 #5737, #5460: Preferences turns off the update check at start, on by default |
| 2026-11-01 | patch | M7 #5657: header/footer, watermark and background operations commit all selected pages together; later-page errors preserve marks, content, shared/inherited resources and pending edits; estimates unchanged. |
| 2026-10-31 | patch | #1577: reconcile optional page shadows with current viewer/organizer and settings modules; CI validation pending. |
parity/acrobat-features.toml
+26 -26
View File
@@ -818,8 +818,8 @@ milestone = "M2"
status = "shipped"
commands = []
tools = []
evidence = ["crates/render/src/raster/tests.rs::widget_appearance_states", "crates/render/src/raster/tests.rs::highlights_without_an_appearance_are_drawn", "crates/engine/src/tests.rs::comments_without_appearances_are_drawn_but_not_saved", "crates/engine/src/tests.rs::cli_display_adds_missing_appearances_without_changing_the_source", "crates/engine/src/tests.rs::fields_left_to_the_viewer_are_displayed_from_their_values", "crates/forms/src/tests.rs::need_appearances_draws_every_text_and_choice_field_in_the_display_copy", "crates/forms/src/tests.rs::the_field_appearance_cache_sees_every_input", "crates/engine/src/tests.rs::buttons_and_combs_left_to_the_viewer_are_displayed", "crates/forms/src/tests.rs::check_boxes_without_an_appearance_show_their_caption_symbol", "crates/forms/src/tests.rs::push_buttons_without_an_appearance_show_their_caption_and_frame", "crates/forms/src/tests.rs::need_appearances_draws_check_boxes_and_radio_buttons_from_their_captions", "crates/engine/src/tests.rs::need_appearances_buttons_are_displayed_from_their_captions", "crates/render/src/raster/tests.rs::invisible_unknown_annotations_and_popups_paint_nothing", "crates/print/src/tests.rs::invisible_annotations_of_unknown_types_do_not_print", "crates/engine/src/tests.rs::stamps_without_appearances_draw_the_stamp_they_name", "crates/automation/tests/it/automation.rs::page_render_draws_stamps_without_an_appearance", "crates/annot/src/tests.rs::a_four_entry_colour_with_a_null_is_cmyk_with_zero", "crates/forms/src/tests.rs::a_four_entry_mk_colour_with_a_null_is_cmyk_with_zero"]
notes = "comments without /AP (FreeText, Ink, notes, stamps, shapes, markup) are drawn from their dictionaries in the page view, page_render and page images, including the CLI render command, never saved; a four-entry colour with a non-number in it (/C, /IC, /MK /BC and /BG written [1 0 0 null]) is read as CMYK with that entry 0; a stamp from another program draws the stamp its /Name names (the fourteen standard names of ISO 32000-2 Table 184, Draft when there is none) or, for any other name, a grey frame with both diagonals; text and choice fields are drawn from their values when the form sets /NeedAppearances (every one) or a widget has no /N (#3470), also display-only and cached per widget (a field edit on a 1000-field /NeedAppearances form takes ~13 ms against ~7 ms on a form that needs none) (a read-only document still gets the renderer's Highlight fallback only); check boxes and radio buttons with no appearance for their /AS are drawn display-only from /MK (background, border, and when on the /MK /CA ZapfDingbats symbol in the /DA colour, a check or a dot by default), and when the form sets /NeedAppearances so is every one that has an /MK, over the appearance the file carries (ISO 32000-2 Table 224; one without /MK keeps its own), and push buttons with none from /MK and /BS with their caption; Link borders without /AP and printing are not covered yet; an annotation of a non-standard type with the Invisible flag (ISO 32000-2 Table 167) is neither drawn nor printed, and a popup's own /AP is never painted on the page (its parent's text shows in the popup window, §12.5.6.14)"
evidence = ["crates/render/src/raster/tests.rs::widget_appearance_states", "crates/render/src/raster/tests.rs::highlights_without_an_appearance_are_drawn", "crates/engine/src/tests.rs::comments_without_appearances_are_drawn_but_not_saved", "crates/engine/src/tests.rs::cli_display_adds_missing_appearances_without_changing_the_source", "crates/engine/src/tests.rs::fields_left_to_the_viewer_are_displayed_from_their_values", "crates/forms/src/tests.rs::need_appearances_draws_every_text_and_choice_field_in_the_display_copy", "crates/forms/src/tests.rs::the_field_appearance_cache_sees_every_input", "crates/engine/src/tests.rs::buttons_and_combs_left_to_the_viewer_are_displayed", "crates/forms/src/tests.rs::check_boxes_without_an_appearance_show_their_caption_symbol", "crates/forms/src/tests.rs::push_buttons_without_an_appearance_show_their_caption_and_frame", "crates/forms/src/tests.rs::need_appearances_draws_check_boxes_and_radio_buttons_from_their_captions", "crates/engine/src/tests.rs::need_appearances_buttons_are_displayed_from_their_captions", "crates/render/src/raster/tests.rs::invisible_unknown_annotations_and_popups_paint_nothing", "crates/print/src/tests.rs::invisible_annotations_of_unknown_types_do_not_print", "crates/engine/src/tests.rs::stamps_without_appearances_draw_the_stamp_they_name", "crates/automation/tests/it/automation.rs::page_render_draws_stamps_without_an_appearance", "crates/annot/src/tests.rs::a_four_entry_colour_with_a_null_is_cmyk_with_zero", "crates/forms/src/tests.rs::a_four_entry_mk_colour_with_a_null_is_cmyk_with_zero", "crates/annot/src/tests.rs::text_markup_whose_quads_reach_past_its_rect_is_drawn_over_them", "crates/annot/src/tests.rs::text_markup_reach_stops_at_the_page_and_the_quad_cap", "crates/annot/src/tests.rs::text_markup_reach_reads_quads_like_the_drawing", "crates/annot/src/tests.rs::ink_whose_strokes_reach_past_its_rect_is_drawn_through_them", "crates/annot/src/tests.rs::polyline_whose_vertices_reach_past_its_rect_is_drawn_through_them", "crates/annot/src/tests.rs::polygon_whose_vertices_reach_past_its_rect_is_drawn_through_them", "crates/annot/src/tests.rs::line_whose_points_reach_past_its_rect_is_drawn_through_them", "crates/annot/src/tests.rs::path_reach_stops_at_the_point_cap_and_is_saved_on_restyle", "crates/annot/src/tests.rs::indirect_border_style_and_colour_are_drawn", "crates/annot/src/tests.rs::callout_without_rd_is_drawn", "crates/forms/src/tests.rs::normal_appearances_that_give_no_stream_are_drawn"]
notes = "comments without /AP (FreeText, Ink, notes, stamps, shapes, markup; text markup, lines, polylines, polygons and ink through their /QuadPoints, /L, /Vertices or /InkList where these reach past /Rect, as far as the crop box) are drawn from their dictionaries in the page view, page_render and page images, including the CLI render command, never saved, with style values given by reference (a /BS, colour, /RD or /CL as an indirect object, ISO 32000-2 §7.3.10) followed and a callout without /RD (optional, Table 177) boxed at its /Rect; a four-entry colour with a non-number in it (/C, /IC, /MK /BC and /BG written [1 0 0 null]) is read as CMYK with that entry 0; a stamp from another program draws the stamp its /Name names (the fourteen standard names of ISO 32000-2 Table 184, Draft when there is none) or, for any other name, a grey frame with both diagonals; text and choice fields are drawn from their values when the form sets /NeedAppearances (every one) or a widget's /N gives no stream to draw (none, null, a dictionary without its stream data, or a state subdictionary without a stream for /AS; #3470), also display-only and cached per widget (a field edit on a 1000-field /NeedAppearances form takes ~13 ms against ~7 ms on a form that needs none) (a read-only document still gets the renderer's Highlight fallback only); check boxes and radio buttons with no appearance for their /AS are drawn display-only from /MK (background, border, and when on the /MK /CA ZapfDingbats symbol in the /DA colour, a check or a dot by default), and when the form sets /NeedAppearances so is every one that has an /MK, over the appearance the file carries (ISO 32000-2 Table 224; one without /MK keeps its own), and push buttons with none from /MK and /BS with their caption; Link borders without /AP and printing are not covered yet; an annotation of a non-standard type with the Invisible flag (ISO 32000-2 Table 167) is neither drawn nor printed, and a popup's own /AP is never painted on the page (its parent's text shows in the popup window, §12.5.6.14)"
[[feature]]
id = "view.render-error-isolation"
@@ -1406,8 +1406,8 @@ milestone = "M3"
status = "shipped"
commands = ["edit.find"]
tools = ["text_find"]
evidence = ["crates/ui-egui/tests/it/ui.rs::find_counts_matches_across_pages", "crates/ui-egui/tests/it/ui.rs::find_reports_no_matches", "crates/automation/tests/it/automation.rs::open_inspect_render_and_find", "crates/render/src/text/tests.rs::identity_v_columns_are_searchable_in_reading_order", "crates/render/src/text/tests.rs::toc_leaders_do_not_hide_word_gaps", "crates/render/src/text/tests.rs::short_tracked_ellipses_remain_searchable", "crates/render/tests/it/search.rs::find_skips_soft_hyphens_and_zero_width_characters", "crates/render/tests/it/search.rs::find_joins_a_word_divided_at_a_line_end_soft_hyphen", "crates/render/tests/it/search.rs::a_zero_width_space_is_a_word_boundary_for_whole_words", "crates/render/tests/it/search.rs::a_chinese_or_japanese_word_divided_by_a_line_end_is_found_on_both_lines", "crates/render/tests/it/search.rs::a_line_break_stays_a_space_beside_latin_or_hangul_and_between_side_by_side_text", "crates/render/tests/it/search.rs::a_word_divided_between_vertical_columns_is_found_in_both", "crates/render/tests/it/search.rs::each_unspaced_range_joins_and_spaced_scripts_do_not", "crates/render/tests/it/search.rs::a_soft_hyphen_and_a_chinese_or_japanese_line_end_combine"]
notes = "Horizontal and WMode 1 vertical text are grouped independently, including a horizontal heading above right-to-left vertical columns. Dotted TOC leaders preserve word spaces for phrase search (#3086), while short tracked ellipses remain intact and searchable; sparse lines and condensed OCR gaps below the existing height-based threshold remain limitations. Complex interleaved flows still use a geometric reading-order heuristic. Soft hyphens, zero width spaces and joiners, word joiners and U+FEFF inside a word are skipped, in the page and in the query; a skipped zero width space still ends a word for Whole words only. A line break reads as no word break when the line ends in a soft hyphen, or when it runs on between two Chinese or Japanese characters (East Asian Wide, Fullwidth or Halfwidth, not Hangul and not fullwidth Latin letters or digits), so a divided word is found, in horizontal lines and in vertical columns that continue to the left; the break stays a space beside Latin or Korean text and between text set side by side on one line or in staggered cells."
evidence = ["crates/ui-egui/tests/it/ui.rs::find_counts_matches_across_pages", "crates/ui-egui/tests/it/ui.rs::find_reports_no_matches", "crates/automation/tests/it/automation.rs::open_inspect_render_and_find", "crates/render/src/text/tests.rs::identity_v_columns_are_searchable_in_reading_order", "crates/render/src/text/tests.rs::toc_leaders_do_not_hide_word_gaps", "crates/render/src/text/tests.rs::short_tracked_ellipses_remain_searchable", "crates/render/tests/it/search.rs::find_skips_soft_hyphens_and_zero_width_characters", "crates/render/tests/it/search.rs::find_joins_a_word_divided_at_a_line_end_soft_hyphen", "crates/render/tests/it/search.rs::a_zero_width_space_is_a_word_boundary_for_whole_words", "crates/render/tests/it/search.rs::a_chinese_or_japanese_word_divided_by_a_line_end_is_found_on_both_lines", "crates/render/tests/it/search.rs::a_line_break_stays_a_space_beside_latin_or_hangul_and_between_side_by_side_text", "crates/render/tests/it/search.rs::a_word_divided_between_vertical_columns_is_found_in_both", "crates/render/tests/it/search.rs::each_unspaced_range_joins_and_spaced_scripts_do_not", "crates/render/tests/it/search.rs::a_soft_hyphen_and_a_chinese_or_japanese_line_end_combine", "crates/render/src/text/tests.rs::thai_marks_drawn_one_glyph_at_a_time_keep_their_place_and_words"]
notes = "Horizontal and WMode 1 vertical text are grouped independently, including a horizontal heading above right-to-left vertical columns. Dotted TOC leaders preserve word spaces for phrase search (#3086), while short tracked ellipses remain intact and searchable; sparse lines and condensed OCR gaps below the existing height-based threshold remain limitations. Combining marks (Mn/Me) of left-to-right scripts drawn right after their letter stay with it in content order, and Thai or Indic word gaps without a space glyph are found as spaces (#6517). Complex interleaved flows still use a geometric reading-order heuristic. Soft hyphens, zero width spaces and joiners, word joiners and U+FEFF inside a word are skipped, in the page and in the query; a skipped zero width space still ends a word for Whole words only. A line break reads as no word break when the line ends in a soft hyphen, or when it runs on between two Chinese or Japanese characters (East Asian Wide, Fullwidth or Halfwidth, not Hangul and not fullwidth Latin letters or digits), so a divided word is found, in horizontal lines and in vertical columns that continue to the left; the break stays a space beside Latin or Korean text and between text set side by side on one line or in staggered cells."
[[feature]]
id = "view.find-next-previous"
@@ -1550,8 +1550,8 @@ milestone = "M3"
status = "shipped"
commands = ["edit.copy"]
tools = ["text_extract"]
evidence = ["crates/ui-egui/tests/it/ui.rs::drag_selects_text_and_copy_returns_it", "crates/ui-egui/tests/it/text_select.rs::selected_text_copy_edit_and_context_menus", "crates/ui-egui/tests/it/text_select.rs::selected_text_copy_floating_button_honours_permissions", "crates/ui-egui/tests/it/text_select.rs::selected_text_copy_keeps_focused_input_clipboard_ownership"]
notes = "Keyboard, Edit-menu, page-context-menu and floating-selection Copy honour document copy permissions. Menus require selected page text; focused text inputs and Organize page copying retain their own clipboard handling. Headless text extraction uses text_extract."
evidence = ["crates/ui-egui/tests/it/ui.rs::drag_selects_text_and_copy_returns_it", "crates/ui-egui/tests/it/text_select.rs::selected_text_copy_edit_and_context_menus", "crates/ui-egui/tests/it/text_select.rs::selected_text_copy_floating_button_honours_permissions", "crates/ui-egui/tests/it/text_select.rs::selected_text_copy_keeps_focused_input_clipboard_ownership", "crates/render/src/text/tests.rs::a_space_taken_back_under_the_next_letter_stays_before_it", "crates/render/src/text/tests.rs::issue_11656_layout_reads_whole_words", "crates/render/src/text/tests.rs::combining_marks_of_other_scripts_keep_their_place"]
notes = "Keyboard, Edit-menu, page-context-menu and floating-selection Copy honour document copy permissions. Menus require selected page text; focused text inputs and Organize page copying retain their own clipboard handling. Headless text extraction uses text_extract. Copied Thai, Devanagari and Latin text keeps each combining mark after its letter when every glyph is drawn on its own (#6517). A space that starts a right-to-left show string and is taken back under the next letter (ISO 32000-2 §14.8.2.5.3) stays a word break after its word instead of splitting it, also when a mark is drawn between them; punctuation taken back that way moves only inside right-to-left text."
[[feature]]
id = "view.select-all"
@@ -1586,8 +1586,8 @@ milestone = "M3"
status = "shipped"
commands = ["edit.snapshot"]
tools = ["page_render"]
evidence = ["crates/ui-egui/tests/it/viewer.rs::marquee_zoom_and_snapshot"]
notes = "copies the area at the current zoom to the clipboard as an image"
evidence = ["crates/ui-egui/tests/it/viewer.rs::marquee_zoom_and_snapshot", "crates/ui-egui/tests/it/viewer.rs::a_snapshot_keeps_its_resolution_when_zoomed_out", "crates/ui-egui/src/canvas/zoom_snap.rs::a_snapshot_is_drawn_at_the_finer_of_the_zoom_and_the_minimum", "crates/ui-egui/src/canvas/zoom_snap.rs::a_snapshot_stays_within_what_the_renderer_draws_at_once", "crates/ui-egui/tests/it/viewer.rs::a_snapshot_of_a_large_page_reaches_its_far_corner"]
notes = "copies the area to the clipboard as an image, at 300 ppi or the current zoom if that is finer (#5716), or as fine as the renderer draws in one tile (4096 pixels a side), drawn while the app waits. Missing: a choice of resolution, and a selection that stays active to copy later"
[[feature]]
id = "view.copy-image"
@@ -3538,8 +3538,8 @@ milestone = "M3"
status = "shipped"
commands = ["file.properties"]
tools = ["doc_info"]
evidence = ["crates/ui-egui/tests/it/editing.rs::document_properties_edit_is_one_undoable_step", "crates/ui-egui/tests/it/editing.rs::cancelling_properties_discards_the_draft", "crates/organize/src/tests.rs::document_info_falls_back_to_the_xmp_packet", "crates/engine/src/tests.rs::document_information_stated_only_in_xmp_is_shown", "crates/ui-egui/tests/it/dialog_chrome.rs::document_properties_values_line_up_and_long_fields_wrap", "crates/ui-egui/tests/it/dialog_chrome.rs::space_audit_tab_measures_the_file_and_stays_in_the_card", "crates/render/src/inspect/tests.rs::creation_and_modification_dates_are_read_for_document_properties"]
notes = "Description, Initial View, Security, Fonts, Custom (organize.custom-properties) and Advanced, plus a Space Audit tab Acrobat does not have (its audit is only in the PDF Optimizer); the audit runs when the tab opens and again after an edit. Title, Author, Subject and Keywords (and Creator, Producer in doc_info) come from /Info, else from the XMP packet (dc:title, dc:creator, dc:description, pdf:Keywords, xmp:CreatorTool, pdf:Producer) for files that state them only there; edits write both. Description and doc_info show the /Info CreationDate and ModDate as written (YYYY-MM-DD HH:MM, not converted to local time; #5534). Missing: the dates from XMP (xmp:CreateDate, xmp:ModifyDate) when /Info has none, editing them, a UTF-16 XMP packet isn't read, and /Info kept only in an older revision's trailer isn't looked up"
evidence = ["crates/ui-egui/tests/it/editing.rs::document_properties_edit_is_one_undoable_step", "crates/ui-egui/tests/it/editing.rs::cancelling_properties_discards_the_draft", "crates/organize/src/tests.rs::document_info_falls_back_to_the_xmp_packet", "crates/engine/src/tests.rs::document_information_stated_only_in_xmp_is_shown", "crates/ui-egui/tests/it/dialog_chrome.rs::document_properties_values_line_up_and_long_fields_wrap", "crates/ui-egui/tests/it/dialog_chrome.rs::space_audit_tab_measures_the_file_and_stays_in_the_card"]
notes = "Description, Initial View, Security, Fonts, Custom (organize.custom-properties) and Advanced, plus a Space Audit tab Acrobat does not have (its audit is only in the PDF Optimizer); the audit runs when the tab opens and again after an edit. Title, Author, Subject and Keywords (and Creator, Producer in doc_info) come from /Info, else from the XMP packet (dc:title, dc:creator, dc:description, pdf:Keywords, xmp:CreatorTool, pdf:Producer) for files that state them only there; edits write both. Missing: a UTF-16 XMP packet isn't read, and /Info kept only in an older revision's trailer isn't looked up"
[[feature]]
id = "organize.edit-metadata"
@@ -4334,8 +4334,8 @@ milestone = "M5"
status = "shipped"
commands = ["comment.import"]
tools = ["doc_import_data"]
evidence = ["crates/xfdf/src/tests.rs::xfdf_round_trips_comments_and_values", "crates/xfdf/src/tests.rs::fdf_round_trips_values_and_comments", "crates/automation/tests/it/automation.rs::comments_and_form_data_travel_as_xfdf_fdf_and_text"]
notes = "replies, pop-ups, ink, quads, line ends; same /NM replaces"
evidence = ["crates/xfdf/src/tests.rs::xfdf_round_trips_comments_and_values", "crates/xfdf/src/tests.rs::fdf_round_trips_values_and_comments", "crates/automation/tests/it/automation.rs::comments_and_form_data_travel_as_xfdf_fdf_and_text", "crates/xfdf/src/tests_format.rs::formatting_written_by_another_tool_is_imported", "crates/xfdf/src/tests_format.rs::an_appearance_written_by_another_tool_is_imported", "crates/xfdf/src/tests_format.rs::unreadable_appearances_are_skipped_recorded_and_drawn_again", "crates/xfdf/src/tests_format.rs::fdf_carries_appearances_with_their_resources", "crates/xfdf/src/tests_format.rs::the_intent_is_written_and_read_as_intent_or_it", "crates/xfdf/src/tests_format.rs::a_chain_of_forms_comes_back_whole_from_fdf_or_is_refused", "crates/xfdf/src/tests_format.rs::an_imported_redaction_mark_is_drawn_even_when_cloudy_or_undrawable", "crates/xfdf/src/tests_format.rs::the_import_budget_is_shared_by_every_comment", "crates/engine/src/tests.rs::a_comment_imported_without_its_appearance_is_said_in_the_repair_log"]
notes = "replies, pop-ups, ink, quads, line ends; same /NM replaces. XFDF formatting comes in as XFDF 3.0 names it: contents-richtext (/RC; plain text in it, or markup without <contents>, also gives /Contents), defaultstyle, width/style/dashes (a dashed border stays dashed), cloudy style and intensity, rotation, justification, fringe, leaderLength/leaderExtend/leader-offset, caption attributes, caret symbol, redaction overlay text, the intent (`intent` or `IT`). A stamp's XFDF <appearance> (base 64 of the DICT/STREAM/DATA appearance XML other tools write) is kept; an unreadable or oversized one (16 MiB each, 64 MiB per import, 64 levels, 100,000 values) (or one whose normal appearance is several states, which XFDF cannot select) is left out, said in the import report and the repair log, and the comment is drawn by PdfCraft. FDF /AP streams are kept with their resources; an FDF whose comments expand past the import budget (1,000,000 values or 64 MiB of data) or nest more than 256 levels is refused with an error and nothing is imported (an import is all or nothing); a page the file's appearance reaches is not copied in. A redaction mark is always drawn by PdfCraft, and one with an area it cannot draw is refused. Rich text is kept as data; when PdfCraft draws a text box itself (no appearance in the file), it draws the plain text with /DA, not the rich text's fonts, and does not turn it by /Rotate."
[[feature]]
id = "comment.export-fdf-xfdf"
@@ -4346,8 +4346,8 @@ milestone = "M5"
status = "shipped"
commands = ["comment.export"]
tools = ["doc_export_data"]
evidence = ["crates/xfdf/src/tests.rs::fdf_export_nulls_nested_page_links_without_copying_the_source_page_tree", "crates/xfdf/src/tests.rs::fdf_export_bounds_indirect_type_alias_lookups", "crates/xfdf/src/tests.rs::fdf_export_preserves_metadata_with_stale_type_references", "crates/xfdf/src/tests.rs::fdf_export_keeps_indirect_values_nested_in_annotation_properties", "crates/xfdf/src/tests.rs::fdf_export_preserves_shared_cycles_and_nested_streams_without_expansion", "crates/xfdf/src/tests.rs::fdf_export_refuses_excessive_direct_depth_values_and_payload", "crates/xfdf/src/tests.rs::fdf_export_refuses_aliases_before_cloning_large_annotation_values", "crates/xfdf/src/tests.rs::fdf_export_refuses_repeated_saved_annotations_after_cache_eviction", "crates/automation/tests/it/automation.rs::fdf_nested_comment_values_survive_export_import_and_save_through_tools", "crates/automation/tests/it/automation.rs::refused_fdf_export_keeps_the_destination_and_source_through_tools", "crates/xfdf/src/tests.rs::xfdf_round_trips_comments_and_values", "crates/xfdf/src/tests.rs::fdf_round_trips_values_and_comments", "crates/automation/tests/it/automation.rs::comments_and_form_data_travel_as_xfdf_fdf_and_text", "crates/xfdf/src/tests.rs::xfdf_exports_preserve_whitespace_in_comment_attributes"]
notes = "FDF preserves shared/cyclic nested values. Discovery keeps lazy array owners and annotation positions; retained entries are budgeted before cloning, including after saved-PDF cache eviction. Copied-graph limits are 100,000 comments/objects/values, 64 direct levels and a conservative 64 MiB data budget; refusal precedes writing. Source parsing and page discovery retain existing COS/parser limits, with no total source-memory bound. Source /Page and /Pages dictionaries become null at every copy depth; indirect Type names check each reference generation with iterative cycle detection and a 64-reference limit; stale Type references preserve unrelated metadata; a saved/reopened three-page fixture verifies small output and retained unrelated metadata. Appearances and source-document links are not carried; FDF import semantics are unchanged. XFDF exports retain tabs and line breaks in comment identities and metadata."
evidence = ["crates/xfdf/src/tests.rs::fdf_export_nulls_nested_page_links_without_copying_the_source_page_tree", "crates/xfdf/src/tests.rs::fdf_export_bounds_indirect_type_alias_lookups", "crates/xfdf/src/tests.rs::fdf_export_preserves_metadata_with_stale_type_references", "crates/xfdf/src/tests.rs::fdf_export_keeps_indirect_values_nested_in_annotation_properties", "crates/xfdf/src/tests.rs::fdf_export_preserves_shared_cycles_and_nested_streams_without_expansion", "crates/xfdf/src/tests.rs::fdf_export_refuses_excessive_direct_depth_values_and_payload", "crates/xfdf/src/tests.rs::fdf_export_refuses_aliases_before_cloning_large_annotation_values", "crates/xfdf/src/tests.rs::fdf_export_refuses_repeated_saved_annotations_after_cache_eviction", "crates/automation/tests/it/automation.rs::fdf_nested_comment_values_survive_export_import_and_save_through_tools", "crates/automation/tests/it/automation.rs::refused_fdf_export_keeps_the_destination_and_source_through_tools", "crates/xfdf/src/tests.rs::xfdf_round_trips_comments_and_values", "crates/xfdf/src/tests.rs::fdf_round_trips_values_and_comments", "crates/automation/tests/it/automation.rs::comments_and_form_data_travel_as_xfdf_fdf_and_text", "crates/xfdf/src/tests.rs::xfdf_exports_preserve_whitespace_in_comment_attributes", "crates/xfdf/src/tests_format.rs::free_text_formatting_is_exported_as_xfdf_elements_and_attributes", "crates/xfdf/src/tests_format.rs::line_leaders_and_caption_round_trip", "crates/xfdf/src/tests_format.rs::square_circle_caret_and_redaction_attributes_round_trip", "crates/xfdf/src/tests_format.rs::an_image_stamp_keeps_its_appearance_through_xfdf", "crates/xfdf/src/tests_format.rs::formatting_round_trips", "crates/xfdf/src/tests_format.rs::fdf_carries_only_stamp_appearances"]
notes = "FDF preserves shared/cyclic nested values. Discovery keeps lazy array owners and annotation positions; retained entries are budgeted before cloning, including after saved-PDF cache eviction. Copied-graph limits are 100,000 comments/objects/values, 64 direct levels and a conservative 64 MiB data budget; refusal precedes writing. Source parsing and page discovery retain existing COS/parser limits, with no total source-memory bound. Source /Page and /Pages dictionaries become null at every copy depth; indirect Type names check each reference generation with iterative cycle detection and a 64-reference limit; stale Type references preserve unrelated metadata; a saved/reopened three-page fixture verifies small output and retained unrelated metadata. FDF carries a stamp's appearance (/AP with the forms and images it draws), as XFDF does, and other comments their formatting entries without /AP; source-document links stay behind; FDF import copies a comment's streams only when the comment lands on a page. XFDF exports retain tabs and line breaks in comment identities and metadata, and write each comment's formatting as XFDF 3.0 elements and attributes (contents-richtext, defaultstyle, width/style/dashes also from an older /Border, cloudy style, rotation, justification, fringe on every type, line leaders and captions, caret symbol, redaction overlay text, the intent as both `intent` and `IT`) and a stamp's appearance as <appearance>; an appearance past the limits, or on a rect with no area, is left out. Readers that don't know an element skip it."
[[feature]]
id = "comment.export-selected"
@@ -4358,8 +4358,8 @@ milestone = "M5"
status = "partial"
commands = ["comment.copy"]
tools = ["comments_copy"]
evidence = ["crates/xfdf/src/tests.rs::exporting_selected_comments_takes_them_and_their_replies", "crates/automation/tests/it/automation.rs::comments_copy_and_paste_between_documents_through_tools", "crates/xfdf/src/tests.rs::copies_keep_their_intent_cloud_and_pdfcraft_keys"]
notes = "The selected comments, each with its replies, as XFDF: Copy puts it on the clipboard (desktop) and comments_copy returns it headlessly. Each keeps what it is meant as (`intent`: typewriter text, a cloud, an arrow), a cloudy border (`style`, `intensity`; both also in File ▸ Export comments) and PdfCraft's own keys (`pdfcraft:keys`, in a namespace of its own: a Fill & Sign item, a turned comment's box and angle), so a paste is the same comment. Not yet: a Save as… to a data file from the interface."
evidence = ["crates/xfdf/src/tests.rs::exporting_selected_comments_takes_them_and_their_replies", "crates/automation/tests/it/automation.rs::comments_copy_and_paste_between_documents_through_tools", "crates/xfdf/src/tests.rs::copies_keep_their_intent_cloud_and_pdfcraft_keys", "crates/xfdf/src/tests_format.rs::file_export_carries_only_stamp_appearances_and_copies_carry_every_one"]
notes = "The selected comments, each with its replies, as XFDF: Copy puts it on the clipboard (desktop) and comments_copy returns it headlessly. Each keeps what it is meant as (`intent`: typewriter text, a cloud, an arrow), a cloudy border (`style`, `intensity`; both also in File ▸ Export comments) and PdfCraft's own keys (`pdfcraft:keys`, in a namespace of its own: a Fill & Sign item, a turned comment's box and angle), its formatting (rich text, default style, border style and dashes, rotation, justification, fringe, line leaders) and its appearance (`<appearance>`, for every comment type, not only stamps), so a paste is the same comment. Not yet: a Save as… to a data file from the interface."
[[feature]]
id = "comment.multi-select"
@@ -4382,8 +4382,8 @@ milestone = "M5"
status = "shipped"
commands = ["comment.copy", "comment.paste"]
tools = ["comments_copy", "comments_paste"]
evidence = ["crates/ui-egui/tests/it/comments.rs::copied_comments_paste_into_another_document", "crates/ui-egui/tests/it/comments.rs::copied_comments_take_the_click_from_the_tool_that_is_on", "crates/engine/src/tests.rs::pasting_copied_comments_adds_copies_that_undo_as_one_step", "crates/xfdf/src/tests.rs::importing_comments_as_copies_adds_them_on_the_page_with_fresh_names", "crates/xfdf/src/tests.rs::copied_marks_and_text_boxes_show_what_they_are_while_placed_and_once_pasted", "crates/automation/tests/it/automation.rs::comments_copy_and_paste_between_documents_through_tools"]
notes = "⌘C with comments selected (and no text selected) copies them as XFDF, each with its replies, to the app's clipboard (every tab and window) and as text to the system clipboard. ⌘V or Edit ▸ Paste comments hangs the copies on the pointer as one bundle (their box, with a count, and each comment's outline in its colour: Fill & Sign ticks, crosses, dots and lines as themselves, text boxes with their text); a click places them, centred there, on the page clicked (whatever tool is on: Fill & Sign's Text tool opens no box, a stamp tool places no stamp), so a copy never lands on its original and a bundle from an A0 page goes where it fits on an A4 one (placed as it is when larger than the page, with a notice); Escape or a right-click lets the bundle go. The page's context menu places the bundle at the point the menu was opened at. The copies are new comments under names of their own (nothing is replaced; pasting twice gives two sets) and are selected afterwards. XFDF with comments on the system clipboard pastes too, so comments travel between PdfCraft processes. The Comments panel's card menu copies that card's comment. Headless: comments_copy / comments_paste (`at` places the bundle's centre), undoable as one step. Not yet: appearance streams (the copies are drawn again), file attachments' data (the attachment comment is copied without its file), pasting in the web build's other tabs through the system clipboard."
evidence = ["crates/ui-egui/tests/it/comments.rs::copied_comments_paste_into_another_document", "crates/ui-egui/tests/it/comments.rs::copied_comments_take_the_click_from_the_tool_that_is_on", "crates/engine/src/tests.rs::pasting_copied_comments_adds_copies_that_undo_as_one_step", "crates/xfdf/src/tests.rs::importing_comments_as_copies_adds_them_on_the_page_with_fresh_names", "crates/xfdf/src/tests.rs::copied_marks_and_text_boxes_show_what_they_are_while_placed_and_once_pasted", "crates/automation/tests/it/automation.rs::comments_copy_and_paste_between_documents_through_tools", "crates/xfdf/src/tests_format.rs::file_export_carries_only_stamp_appearances_and_copies_carry_every_one"]
notes = "⌘C with comments selected (and no text selected) copies them as XFDF, each with its replies, to the app's clipboard (every tab and window) and as text to the system clipboard. ⌘V or Edit ▸ Paste comments hangs the copies on the pointer as one bundle (their box, with a count, and each comment's outline in its colour: Fill & Sign ticks, crosses, dots and lines as themselves, text boxes with their text); a click places them, centred there, on the page clicked (whatever tool is on: Fill & Sign's Text tool opens no box, a stamp tool places no stamp), so a copy never lands on its original and a bundle from an A0 page goes where it fits on an A4 one (placed as it is when larger than the page, with a notice); Escape or a right-click lets the bundle go. The page's context menu places the bundle at the point the menu was opened at. The copies are new comments under names of their own (nothing is replaced; pasting twice gives two sets) and are selected afterwards. XFDF with comments on the system clipboard pastes too, so comments travel between PdfCraft processes. The Comments panel's card menu copies that card's comment. Headless: comments_copy / comments_paste (`at` places the bundle's centre), undoable as one step. Each copy keeps its formatting and its appearance stream, so it looks as the original did. Not yet: file attachments' data (the attachment comment is copied without its file), pasting in the web build's other tabs through the system clipboard."
[[feature]]
id = "comment.import-from-pdf"
@@ -4590,8 +4590,8 @@ milestone = "M3"
status = "shipped"
commands = ["form.fields"]
tools = ["doc_info"]
evidence = ["crates/ui-egui/tests/it/view.rs::field_list_jumps_to_a_fields_page", "crates/render/src/inspect/tests.rs::fields_listed_only_on_the_pages_are_listed", "crates/render/src/inspect/tests.rs::a_widget_whose_parent_is_missing_is_listed", "crates/forms/src/tests.rs::a_widget_whose_parent_is_missing_is_a_field_of_its_own", "crates/ui-egui/tests/it/bulk_fields.rs::fields_panel_selects_multiple_fields_and_opens_their_shared_properties", "crates/ui-egui/tests/it/bulk_fields.rs::double_clicking_a_field_in_the_panel_opens_its_individual_properties"]
notes = "list with jump-to-field; in Prepare a form, Shift/Ctrl/Command-click selects several fields and Properties edits their shared properties; double-click opens field properties; fields a file lists only as page widgets (empty /Fields) are adopted too, and a widget whose /Parent names a missing object is a field of its own"
evidence = ["crates/ui-egui/tests/it/view.rs::field_list_jumps_to_a_fields_page", "crates/render/src/inspect/tests.rs::fields_listed_only_on_the_pages_are_listed", "crates/render/src/inspect/tests.rs::a_widget_whose_parent_is_missing_is_listed", "crates/forms/src/tests.rs::a_widget_whose_parent_is_missing_is_a_field_of_its_own", "crates/ui-egui/tests/it/bulk_fields.rs::fields_panel_selects_multiple_fields_and_opens_their_shared_properties", "crates/ui-egui/tests/it/bulk_fields.rs::double_clicking_a_field_in_the_panel_opens_its_individual_properties", "crates/render/src/inspect/tests.rs::a_named_kid_without_a_parent_keeps_its_own_name", "crates/forms/src/tests.rs::a_named_kid_without_a_parent_keeps_its_own_name", "crates/forms/src/tests.rs::a_radio_option_of_any_field_sharing_the_name_can_be_chosen", "crates/forms/src/tests.rs::a_radio_choice_by_the_name_turns_one_button_on_in_kids_without_a_page_that_disagree", "crates/forms/src/tests.rs::a_field_merged_with_a_widget_that_lists_widget_kids_has_the_kids_as_buttons", "crates/forms/src/tests.rs::nameless_widgets_of_an_unlisted_field_are_options_of_its_name"]
notes = "list with jump-to-field; in Prepare a form, Shift/Ctrl/Command-click selects several fields and Properties edits their shared properties; double-click opens field properties; fields a file lists only as page widgets (empty /Fields) are adopted too, and a widget whose /Parent names a missing object is a field of its own; a named kid whose /Parent does not name the field listing it is named up its own /Parent chain; fields sharing a name are listed apart, and a radio option any of them draws is chosen by the name, leaving one button on even when their kids have no /P and were saved both on; every field of the name, and a field listing them that shares it, takes the chosen /V; a field dictionary merged with a widget that also lists widget kids has those kids as its buttons; widget annotations listed beside named kids, or listed in /Fields under an unlisted /Parent, are read as buttons of that parent's name"
[[feature]]
id = "form.highlight-fields"
@@ -6767,8 +6767,8 @@ milestone = "M10"
status = "partial"
commands = ["edit.edit_text", "ocr.correct"]
tools = ["text_edit", "ocr_recognize"]
evidence = ["crates/engine/tests/it/ocr_edit.rs::editing_an_ocr_word_covers_the_scan_and_shows_the_new_text", "crates/engine/tests/it/ocr_edit.rs::an_explicit_colour_is_kept_for_the_new_text", "crates/engine/tests/it/ocr_edit.rs::ocr_words_group_into_phrases_that_edit_in_place", "crates/engine/tests/it/ocr_edit.rs::a_longer_ocr_phrase_grows_rightwards", "crates/engine/tests/it/ocr_edit.rs::a_rewritten_ocr_phrase_keeps_its_word_widths", "crates/engine/tests/it/ocr_edit.rs::an_edited_ocr_phrase_is_drawn_at_one_size", "crates/engine/tests/it/ocr_edit.rs::an_ocr_layer_under_the_scan_image_shows_its_edit", "crates/engine/tests/it/ocr_edit.rs::an_ocr_phrase_split_from_visible_text_under_the_scan_shows_its_edit", "crates/ui-egui/tests/it/ocr.rs::an_ocr_word_opens_for_editing_and_stays_open", "crates/ui-egui/tests/it/ocr.rs::editing_a_scanned_page_recognizes_its_text_first_and_only_once", "crates/engine/src/ocr.rs::editable_writer_refuses_empty_and_writes_visible_words", "crates/engine/src/ocr.rs::editable_output_round_trips_non_win_ansi_words", "crates/engine/src/ocr.rs::editable_writer_maps_rotated_pages", "crates/automation/tests/it/automation.rs::ocr_recognize_editable_output_writes_visible_text"]
notes = "On demand, per edit: in Edit text & images, recognised text (the invisible OCR layer over a scan) is grouped into phrases (words across a word space; a wider gap such as a table cell, and each scanned line, stay separate) and opens for editing on a click. Applying draws the new text visibly in the scan's sampled ink colour (or the chosen one) and first covers the old words with the paper colour sampled around them, so the scanned word no longer shows; when an image or form is drawn after the OCR layer over where the edit lands (its old box, moved, rewrapped or grown to more lines), or an inline image or shading after it (a layer written under its scan, #3973), the cover and new text are drawn on top of the page instead, so they show; the phrase keeps each word's position and width and grows rightwards instead of wrapping, and is set in one size, the median of its words' (OCR sizes each word from its own box, several points apart), both while typing and when applied (#3313). Turning Edit text & images on for a scanned page (an image covering at least half of a page that has no text of its own, once per page per window; comments and text boxes over it do not count as text, a page with real text and a background picture is left alone; one undoable \"Recognize text\" step) now recognizes its words first, as Acrobat's \"Converting scanned page to Editable Text and Images\" does (English only until more OCR languages exist, gap 7). Still missing: separating the photographs on the page from the lettering, matching the scanned typeface (the layer's Helvetica is used), covering on a textured or patterned background (one flat colour is used), a scan painted by a pattern fill or a filled rectangle over a layer written under it, and, for text drawn on top, the clip, optional content and marked content (tags) that enclosed the OCR layer."
evidence = ["crates/engine/tests/it/ocr_edit.rs::editing_an_ocr_word_covers_the_scan_and_shows_the_new_text", "crates/engine/tests/it/ocr_edit.rs::an_explicit_colour_is_kept_for_the_new_text", "crates/engine/tests/it/ocr_edit.rs::ocr_words_group_into_phrases_that_edit_in_place", "crates/engine/tests/it/ocr_edit.rs::a_longer_ocr_phrase_grows_rightwards", "crates/engine/tests/it/ocr_edit.rs::a_rewritten_ocr_phrase_keeps_its_word_widths", "crates/engine/tests/it/ocr_edit.rs::deleting_an_ocr_word_leaves_the_rest_of_the_line_as_the_scan", "crates/engine/tests/it/ocr_edit.rs::replacing_an_ocr_word_redraws_only_that_word", "crates/engine/tests/it/ocr_edit.rs::an_edited_ocr_phrase_is_drawn_at_one_size", "crates/engine/tests/it/ocr_edit.rs::an_ocr_layer_under_the_scan_image_shows_its_edit", "crates/engine/tests/it/ocr_edit.rs::an_ocr_phrase_split_from_visible_text_under_the_scan_shows_its_edit", "crates/ui-egui/tests/it/ocr.rs::an_ocr_word_opens_for_editing_and_stays_open", "crates/ui-egui/tests/it/ocr.rs::editing_a_scanned_page_recognizes_its_text_first_and_only_once", "crates/engine/src/ocr.rs::editable_writer_refuses_empty_and_writes_visible_words", "crates/engine/src/ocr.rs::editable_output_round_trips_non_win_ansi_words", "crates/engine/src/ocr.rs::editable_writer_maps_rotated_pages", "crates/automation/tests/it/automation.rs::ocr_recognize_editable_output_writes_visible_text"]
notes = "On demand, per edit: in Edit text & images, recognised text (the invisible OCR layer over a scan) is grouped into phrases (words across a word space; a wider gap such as a table cell, and each scanned line, stay separate) and opens for editing on a click. Applying draws the new text visibly in the scan's sampled ink colour (or the chosen one) and first covers the old words with the paper colour sampled around them, so the scanned word no longer shows; a text edit covers and redraws only the words it changes, and the rest of the line stays the scan, with a deleted word leaving its gap (a replacement a little too wide is squeezed across to fit, wider than that it takes over the word it would run into, and a change of formatting redraws the whole phrase, #3313); when an image or form is drawn after the OCR layer over where the edit lands (its old box, moved, rewrapped or grown to more lines), or an inline image or shading after it (a layer written under its scan, #3973), the cover and new text are drawn on top of the page instead, so they show; the phrase keeps each word's position and width and grows rightwards instead of wrapping, and is set in one size, the median of its words' (OCR sizes each word from its own box, several points apart), both while typing and when applied (#3313). Turning Edit text & images on for a scanned page (an image covering at least half of a page that has no text of its own, once per page per window; comments and text boxes over it do not count as text, a page with real text and a background picture is left alone; one undoable \"Recognize text\" step) now recognizes its words first, as Acrobat's \"Converting scanned page to Editable Text and Images\" does (English only until more OCR languages exist, gap 7). Still missing: separating the photographs on the page from the lettering, matching the scanned typeface (the layer's Helvetica is used), covering on a textured or patterned background (one flat colour is used), a scan painted by a pattern fill or a filled rectangle over a layer written under it, and, for text drawn on top, the clip, optional content and marked content (tags) that enclosed the OCR layer."
[[feature]]
id = "ocr.ocr-downsample"
@@ -8017,8 +8017,8 @@ milestone = "M11"
status = "partial"
commands = ["standards.pdfa"]
tools = ["pdfa_verify"]
evidence = ["crates/preflight/src/tests.rs::verify_reports_what_breaks_pdfa", "crates/preflight/src/tests.rs::standards_identification_uses_namespace_not_prefix", "crates/preflight/src/tests.rs::convert_fixes_all_but_fonts", "crates/preflight/src/tests.rs::verify_reports_an_output_intent_that_does_not_match_device_colour", "crates/preflight/src/tests.rs::xmp_simple_values_have_their_lexical_forms", "crates/preflight/src/tests.rs::xmp_arrays_have_their_kind_and_item_types", "crates/preflight/src/tests.rs::xmp_structures_have_their_field_types", "crates/preflight/src/tests.rs::xmp_value_types_are_reported_by_verify", "crates/preflight/src/tests.rs::xmp_mistyped_properties_name_their_type", "crates/preflight/src/tests.rs::xmp_properties_are_known_by_namespace_uri_and_named_as_written", "crates/automation/tests/it/automation.rs::pdfa_through_tools", "crates/ui-egui/tests/it/compare.rs::pdfa_dialog_verifies_and_converts"]
notes = "PDF/A-2b/3b structural rules: identification, the value types of the predefined XMP schemas (6.6.2.3.1: dc, xmp, xmpRights, xmpMM, xmpBJ, xmpTPg, xmpDM, pdf, photoshop, crs and their structures; the tiff, exif and aux namespaces, whose tables XMP Part 2 defers to CIPA DC-010, are not yet checked), encryption, output intent (present, and RGB/CMYK to match DeviceRGB/DeviceCMYK use), fonts embedded, annotations, actions, images, embedded files. Not yet: content-stream operator checks, transparency details, PDF/A-1 and -4, a veraPDF comparison."
evidence = ["crates/preflight/src/tests.rs::verify_reports_what_breaks_pdfa", "crates/preflight/src/tests.rs::standards_identification_uses_namespace_not_prefix", "crates/preflight/src/tests.rs::convert_fixes_all_but_fonts", "crates/preflight/src/tests.rs::verify_reports_an_output_intent_that_does_not_match_device_colour", "crates/preflight/src/tests.rs::xmp_simple_values_have_their_lexical_forms", "crates/preflight/src/tests.rs::xmp_arrays_have_their_kind_and_item_types", "crates/preflight/src/tests.rs::xmp_structures_have_their_field_types", "crates/preflight/src/tests.rs::xmp_value_types_are_reported_by_verify", "crates/preflight/src/tests.rs::xmp_exif_schemas_have_their_value_types", "crates/preflight/src/tests.rs::xmp_exif_properties_are_known_by_namespace_uri", "crates/preflight/src/tests.rs::xmp_extension_schema_descriptions_are_complete", "crates/preflight/src/tests.rs::xmp_extension_findings_say_where_they_are", "crates/preflight/src/tests.rs::xmp_extension_naming_entries_are_not_empty", "crates/preflight/src/tests.rs::xmp_mistyped_properties_name_their_type", "crates/preflight/src/tests.rs::xmp_properties_are_known_by_namespace_uri_and_named_as_written", "crates/automation/tests/it/automation.rs::pdfa_through_tools", "crates/ui-egui/tests/it/compare.rs::pdfa_dialog_verifies_and_converts", "crates/preflight/src/tests.rs::implementation_limits_are_reported", "crates/preflight/src/tests.rs::implementation_limits_reach_the_trailer_split_content_deep_values_and_huge_boxes", "crates/preflight/src/tests.rs::implementation_limits_cover_reals_and_cids", "crates/preflight/src/tests.rs::separation_and_devicen_spaces_are_checked", "crates/preflight/src/tests.rs::separation_and_devicen_checks_follow_references_depth_and_nchannel", "crates/preflight/src/tests.rs::file_header_is_checked_and_rewritten", "crates/preflight/src/tests.rs::extended_graphics_states_are_checked", "crates/preflight/src/tests.rs::graphics_state_scan_reads_each_object_once_whatever_the_generation", "crates/preflight/src/tests.rs::trailer_file_identifier_is_checked_and_added", "crates/preflight/src/tests.rs::set_state_no_op_and_named_actions_are_checked"]
notes = "PDF/A-2b/3b structural rules: the file header (6.1.2), identification, the trailer file identifier (6.1.3), the value types of the predefined XMP schemas (6.6.2.3.1: dc, xmp, xmpRights, xmpMM, xmpBJ, xmpTPg, xmpDM, pdf, photoshop, crs, and tiff, exif and aux from the September 2005 XMP Specification, with their structures), XMP extension schema descriptions (6.6.2.3.3, after PDF Association TechNote 0009 section 4: required entries of the schema, property, value type and field descriptions, none of the naming entries empty, and the five required prefixes; that a property's custom value type has its own pdfaSchema:valueType description is not yet checked), encryption, output intent (present, and RGB/CMYK to match DeviceRGB/DeviceCMYK use), fonts embedded, graphics states (6.2.5: TR, TR2, HTP, halftone types, names and transfer functions with the primary components of ISO 32000-1 10.5.5.6, rendering intents), annotations, actions (with SetState, NOP and Named actions other than the four page moves), images, embedded files, the implementation limits of 6.1.13 (integers, reals, strings, names, CIDs and page boundaries, ISO 32000-1 Annex C), Separation and DeviceN spaces (6.2.4.4: one definition per colorant name, spot colorants in /Colorants; a device alternate without a matching output intent is not yet reported), and transparency (6.2.10: blend modes ISO 32000-1 defines, and a blending colour space on pages with transparency when there is no PDF/A output intent). Not yet: content-stream operator checks, a transparency group blending colour space checked against the output intent, PDF/A-1 and -4, a veraPDF comparison."
[[feature]]
id = "optimize.pdfa-convert"
@@ -8029,8 +8029,8 @@ milestone = "M11"
status = "partial"
commands = ["standards.pdfa"]
tools = ["pdfa_convert"]
evidence = ["crates/preflight/src/tests.rs::convert_fixes_all_but_fonts", "crates/preflight/src/tests.rs::convert_does_not_give_cmyk_documents_an_srgb_output_intent", "crates/automation/tests/it/automation.rs::pdfa_through_tools", "crates/ui-egui/tests/it/compare.rs::pdfa_dialog_verifies_and_converts", "crates/engine/src/tests.rs::convert_to_pdfa_needs_the_owners_rights_on_an_encrypted_document"]
notes = "Fixes XMP, output intent (generated sRGB ICC profile, only for documents without DeviceCMYK; CMYK documents need a user-chosen CMYK output intent, not yet offered), forbidden actions, annotation flags, interpolation, encryption, AFRelationship. Not yet: embedding fonts, converting device CMYK, flattening transparency for PDF/A-1. On an encrypted document it needs the owner password, as Protect and Remove security do, since it removes the encryption (#3921)."
evidence = ["crates/preflight/src/tests.rs::convert_fixes_all_but_fonts", "crates/preflight/src/tests.rs::file_header_is_checked_and_rewritten", "crates/engine/src/tests.rs::save_as_pdfa_saves_a_compliant_file_header", "crates/preflight/src/tests.rs::convert_does_not_give_cmyk_documents_an_srgb_output_intent", "crates/automation/tests/it/automation.rs::pdfa_through_tools", "crates/ui-egui/tests/it/compare.rs::pdfa_dialog_verifies_and_converts", "crates/engine/src/tests.rs::convert_to_pdfa_needs_the_owners_rights_on_an_encrypted_document", "crates/preflight/src/tests.rs::trailer_file_identifier_is_checked_and_added"]
notes = "Fixes XMP, output intent (generated sRGB ICC profile, only for documents without DeviceCMYK; CMYK documents need a user-chosen CMYK output intent, not yet offered), forbidden actions, annotation flags, interpolation, encryption, AFRelationship, the file header (a full save at version 1.7 at most), a missing or malformed file identifier. Not yet: embedding fonts, converting device CMYK, flattening transparency for PDF/A-1. On an encrypted document it needs the owner password, as Protect and Remove security do, since it removes the encryption (#3921)."
[[feature]]
id = "optimize.pdfa-accessible-levels"