CLI: --strict on convert and render refuses missing glyphs and pictures (#6039)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Martin BallandClaude Opus 5.5 authored and GitHub committed 2026-11-09 20:16:05 +01:00
1 parent 05f781cfda
commit d40d1b2718
54 files changed
+3051 -167

No files matched your search

Generated
+3
View File
@@ -6846,13 +6846,16 @@ dependencies = [
name = "wordcraft-cli"
version = "0.15.1"
dependencies = [
"image",
"serde_json",
"sys-locale",
"wordcraft-doc",
"wordcraft-engine",
"wordcraft-fonts",
"wordcraft-mcp",
"wordcraft-render",
"wordcraft-zotero",
"zip 3.0.0",
]
[[package]]
+18
View File
@@ -48,3 +48,21 @@ _Avoid_: User dictionary (in the interface)
A folder of Hunspell dictionaries you supply, used before the system's (an installed Language pack
comes after it).
_Avoid_: Custom dictionaries
## Export
**Strict export**:
An export that writes nothing, and reports why, when a character or picture the output is meant
to show would be missing from it. Text the output doesn't show (hidden text, deletions without
markup) isn't checked.
_Avoid_: Safe mode, validated export
**Missing glyph**:
A character the output draws that no face in the font fallback chain can draw, so it would appear
as an empty box.
_Avoid_: Tofu, box, notdef character
**Missing picture**:
A picture the document refers to that the output cannot include: its part is absent, its data
cannot be decoded, its format is unsupported, or it is linked from outside the file.
_Avoid_: Broken image, dropped image
+35
View File
@@ -157,6 +157,7 @@ Command line:
```sh
wordcraft-cli convert report.docx report.pdf # docx, pdf, odt, rtf, html, md, tex, txt, png
wordcraft-cli convert report.docx report.pdf --strict # exit 3, no file, if a character or picture would be left out
wordcraft-cli text report.docx # plain text
wordcraft-cli inspect report.docx # structure as JSON
wordcraft-cli run --template sample \
@@ -164,6 +165,40 @@ wordcraft-cli run --template sample \
wordcraft-cli run --script job.json # the same from a file: commands, then save to docx and pdf
```
`convert --strict` (a `.pdf` or `.png` output only) and `render --strict` (a PNG of one page) refuse a silently incomplete render: they exit 3
and write nothing if the output would draw a character no font has (an empty box) or leave out a
picture. Exit codes are 0 (written), 1 (unreadable input, usage error, or `--strict` with another
output type, checked before the input is read) and 3 (strict refusal). On exit 3, stderr has one
`wordcraft-cli: strict: …` line per kind, then one JSON line:
```sh
wordcraft-cli convert in.docx out.pdf --strict
# wordcraft-cli: strict: 1 character cannot be drawn: U+2D30 (×3)
# wordcraft-cli: strict: 1 picture(s) left out: word/media/image2.bin (header2.xml): unsupported format
# {"schema":1,"strict":"failed","missing_glyphs":[{"cp":"U+2D30","count":3}],"missing_pictures":[{"part":"word/media/image2.bin","source_part":"word/header2.xml","reason":"unsupported_format"}],"truncated":false,"total_missing_glyphs":1,"total_missing_pictures":1,"glyphs_truncated":false,"pictures_truncated":false}
```
The JSON line (schema 1, also what `file.strictReport` returns) has `schema`, `strict`
(`"passed"` or `"failed"`), `missing_glyphs` (entries `{"cp", "count"}`, sorted by code point),
`missing_pictures` (entries `{"part", "source_part", "reason"}`: the picture's part or target, the
story part it sits in such as `word/header2.xml`, and a reason code; `source_part` is also
`background` (the page picture), `body`, `header`, `footer`, `footnote`, `endnote`, `comment` or
`text box` where the part name is not known (a document not opened from a DOCX), or a chunk path such as
`word/afchunk1.docx > word/header1.xml` for a picture inside an embedded DOCX chunk), `truncated`, the two totals
`total_missing_glyphs` and `total_missing_pictures`, and `glyphs_truncated` and
`pictures_truncated` for each list. The reason codes are `missing_part`, `external_target`,
`link_only`, `undecodable` and `unsupported_format`; the stderr lines use plain words for them. When a list is cut, the stderr line gives the total
(`300 characters cannot be drawn (first 256 listed)`).
Each list stops at 256 entries separately; `truncated` is true when either was cut, and the totals
count every distinct character and picture found, those past the cap included. Only characters the output draws count (not hidden text, or deleted text
without `--markup`), and a character a fallback font draws is fine. Text inside a WMF or EMF picture counts too, shaped in the font the picture names with no fallback. A Private Use Area character
fails only if no installed font maps it. The result depends on the fonts installed on the host
(see #5624 for a fixed font set). An external picture target is never fetched: it is reported, not
loaded. Pictures dropped when a DOCX is opened (a missing part, an external target, or a `link only` picture with no embedded copy) are reported for DOCX input only. They have no page, so a PNG of any page is also refused for a missing or external picture anywhere in the document, unused headers included. `render --strict` checks only the page it
writes (`--page`; `--scale` doesn't matter), prints nothing on stdout when it refuses, and reports the
same way. The engine takes `strict: true` on `file.save`, `file.saveAs`, `file.exportPdf` and
`file.exportPng`, and `file.strictReport {markup?, page?}` returns the report without writing.
Web: `cd apps/wordcraft-web && trunk serve`, then open <http://127.0.0.1:8771/?sample>.
## For agents: CLI and MCP
+4 -1
View File
@@ -8,7 +8,7 @@
**Stage: alpha** · next: beta, ~15 points (60% → 75% ready for real work) and ~150–230 h away
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-11-08 · **Change:** trivial (progress log: ODT columns stating only their shares); previously trivial (progress log: ODT font size for every script); previously trivial (progress log: ODT text turning off what its style turns on); previously trivial (progress log: edited ODT paragraphs keep custom styles with an outline level); previously trivial (progress log: ODT pictures keep the size their file gives); previously trivial (progress log: ODT frames keep the page they are anchored to); previously trivial (pointer zoom stretches page rasters until it stops); previously trivial (dark-theme interface text on Linux no longer looks semi-bold); previously trivial (Control/MCP #2596 review: partial-write outcomes, path-bearing image exports, accept-thread startup cleanup and malformed UTF-8 handling); previously trivial (progress log: a turned inline picture whose file keeps no room for the turn keeps its line to its frame); previously trivial (progress log: the Arabic interface's Styles gallery reads from the right (#5566)); previously trivial (progress log: ODT pictures offered in several formats); previously trivial (progress log: ODT heading text of another size); previously trivial (Protected typing boundary lookup counts preceding objects once and indexes editable permission IDs (#3614); caret movement and enforcement policy are unchanged; previously trivial: progress log: Custom document properties dialog, #2982; previously trivial: progress log: View › New Window; previously trivial: progress log: interface translations come from storytold/craft-i18n through the `craft-i18n` crate; previously trivial: progress log: disabled-control reasons translated in every interface language, #2678; previously trivial: progress log: mail merge record navigation and record ranges for printing, #3028; previously trivial: selection bar for right-to-left paragraphs and table rows; previously trivial: group connector rerouting preserves rotated and flipped free or unresolved endpoints under nonuniform group scaling; readiness estimates unchanged; previously trivial: progress log: a word with no break opportunity in it no longer makes the layout quadratic, #3412; previously trivial: progress log: a relationship the markup names without the prefix is read as it wrote, and an ordinary `id` names no part; previously trivial: progress log: a document with many relationships opens in the time its part takes, and an id the text only mentions is not a reference, #3411; previously trivial: progress log: read only and comments-only protection grey out edits; previously trivial: progress log: an empty paragraph with a hidden mark shows nothing, not even its list number, #3982; previously trivial: progress log: "NaN" in the Font dialog's Size box no longer closes the app, #3663; previously trivial: progress log: a table cell span from a `.json` file no longer overflows, #3637; previously trivial: progress log: a click in the footer while editing the header moves there; previously trivial: explicit documents keep the editor when recovery discovery finishes; readable recovery controls; previously trivial: Slovenian interface catalog, 2,614 entries; previously trivial: Insert Table of Figures dialog for any caption label, #3013; previously trivial: progress log: Home › Underline drop-down, #2738; previously trivial: personal templates in File › New and Save as Word Template, #2852; previously trivial: progress log: the scrollbar page tip stays on one line, #3341; previously trivial: progress log: forms protection greys out the control buttons; previously trivial: progress log: a document with many content controls saves in the time the controls take, #13; previously trivial: progress log: Word signature lines kept on save; previously trivial: a Thai run with Latin finds one installed font covering it all on Linux too, #1549; previously trivial: progress log: closing the header returns to the body selection, #3331; previously trivial: progress log: saving to a NAS share mounted by GVFS, #3003; previously trivial: progress log: a picture dragged near the bottom of a page lands there, #3169; previously trivial: centred list labels after leading page breaks, #2964; previously trivial: progress log: inline pictures drag to an insertion cursor and stay inline, #2233, #2234; previously trivial: progress log: a text file saved over itself closes without asking, #2376; previously trivial: progress log: a text frame in the paragraph defaults frames nothing, #2360; previously trivial: fixed font inputs in four layout controls; previously trivial: typing no longer starts render threads on every repaint, #3027; previously trivial: a right, centre or decimal tab whose text doesn't fit, or a left tab past the right indent, goes to the next line with its text, #1840; previously trivial: progress log: banded rows shade as in Word, #3415; previously trivial: progress log: the table handles sLine truncated
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-11-08 · **Change:** trivial (progress log: `convert --strict` and `render --strict` refuse a silently incomplete PDF or PNG (#6005, #6008)); ; trivial (progress log: ODT columns stating only their shares); previously trivial (progress log: ODT font size for every script); previously trivial (progress log: ODT text turning off what its style turns on); previously trivial (progress log: edited ODT paragraphs keep custom styles with an outline level); previously trivial (progress log: ODT pictures keep the size their file gives); previously trivial (progress log: ODT frames keep the page they are anchored to); previously trivial (pointer zoom stretches page rasters until it stops); previously trivial (dark-theme interface text on Linux no longer looks semi-bold); previously trivial (Control/MCP #2596 review: partial-write outcomes, path-bearing image exports, accept-thread startup cleanup and malformed UTF-8 handling); previously trivial (progress log: a turned inline picture whose file keeps no room for the turn keeps its line to its frame); previously trivial (progress log: the Arabic interface's Styles gallery reads from the right (#5566)); previously trivial (progress log: ODT pictures offered in several formats); previously trivial (progress log: ODT heading text of another size); previously trivial (Protected typing boundary lookup counts preceding objects once and indexes editable permission IDs (#3614); caret movement and enforcement policy are unchanged; previously trivial: progress log: Custom document properties dialog, #2982; previously trivial: progress log: View › New Window; previously trivial: progress log: interface translations come from storytold/craft-i18n through the `craft-i18n` crate; previously trivial: progress log: disabled-control reasons translated in every interface language, #2678; previously trivial: progress log: mail merge record navigation and record ranges for printing, #3028; previously trivial: selection bar for right-to-left paragraphs and table rows; previously trivial: group connector rerouting preserves rotated and flipped free or unresolved endpoints under nonuniform group scaling; readiness estimates unchanged; previously trivial: progress log: a word with no break opportunity in it no longer makes the layout quadratic, #3412; previously trivial: progress log: a relationship the markup names without the prefix is read as it wrote, and an ordinary `id` names no part; previously trivial: progress log: a document with many relationships opens in the time its part takes, and an id the text only mentions is not a reference, #3411; previously trivial: progress log: read only and comments-only protection grey out edits; previously trivial: progress log: an empty paragraph with a hidden mark shows nothing, not even its list number, #3982; previously trivial: progress log: "NaN" in the Font dialog's Size box no longer closes the app, #3663; previously trivial: progress log: a table cell span from a `.json` file no longer overflows, #3637; previously trivial: progress log: a click in the footer while editing the header moves there; previously trivial: explicit documents keep the editor when recovery discovery finishes; readable recovery controls; previously trivial: Slovenian interface catalog, 2,614 entries; previously trivial: Insert Table of Figures dialog for any caption label, #3013; previously trivial: progress log: Home › Underline drop-down, #2738; previously trivial: personal templates in File › New and Save as Word Template, #2852; previously trivial: progress log: the scrollbar page tip stays on one line, #3341; previously trivial: progress log: forms protection greys out the control buttons; previously trivial: progress log: a document with many content controls saves in the time the controls take, #13; previously trivial: progress log: Word signature lines kept on save; previously trivial: a Thai run with Latin finds one installed font covering it all on Linux too, #1549; previously trivial: progress log: closing the header returns to the body selection, #3331; previously trivial: progress log: saving to a NAS share mounted by GVFS, #3003; previously trivial: progress log: a picture dragged near the bottom of a page lands there, #3169; previously trivial: centred list labels after leading page breaks, #2964; previously trivial: progress log: inline pictures drag to an insertion cursor and stay inline, #2233, #2234; previously trivial: progress log: a text file saved over itself closes without asking, #2376; previously trivial: progress log: a text frame in the paragraph defaults frames nothing, #2360; previously trivial: fixed font inputs in four layout controls; previously trivial: typing no longer starts render threads on every repaint, #3027; previously trivial: a right, centre or decimal tab whose text doesn't fit, or a left tab past the right indent, goes to the next line with its text, #1840; pLine truncated
>
> **Recipient CSV update:** 2026-10-27 · Complete quoted header records choose the delimiter (#4249); detection and regressions follow the current mailings module split. Recipient limits and readiness estimates are unchanged.
>
@@ -150,6 +150,7 @@ Ranked, to beta (detail and remaining milestones in [docs/roadmap.md](docs/roadm
## Progress log
- 2026-11-08: `wordcraft-cli convert --strict` (#6005) and `render --strict` (#6008): a PDF or PNG export (render: the page it writes) refuses, exit 3 and nothing written, when it would draw a character no font has or leave out a picture (missing part, external target, link only, undecodable or unsupported format); the report is on stderr with one JSON line. Engine: `strict: true` on `file.save`, `file.saveAs`, `file.exportPdf` and `file.exportPng`, and the pure `file.strictReport`. External picture targets are never fetched. The result depends on the host's installed fonts (#5624); readiness estimates unchanged.
- 2026-11-05: ODT tables with a usable width keep their column proportions when the file gives only shares (gap 19): LibreOffice's CV template opened and saved with equal columns.
- 2026-11-05: ODT font sizes reach Asian and complex-script text (gap 19): Chinese text set to 24 pt stayed at the default size in LibreOffice.
- 2026-11-05: ODT text can turn off what its style turns on (gap 19): a word not bold in a bold heading showed bold, and came back bold in LibreOffice after a save.
@@ -882,6 +883,8 @@ MATH glyphs that reference the same construction now share its immutable variant
## Revision history
> 2026-11-08 · Progress log: `convert --strict` refuses a PDF or PNG that would silently lack a character or a picture (#6005).
>
> 2026-11-08 · Progress log: turned inline pictures without a recorded effect extent keep their line to the frame, as in Word.
>
> 2026-11-05 · Linux self-update ignores another program's AppImage variables (#5653); recorded in the progress log, readiness unchanged.
+5
View File
@@ -21,3 +21,8 @@ wordcraft-mcp = { workspace = true }
wordcraft-zotero = { workspace = true }
serde_json = { workspace = true }
sys-locale = "0.3"
[dev-dependencies]
wordcraft-fonts = { workspace = true }
image = { workspace = true }
zip = { workspace = true }
+173 -55
View File
@@ -17,17 +17,23 @@ const USAGE: &str = "\
wordcraft-cli — WordCraft from the command line
USAGE:
wordcraft-cli convert <in> <out> [--markup] [--update-fields] [--format wordml2003|flatopc]
wordcraft-cli convert <in> <out> [--markup] [--update-fields] [--strict] [--format wordml2003|flatopc]
convert between formats (docx, doc [read], pdf, odt, rtf, html, md, tex, txt, json, png,
xml); --update-fields refreshes unlocked fields before saving;
an .xml file is a Word XML Document (Flat OPC), or with
--format wordml2003 a Word 2003 XML Document;
--markup (Print Markup): comments beside each page and
tracked changes in pdf/png, a comment list in html/md/txt
tracked changes in pdf/png, a comment list in html/md/txt;
--strict (pdf or png output only) writes nothing and exits 3
if a character can't be drawn or a picture is missing or
undecodable; the report is on stderr, with one JSON line
wordcraft-cli info <file> pages, words, paragraphs, properties (JSON)
wordcraft-cli text <file> plain text
wordcraft-cli inspect <file> document structure (JSON)
wordcraft-cli render <file> <out.png> [--page N] [--scale S]
wordcraft-cli render <file> <out.png> [--page N] [--scale S] [--strict]
--strict writes nothing and exits 3 if a character on the
page can't be drawn or a picture is missing or undecodable
(report on stderr, as for convert --strict)
wordcraft-cli run [--file F | --template T] [--script FILE|-] --cmd 'id={json}' [--cmd …]
[--save OUT] [--print] [--author NAME]
run commands headlessly, then save (--author names
@@ -77,6 +83,14 @@ fn wants_help(cmd: &str, rest: &[String]) -> bool {
SUBCOMMANDS.contains(&cmd) && rest.iter().any(|a| a == "--help" || a == "-h")
}
/// A failure whose message may carry text from a file or from the engine's reading of one (a
/// picture's part name, a parse error quoting the input): control, bidi and invisible characters
/// are escaped, so it can't forge or spoof a line (the CLI's own messages, which have line breaks
/// on purpose, are not routed through here). Backslashes stay: messages quote Windows paths.
fn engine_error(e: impl fmt::Display) -> Failure {
Failure::Error(escape_text(&e.to_string(), false))
}
/// The error every subcommand gives when a required path is not there.
fn missing_argument() -> Failure {
format!("missing argument\n\n{USAGE}").into()
@@ -85,11 +99,11 @@ fn missing_argument() -> Failure {
/// Open a document for a subcommand. A password-protected Word file needs its password
/// ([`password_from`]). As in the app, a document opened with one is saved with it again, so
/// `convert` does not quietly write a copy the password no longer protects.
fn open(path: &Path, password: Option<&str>) -> Result<Session, String> {
fn open(path: &Path, password: Option<&str>) -> Result<Session, Failure> {
// A file that gives no page size (HTML, Markdown, plain text) opens on the system region's
// paper, as in the app: A4 in Italy, Letter in the US (#1381).
let region = system_region();
let (doc, encrypted) = wordcraft_engine::io::open_path_in(path, password, &region)?;
let (doc, encrypted) = wordcraft_engine::io::open_path_in(path, password, &region).map_err(engine_error)?;
let mut s = session(doc);
s.path = Some(path.into());
if encrypted {
@@ -198,7 +212,7 @@ fn listed_text(doc: &wordcraft_doc::Document) -> Result<String, String> {
/// The command also checks whether the open document is protected: this name determines which
/// named editing exceptions the session can use, so the flag must not bypass that check.
fn set_author(s: &mut Session, name: &str) -> Result<(), String> {
s.run("file.setAuthor", &json!({"name": name})).map(|_| ()).map_err(|e| format!("--author: {e}"))
s.run("file.setAuthor", &json!({"name": name})).map(|_| ()).map_err(|e| escape_text(&format!("--author: {e}"), false))
}
fn arg_value(args: &[String], name: &str) -> Option<String> {
@@ -310,10 +324,12 @@ const CONVERT_OPTIONS: &[OptionSpec] = &[
OptionSpec { name: "--markup", takes_value: false },
OptionSpec { name: "--format", takes_value: true },
OptionSpec { name: "--update-fields", takes_value: false },
OptionSpec { name: "--strict", takes_value: false },
OptionSpec { name: "--password", takes_value: true },
OptionSpec { name: "--password-file", takes_value: true },
];
const RENDER_OPTIONS: &[OptionSpec] = &[
OptionSpec { name: "--strict", takes_value: false },
OptionSpec { name: "--page", takes_value: true },
OptionSpec { name: "--scale", takes_value: true },
OptionSpec { name: "--password", takes_value: true },
@@ -350,6 +366,9 @@ enum Failure {
/// Whoever read stdout closed it (`wordcraft-cli commands --json | head`): there is nobody left
/// to write to, so stop quietly and exit 0 like other command-line tools.
StdoutClosed,
/// `convert --strict` refused: the text for stderr (the report's lines, then its JSON line).
/// Exit status 3.
Strict(String),
}
impl fmt::Display for Failure {
@@ -357,6 +376,7 @@ impl fmt::Display for Failure {
match self {
Failure::Error(e) => f.write_str(e),
Failure::StdoutClosed => f.write_str("stdout closed"),
Failure::Strict(report) => f.write_str(report),
}
}
}
@@ -402,11 +422,80 @@ fn valid_connect_addr(addr: &str) -> bool {
!host.is_empty() && port.parse::<u16>().is_ok()
}
/// A failed `--strict` export: a strict refusal becomes [`Failure::Strict`] carrying what stderr
/// shows (each line of the report as `wordcraft-cli: …`, then its JSON on one line, from
/// `file.strictReport` for `page` of a page image, else the PDF); any other error is an ordinary
/// failure.
fn refusal(s: &mut Session, e: wordcraft_engine::CmdError, markup: bool, page: Option<u64>) -> Failure {
if !matches!(&e, wordcraft_engine::CmdError::Failed(m) if m.starts_with("strict: ")) {
return engine_error(e);
}
let mut params = json!({"markup": markup});
if let Some(page) = page {
params["page"] = json!(page);
}
Failure::Strict(match s.run("file.strictReport", &params).map_err(|e| e.to_string()) {
Ok(json) => {
let mut text = String::new();
for line in report_lines(&json) {
text += &format!("wordcraft-cli: {}\n", escape_peer(&line));
}
text + &json_line_safe(&json.to_string())
}
Err(e) => format!("wordcraft-cli: strict: refused, and the report could not be made: {}", escape_text(&e, false)),
})
}
/// The report's human lines, from its JSON (the engine's `StrictReport::lines`).
fn report_lines(json: &Value) -> Vec<String> {
wordcraft_engine::strict::lines_from_json(json)
}
/// `convert <in> <out>`: `rest` is the command line after `convert`, `open` opens the input.
fn run_convert(rest: &[String], open: impl Fn(&str, Option<&str>) -> Result<Session, Failure>) -> Result<(), Failure> {
validate_options("convert", rest, CONVERT_OPTIONS)?;
// `--format wordml2003` (or `flatopc`) picks the kind of `.xml` file; its value isn't a path.
let format = match find_option_arg(rest, "--format") {
OptionArg::Absent => None,
OptionArg::Duplicate => return Err("convert: duplicate --format option".into()),
OptionArg::MissingValue => return Err("convert: --format requires a value".into()),
OptionArg::Value(value) => Some(value),
};
let paths = positionals(rest, CONVERT_OPTIONS);
let (i, o) = (paths.first().ok_or_else(missing_argument)?.to_string(), paths.get(1).ok_or_else(missing_argument)?.to_string());
let markup = rest.iter().any(|a| a == "--markup");
let strict = rest.iter().any(|a| a == "--strict");
// Before the input is opened: a wrong output type is a usage error, not a file error.
let to_png = o.to_ascii_lowercase().ends_with(".png");
if strict && !(to_png || o.to_ascii_lowercase().ends_with(".pdf")) {
return Err("convert: --strict needs a .pdf or .png output".into());
}
let password = password_from(rest)?;
let mut s = open(&i, password.as_deref())?;
if rest.iter().any(|a| a == "--update-fields") {
s.run("references.updateFields", &json!({})).map_err(engine_error)?;
}
let r = if to_png {
s.run("file.exportPng", &json!({"path": o, "page": 1, "markup": markup, "strict": strict}))
} else {
let mut params = json!({"path": o, "markup": markup, "strict": strict});
if let Some(f) = format {
params["format"] = json!(f);
}
s.run("file.save", &params)
};
if let Err(e) = r {
return Err(if strict { refusal(&mut s, e, markup, to_png.then_some(1)) } else { engine_error(e) });
}
eprintln!("wrote {o}");
Ok(())
}
fn load_run_script(rest: &[String], script_path: Option<&Path>) -> Result<Option<script::Script>, Failure> {
match arg_value(rest, "--script") {
Some(path) if !path.starts_with("--") => {
let source = script_path.unwrap_or(Path::new(&path));
Ok(Some(script::load(source)?))
Ok(Some(script::load(source).map_err(engine_error)?))
}
_ if rest.iter().any(|arg| arg == "--script") => Err("run: --script requires a value".into()),
_ => Ok(None),
@@ -438,42 +527,13 @@ fn run<S: AsRef<OsStr>>(args: &[S]) -> Result<(), Failure> {
return write_stdout(|o| write!(o, "{USAGE}"));
}
match cmd {
"convert" => {
validate_options("convert", &rest, CONVERT_OPTIONS)?;
// `--format wordml2003` (or `flatopc`) picks the kind of `.xml` file; its value isn't a path.
let format = match find_option_arg(&rest, "--format") {
OptionArg::Absent => None,
OptionArg::Duplicate => return Err("convert: duplicate --format option".into()),
OptionArg::MissingValue => return Err("convert: --format requires a value".into()),
OptionArg::Value(value) => Some(value),
};
let paths = positionals(&rest, CONVERT_OPTIONS);
let (i, o) = (paths.first().ok_or_else(missing_argument)?.to_string(), paths.get(1).ok_or_else(missing_argument)?.to_string());
let markup = rest.iter().any(|a| a == "--markup");
let password = password_from(&rest)?;
let mut s = open(&i, password.as_deref())?;
if rest.iter().any(|a| a == "--update-fields") {
s.run("references.updateFields", &json!({})).map_err(|e| e.to_string())?;
}
let r = if o.to_ascii_lowercase().ends_with(".png") {
s.run("file.exportPng", &json!({"path": o, "page": 1, "markup": markup}))
} else {
let mut params = json!({"path": o, "markup": markup});
if let Some(f) = format {
params["format"] = json!(f);
}
s.run("file.save", &params)
};
r.map_err(|e| e.to_string())?;
eprintln!("wrote {o}");
Ok(())
}
"convert" => run_convert(&rest, open),
"info" => {
validate_options("info", &rest, PASSWORD_OPTIONS)?;
let paths = positionals(&rest, PASSWORD_OPTIONS);
let password = password_from(&rest)?;
let mut s = open(paths.first().ok_or_else(missing_argument)?, password.as_deref())?;
let info = serde_json::to_string_pretty(&s.run("file.info", &json!({})).map_err(|e| e.to_string())?).unwrap_or_default();
let info = serde_json::to_string_pretty(&s.run("file.info", &json!({})).map_err(engine_error)?).unwrap_or_default();
write_stdout(|o| writeln!(o, "{info}"))
}
"text" => {
@@ -489,7 +549,7 @@ fn run<S: AsRef<OsStr>>(args: &[S]) -> Result<(), Failure> {
let paths = positionals(&rest, PASSWORD_OPTIONS);
let password = password_from(&rest)?;
let mut s = open(paths.first().ok_or_else(missing_argument)?, password.as_deref())?;
let tree = serde_json::to_string_pretty(&s.run("document.inspect", &json!({})).map_err(|e| e.to_string())?).unwrap_or_default();
let tree = serde_json::to_string_pretty(&s.run("document.inspect", &json!({})).map_err(engine_error)?).unwrap_or_default();
write_stdout(|o| writeln!(o, "{tree}"))
}
"render" => {
@@ -503,7 +563,17 @@ fn run<S: AsRef<OsStr>>(args: &[S]) -> Result<(), Failure> {
let page = parse_page_arg("render", &rest)?.unwrap_or(1);
let scale = parse_scale_arg("render", &rest)?.unwrap_or(2.0);
let mut s = open(&i, password.as_deref())?;
let r = s.run("file.exportPng", &json!({"path": o, "page": page, "scale": scale})).map_err(|e| e.to_string())?;
let strict = rest.iter().any(|a| a == "--strict");
let mut params = json!({"path": o, "page": page, "scale": scale, "strict": strict});
if strict {
// The check and its report must see the page as it is written.
params["markup"] = json!(false);
}
let r = match s.run("file.exportPng", &params) {
Ok(r) => r,
Err(e) if strict => return Err(refusal(&mut s, e, false, Some(page))),
Err(e) => return Err(engine_error(e)),
};
write_stdout(|o| writeln!(o, "{r}"))
}
"run" => {
@@ -522,8 +592,8 @@ fn run<S: AsRef<OsStr>>(args: &[S]) -> Result<(), Failure> {
let mut s = session(wordcraft_doc::Document::new());
let first = s.document_id();
let params = t.map_or_else(|| json!({}), |t| json!({"template": t}));
s.run("file.new", &params).map_err(|e| e.to_string())?;
s.close_document(first).map_err(|e| e.to_string())?;
s.run("file.new", &params).map_err(engine_error)?;
s.close_document(first).map_err(engine_error)?;
s
}
};
@@ -532,7 +602,7 @@ fn run<S: AsRef<OsStr>>(args: &[S]) -> Result<(), Failure> {
}
let print = rest.iter().any(|a| a == "--print");
for (n, (id, params)) in script.iter().flat_map(|c| c.commands.iter()).enumerate() {
let r = s.run(id, params).map_err(|e| format!("command {} ({id}): {e}", n + 1))?;
let r = s.run(id, params).map_err(|e| engine_error(format!("command {} ({id}): {e}", n + 1)))?;
if print {
write_stdout(|o| writeln!(o, "{id}: {r}"))?;
}
@@ -542,10 +612,12 @@ fn run<S: AsRef<OsStr>>(args: &[S]) -> Result<(), Failure> {
if rest.get(i).map(String::as_str) == Some("--cmd") {
let spec = rest.get(i + 1).ok_or("--cmd needs a value")?;
let (id, params) = match spec.split_once('=') {
Some((id, p)) => (id.to_string(), serde_json::from_str::<Value>(p).map_err(|e| format!("{id}: bad JSON params: {e}"))?),
Some((id, p)) => {
(id.to_string(), serde_json::from_str::<Value>(p).map_err(|e| engine_error(format!("{id}: bad JSON params: {e}")))?)
}
None => (spec.clone(), json!({})),
};
let r = s.run(&id, &params).map_err(|e| format!("{id}: {e}"))?;
let r = s.run(&id, &params).map_err(|e| engine_error(format!("{id}: {e}")))?;
if print {
write_stdout(|o| writeln!(o, "{id}: {r}"))?;
}
@@ -565,7 +637,7 @@ fn run<S: AsRef<OsStr>>(args: &[S]) -> Result<(), Failure> {
return Err(format!("{out}: is a folder, not a file").into());
}
if !out.to_ascii_lowercase().ends_with(".png") {
wordcraft_engine::io::save_bytes(out, &s.doc)?;
wordcraft_engine::io::save_bytes(out, &s.doc).map_err(engine_error)?;
}
}
for out in outputs {
@@ -581,7 +653,7 @@ fn run<S: AsRef<OsStr>>(args: &[S]) -> Result<(), Failure> {
} else {
s.run("file.save", &json!({"path": out}))
};
r.map_err(|e| e.to_string())?;
r.map_err(engine_error)?;
eprintln!("wrote {out}");
}
Ok(())
@@ -680,10 +752,12 @@ fn run<S: AsRef<OsStr>>(args: &[S]) -> Result<(), Failure> {
let mut s = open(file, None)?;
for spec in rest.iter().zip(rest.iter().skip(1)).filter(|(a, _)| *a == "--cmd").map(|(_, v)| v) {
let (id, params) = match spec.split_once('=') {
Some((id, p)) => (id.to_string(), serde_json::from_str::<Value>(p).map_err(|e| format!("{id}: bad JSON params: {e}"))?),
Some((id, p)) => {
(id.to_string(), serde_json::from_str::<Value>(p).map_err(|e| engine_error(format!("{id}: bad JSON params: {e}")))?)
}
None => (spec.clone(), json!({})),
};
s.run(&id, &params).map_err(|e| format!("{id}: {e}"))?;
s.run(&id, &params).map_err(|e| engine_error(format!("{id}: {e}")))?;
}
let mut opts = wordcraft_zotero::client::Options::default();
if let Some(p) = arg_value(&rest, "--port") {
@@ -700,13 +774,13 @@ fn run<S: AsRef<OsStr>>(args: &[S]) -> Result<(), Failure> {
}
r
})
.map_err(|e| e.to_string())?;
.map_err(engine_error)?;
for a in &host.alerts {
eprintln!("Zotero: {}", escape_peer(a));
}
eprintln!("{} calls, {}", out.calls, if out.completed { "completed" } else { "ended without completing (cancelled?)" });
if let Some(o) = arg_value(&rest, "--save") {
s.run("file.save", &json!({"path": o})).map_err(|e| e.to_string())?;
s.run("file.save", &json!({"path": o})).map_err(engine_error)?;
eprintln!("wrote {o}");
}
Ok(())
@@ -730,22 +804,62 @@ fn clip(s: &str) -> String {
/// overrides are shown as escapes (`\\`, `\n`, `\u{1b}`), so it can't end its line, drive the
/// terminal or pass for other output.
fn escape_peer(s: &str) -> String {
escape_text(s, true)
}
/// Whether `c` must not reach a terminal as itself: a control character, a line or paragraph
/// separator, a bidi control, or one that shows nothing (soft hyphen, zero-width characters, word
/// joiners, the byte order mark, tag characters), which can hide or reorder what a line says.
fn is_unsafe_char(c: char) -> bool {
c.is_control()
|| matches!(
c,
'\u{00ad}'
| '\u{061c}'
| '\u{200b}'..='\u{200f}'
| '\u{2028}'..='\u{202e}'
| '\u{2060}'..='\u{2064}'
| '\u{2066}'..='\u{2069}'
| '\u{feff}'
| '\u{e0000}'..='\u{e007f}'
)
}
/// [`escape_peer`], with backslashes escaped only when `backslash` (error messages quote Windows
/// paths, and a backslash can't forge or hide anything).
fn escape_text(s: &str, backslash: bool) -> String {
let mut out = String::with_capacity(s.len());
for c in s.chars() {
match c {
'\\' => out.push_str("\\\\"),
'\\' if backslash => out.push_str("\\\\"),
'\n' => out.push_str("\\n"),
'\r' => out.push_str("\\r"),
'\t' => out.push_str("\\t"),
c if c.is_control() || matches!(c, '\u{061c}' | '\u{200e}' | '\u{200f}' | '\u{2028}'..='\u{202e}' | '\u{2066}'..='\u{2069}') => {
out.extend(c.escape_unicode())
}
c if is_unsafe_char(c) => out.extend(c.escape_unicode()),
c => out.push(c),
}
}
out
}
/// A JSON line safe to print: serde escapes control characters below U+0020 but not the other
/// characters [`is_unsafe_char`] names, which a crafted file could put in a report. They become
/// `\uXXXX` escapes (a surrogate pair above U+FFFF), so the line parses to the same value and
/// can't drive or spoof a terminal.
fn json_line_safe(json: &str) -> String {
let mut out = String::with_capacity(json.len());
for c in json.chars() {
if is_unsafe_char(c) {
for unit in c.encode_utf16(&mut [0u16; 2]) {
out.push_str(&format!("\\u{unit:04x}"));
}
} else {
out.push(c);
}
}
out
}
fn parity_markdown(p: &Value) -> String {
let mut s = String::from(
"# WordCraft feature parity\n\nGenerated by `cargo xtask parity` (`wordcraft-cli parity --markdown`): the word-processor feature catalog (`crates/engine/src/catalog.rs`) compared with the live command registry.\n\n",
@@ -786,6 +900,10 @@ fn main() -> ExitCode {
eprintln!("wordcraft-cli: {e}");
ExitCode::FAILURE
}
Err(Failure::Strict(report)) => {
eprintln!("{report}");
ExitCode::from(3)
}
}
}
+16
View File
@@ -564,3 +564,19 @@ fn script_parse_resolves_paths_against_the_base() {
assert_eq!(s.save, vec!["o.docx".to_string()]);
assert_eq!(s.commands.len(), 1);
}
#[test]
fn invisible_characters_are_escaped_and_json_lines_still_parse_back() {
let hidden = "a\u{ad}b\u{200b}c\u{200d}d\u{2060}e\u{2064}f\u{feff}g\u{e0041}h\u{e007f}i\u{85}j\u{202e}k";
let shown = escape_peer(hidden);
assert!(!shown.chars().any(is_unsafe_char), "{shown}");
assert!(shown.contains("\\u{e0041}") && shown.contains("\\u{ad}"), "{shown}");
// Errors keep a Windows path readable.
assert_eq!(escape_text("C:\\dir\\a.docx", false), "C:\\dir\\a.docx");
let value = json!({"part": hidden, "plain": "ünïcode ok"});
let line = json_line_safe(&value.to_string());
assert!(!line.chars().any(is_unsafe_char), "{line}");
assert!(line.contains("\\ud800") || line.contains("\\udb40"), "{line}");
assert_eq!(serde_json::from_str::<Value>(&line).unwrap(), value);
}
+1
View File
@@ -6,4 +6,5 @@ mod help;
#[cfg(all(unix, not(target_os = "macos")))]
mod non_utf8_args;
mod picture_then_typing;
mod strict;
mod text_tracked;
+495
View File
@@ -0,0 +1,495 @@
//! `convert --strict` (#6005): exit 3 and a report when a PDF or PNG would silently lack a
//! character or a picture, nothing written; exit 0 otherwise.
use std::io::Write;
use std::path::{Path, PathBuf};
use std::process::{Command, Output};
use wordcraft_doc::Document;
use wordcraft_fonts::FontDb;
const NS: &str = r#"xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships" xmlns:wp="http://schemas.openxmlformats.org/drawingml/2006/wordprocessingDrawing" xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main" xmlns:pic="http://schemas.openxmlformats.org/drawingml/2006/picture""#;
const REL: &str = "http://schemas.openxmlformats.org/officeDocument/2006/relationships";
const RELS_NS: &str = "http://schemas.openxmlformats.org/package/2006/relationships";
fn dir(name: &str) -> PathBuf {
let d = std::env::temp_dir().join(format!("wc-cli-strict-{name}-{}", std::process::id()));
std::fs::create_dir_all(&d).unwrap();
d
}
fn convert(input: &Path, output: &Path, strict: bool) -> Output {
let mut cmd = Command::new(env!("CARGO_BIN_EXE_wordcraft-cli"));
cmd.arg("convert").arg(input).arg(output);
if strict {
cmd.arg("--strict");
}
cmd.output().unwrap()
}
fn stderr(o: &Output) -> String {
String::from_utf8_lossy(&o.stderr).into_owned()
}
/// The report's JSON line: the last line of stderr.
fn report_json(o: &Output) -> serde_json::Value {
let text = stderr(o);
let last = text.lines().last().unwrap_or_default();
serde_json::from_str(last).unwrap_or_else(|e| panic!("last stderr line is not JSON ({e}): {text}"))
}
fn json_input(d: &Path, text: &str) -> PathBuf {
let path = d.join("in.json");
std::fs::write(&path, serde_json::to_vec(&Document::from_text(text)).unwrap()).unwrap();
path
}
/// Built without craft-fonts, the host's own fonts decide what is drawable: skip, as the layout
/// tests do.
fn without_fonts() -> bool {
let missing = option_env!("CRAFT_FONTS_DIR").is_none_or(str::is_empty);
if missing {
eprintln!("skipped: built without craft-fonts (set CRAFT_FONTS_DIR to a checkout)");
}
missing
}
/// A code point in unassigned plane 4 that no face of the host covers.
fn uncovered() -> char {
let db = FontDb::global();
let primary = wordcraft_fonts::word::resolve(&Document::default().settings.minor_font, false, false).face;
(0x40000u32..0x4FFF0)
.filter_map(char::from_u32)
.find(|c| !primary.covers(*c) && db.fallback_styled(*c, primary.id(), false, false).is_none() && db.fallback_for(*c, primary.id()).is_none())
.expect("the host has faces covering all of plane 4")
}
fn png() -> Vec<u8> {
let mut out = Vec::new();
image::RgbaImage::from_pixel(4, 4, image::Rgba([20, 90, 200, 255]))
.write_to(&mut std::io::Cursor::new(&mut out), image::ImageFormat::Png)
.unwrap();
out
}
fn pic(rid: &str) -> String {
format!(
r#"<w:r><w:drawing><wp:inline><wp:extent cx="508000" cy="508000"/><wp:docPr id="1" name="P"/><a:graphic><a:graphicData uri="http://schemas.openxmlformats.org/drawingml/2006/picture"><pic:pic><pic:nvPicPr><pic:cNvPr id="0" name="p"/><pic:cNvPicPr/></pic:nvPicPr><pic:blipFill><a:blip r:embed="{rid}"/></pic:blipFill><pic:spPr><a:xfrm><a:off x="0" y="0"/><a:ext cx="508000" cy="508000"/></a:xfrm><a:prstGeom prst="rect"/></pic:spPr></pic:pic></a:graphicData></a:graphic></wp:inline></w:drawing></w:r>"#
)
}
/// A DOCX whose header holds a picture of `header_media` (`word/media/image2.bin`).
fn docx_with_header_picture(header_media: &[u8]) -> Vec<u8> {
let doc = format!(
r#"<w:document {NS}><w:body><w:p><w:r><w:t>text</w:t></w:r></w:p><w:sectPr><w:headerReference w:type="default" r:id="rIdH"/></w:sectPr></w:body></w:document>"#
);
let drels = format!(r#"<Relationships xmlns="{RELS_NS}"><Relationship Id="rIdH" Type="{REL}/header" Target="header2.xml"/></Relationships>"#);
let hrels = format!(r#"<Relationships xmlns="{RELS_NS}"><Relationship Id="rIdI" Type="{REL}/image" Target="media/image2.bin"/></Relationships>"#);
let hdr = format!(r#"<w:hdr {NS}><w:p>{}</w:p></w:hdr>"#, pic("rIdI"));
package(
&doc,
&drels,
&[("word/header2.xml", hdr.as_bytes()), ("word/_rels/header2.xml.rels", hrels.as_bytes()), ("word/media/image2.bin", header_media)],
)
}
/// A DOCX whose body picture points outside the file, at `target`.
fn docx_with_external_picture(target: &str) -> Vec<u8> {
let doc = format!(r#"<w:document {NS}><w:body><w:p>{}<w:r><w:t>text</w:t></w:r></w:p></w:body></w:document>"#, pic("rIdX"));
let drels = format!(
r#"<Relationships xmlns="{RELS_NS}"><Relationship Id="rIdX" Type="{REL}/image" Target="{target}" TargetMode="External"/></Relationships>"#
);
package(&doc, &drels, &[])
}
fn package(document: &str, document_rels: &str, extra: &[(&str, &[u8])]) -> Vec<u8> {
let rels = format!(
r#"<Relationships xmlns="{RELS_NS}"><Relationship Id="rId1" Type="{REL}/officeDocument" Target="word/document.xml"/></Relationships>"#
);
let types = r#"<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types"><Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/><Default Extension="png" ContentType="image/png"/><Default Extension="bin" ContentType="application/octet-stream"/><Override PartName="/word/document.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml"/></Types>"#;
let mut zip = zip::ZipWriter::new(std::io::Cursor::new(Vec::new()));
let mut parts: Vec<(&str, &[u8])> = vec![
("[Content_Types].xml", types.as_bytes()),
("_rels/.rels", rels.as_bytes()),
("word/document.xml", document.as_bytes()),
("word/_rels/document.xml.rels", document_rels.as_bytes()),
];
parts.extend_from_slice(extra);
for (name, bytes) in parts {
zip.start_file(name, zip::write::SimpleFileOptions::default()).unwrap();
zip.write_all(bytes).unwrap();
}
zip.finish().unwrap().into_inner()
}
#[test]
fn multilingual_names_pass_strict() {
if without_fonts() {
return;
}
let names = "Łukasz Żmijewski\nGheorghe Ștefănescu\nKrejčí\nŞahın Gül\nLlŷr Grŵn\n王小明\n김민수\nสมชาย ใจดี\nराम शर्मा";
let d = dir("names");
let input = json_input(&d, names);
for out in ["names.pdf", "names.png"] {
let out = d.join(out);
let r = convert(&input, &out, true);
assert_eq!(r.status.code(), Some(0), "{}", stderr(&r));
assert!(out.exists());
}
std::fs::remove_dir_all(d).unwrap();
}
#[test]
fn an_uncovered_code_point_fails_strict_with_a_report_and_no_file() {
let c = uncovered();
let cp = format!("U+{:04X}", c as u32);
let d = dir("uncovered");
let input = json_input(&d, &format!("a{c}b{c}{c}"));
for out in ["out.pdf", "out.png"] {
let out = d.join(out);
let r = convert(&input, &out, true);
assert_eq!(r.status.code(), Some(3), "{}", stderr(&r));
let text = stderr(&r);
assert!(text.contains(&format!("wordcraft-cli: strict: 1 character cannot be drawn: {cp} (×3)")), "{text}");
let json = report_json(&r);
assert_eq!(json["schema"], 1);
assert_eq!(json["strict"], "failed");
assert_eq!(json["missing_glyphs"][0]["cp"], cp.as_str());
assert_eq!(json["missing_glyphs"][0]["count"], 3);
assert!(!out.exists(), "{} was written", out.display());
eprintln!("{text}");
// Without --strict the same input converts, as before.
let r = convert(&input, &out, false);
assert_eq!(r.status.code(), Some(0), "{}", stderr(&r));
assert!(out.exists());
std::fs::remove_file(&out).unwrap();
}
std::fs::remove_dir_all(d).unwrap();
}
#[test]
fn a_real_png_in_a_header_passes_strict() {
let d = dir("header-png");
let input = d.join("in.docx");
std::fs::write(&input, docx_with_header_picture(&png())).unwrap();
for out in ["out.pdf", "out.png"] {
let out = d.join(out);
let r = convert(&input, &out, true);
assert_eq!(r.status.code(), Some(0), "{}", stderr(&r));
assert!(out.exists());
}
std::fs::remove_dir_all(d).unwrap();
}
#[test]
fn a_corrupt_header_picture_fails_strict_naming_the_part_and_the_header() {
let d = dir("header-corrupt");
let input = d.join("in.docx");
std::fs::write(&input, docx_with_header_picture(b"corrupt bytes, not a picture")).unwrap();
for out in ["out.pdf", "out.png"] {
let out = d.join(out);
let r = convert(&input, &out, true);
assert_eq!(r.status.code(), Some(3), "{}", stderr(&r));
let text = stderr(&r);
assert!(text.contains("word/media/image2.bin (header2.xml): unsupported format"), "{text}");
let json = report_json(&r);
assert_eq!(json["strict"], "failed");
assert_eq!(
json["missing_pictures"][0],
serde_json::json!({"part": "word/media/image2.bin", "source_part": "word/header2.xml", "reason": "unsupported_format"})
);
assert!(!out.exists());
eprintln!("{text}");
}
// Without --strict the grey box is drawn, as before.
let out = d.join("lenient.pdf");
assert_eq!(convert(&input, &out, false).status.code(), Some(0));
assert!(out.exists());
std::fs::remove_dir_all(d).unwrap();
}
#[test]
fn an_external_picture_target_is_never_fetched_and_is_reported_under_strict() {
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
listener.set_nonblocking(true).unwrap();
let target = format!("http://{}/canary.png", listener.local_addr().unwrap());
let d = dir("external");
let input = d.join("in.docx");
std::fs::write(&input, docx_with_external_picture(&target)).unwrap();
let lenient = convert(&input, &d.join("lenient.pdf"), false);
assert_eq!(lenient.status.code(), Some(0), "{}", stderr(&lenient));
let out = d.join("strict.pdf");
let strict = convert(&input, &out, true);
assert_eq!(strict.status.code(), Some(3), "{}", stderr(&strict));
let json = report_json(&strict);
assert_eq!(json["missing_pictures"][0]["reason"], "external_target");
assert!(stderr(&strict).contains("external target"));
assert!(!out.exists());
match listener.accept() {
Err(e) if e.kind() == std::io::ErrorKind::WouldBlock => {}
Ok((_, from)) => panic!("the converter connected to the external target from {from}"),
Err(e) => panic!("{e}"),
}
std::fs::remove_dir_all(d).unwrap();
}
#[test]
fn bad_input_exits_1_with_the_usual_message_with_and_without_strict() {
let d = dir("bad-input");
let good = docx_with_header_picture(&png());
let truncated = d.join("truncated.docx");
std::fs::write(&truncated, &good[..good.len() / 2]).unwrap();
let not_zip = d.join("not-zip.docx");
std::fs::write(&not_zip, b"this is plainly not a zip archive").unwrap();
for input in [&truncated, &not_zip] {
let plain = convert(input, &d.join("a.pdf"), false);
let strict = convert(input, &d.join("b.pdf"), true);
assert_eq!(plain.status.code(), Some(1), "{}", stderr(&plain));
assert_eq!(strict.status.code(), Some(1), "{}", stderr(&strict));
assert_eq!(stderr(&plain), stderr(&strict));
assert!(stderr(&plain).starts_with("wordcraft-cli: "), "{}", stderr(&plain));
assert!(!d.join("a.pdf").exists() && !d.join("b.pdf").exists());
}
std::fs::remove_dir_all(d).unwrap();
}
#[test]
fn strict_with_another_output_type_exits_1_before_the_input_is_read() {
let d = dir("wrong-output");
let missing = d.join("does-not-exist.docx");
for out in ["out.docx", "out.txt", "out.odt"] {
let r = convert(&missing, &d.join(out), true);
assert_eq!(r.status.code(), Some(1));
let text = stderr(&r);
assert!(text.contains("--strict needs a .pdf or .png output"), "{text}");
assert!(!d.join(out).exists());
}
// Without --strict the same command fails on the missing input.
let r = convert(&missing, &d.join("out.docx"), false);
assert_eq!(r.status.code(), Some(1));
assert!(!stderr(&r).contains("--strict"));
std::fs::remove_dir_all(d).unwrap();
}
fn render(input: &Path, output: &Path, args: &[&str]) -> Output {
Command::new(env!("CARGO_BIN_EXE_wordcraft-cli")).arg("render").arg(input).arg(output).args(args).output().unwrap()
}
/// Page 1 has `a{c}b`, page 2 plain text.
fn two_page_input(d: &Path, text: &str) -> PathBuf {
let mut doc = Document::from_text(&format!("{text}\nSecond page"));
doc.para_mut(wordcraft_doc::StoryRef::Body, &wordcraft_doc::Path::top(1)).unwrap().props.page_break_before = Some(true);
let path = d.join("two.json");
std::fs::write(&path, serde_json::to_vec(&doc).unwrap()).unwrap();
path
}
#[test]
fn render_strict_refuses_an_uncovered_character_on_the_page_with_no_stdout_or_file() {
let c = uncovered();
let cp = format!("U+{:04X}", c as u32);
let d = dir("render-glyph");
let input = two_page_input(&d, &format!("a{c}b"));
let out = d.join("p1.png");
let r = render(&input, &out, &["--strict"]);
assert_eq!(r.status.code(), Some(3), "{}", stderr(&r));
assert!(r.stdout.is_empty(), "{}", String::from_utf8_lossy(&r.stdout));
assert!(!out.exists());
let text = stderr(&r);
assert!(text.contains(&format!("wordcraft-cli: strict: 1 character cannot be drawn: {cp} (×1)")), "{text}");
assert_eq!(report_json(&r)["missing_glyphs"][0]["cp"], cp.as_str());
eprintln!("{text}");
// The character is not on page 2, and --scale does not matter to the check.
let out2 = d.join("p2.png");
let r = render(&input, &out2, &["--page", "2", "--strict", "--scale", "0.5"]);
assert_eq!(r.status.code(), Some(0), "{}", stderr(&r));
assert!(out2.exists());
assert!(String::from_utf8_lossy(&r.stdout).contains("\"path\""));
// And the same page 1 without --strict renders, with the usual stdout.
let out3 = d.join("lenient.png");
let r = render(&input, &out3, &[]);
assert_eq!(r.status.code(), Some(0), "{}", stderr(&r));
assert!(out3.exists());
let json: serde_json::Value = serde_json::from_slice(&r.stdout).unwrap();
assert_eq!(json["path"], out3.to_string_lossy().as_ref());
assert!(json["width"].as_u64().is_some() && json["bytes"].as_u64().is_some());
std::fs::remove_dir_all(d).unwrap();
}
#[test]
fn render_strict_refuses_a_corrupt_header_picture_and_bad_input_still_exits_1() {
let d = dir("render-pic");
let input = d.join("in.docx");
std::fs::write(&input, docx_with_header_picture(b"corrupt bytes, not a picture")).unwrap();
let out = d.join("out.png");
let r = render(&input, &out, &["--strict"]);
assert_eq!(r.status.code(), Some(3), "{}", stderr(&r));
assert!(r.stdout.is_empty());
assert!(stderr(&r).contains("word/media/image2.bin (header2.xml)"), "{}", stderr(&r));
assert!(!out.exists());
let bad = d.join("bad.docx");
std::fs::write(&bad, b"this is plainly not a zip archive").unwrap();
let (plain, strict) = (render(&bad, &out, &[]), render(&bad, &out, &["--strict"]));
assert_eq!((plain.status.code(), strict.status.code()), (Some(1), Some(1)));
assert_eq!(stderr(&plain), stderr(&strict));
std::fs::remove_dir_all(d).unwrap();
}
#[test]
fn a_picture_target_with_a_newline_and_a_bidi_override_cannot_forge_a_report_line() {
let d = dir("escape");
let input = d.join("in.docx");
// XML character references put a newline and U+202E in the target.
std::fs::write(&input, docx_with_external_picture("http://example.invalid/a&#10;wordcraft-cli: strict: forged&#x202E;gnp.png")).unwrap();
let out = d.join("out.pdf");
let r = convert(&input, &out, true);
assert_eq!(r.status.code(), Some(3), "{}", stderr(&r));
let text = stderr(&r);
let lines: Vec<&str> = text.lines().collect();
// One human line for the picture, then the JSON line.
assert_eq!(lines.len(), 2, "{text}");
assert!(lines[0].starts_with("wordcraft-cli: strict: 1 picture(s) left out: "), "{text}");
assert!(lines[0].contains("\\nwordcraft-cli: strict: forged\\u{202e}gnp.png"), "{text}");
assert!(!text.contains('\u{202e}'), "the raw bidi override reached the terminal");
// The JSON keeps the raw value, escaped by the JSON encoder.
let json = report_json(&r);
let part = json["missing_pictures"][0]["part"].as_str().unwrap();
assert!(part.contains('\n') && part.contains('\u{202e}'), "{part:?}");
std::fs::remove_dir_all(d).unwrap();
}
#[test]
fn run_with_strict_cannot_forge_an_error_line_either() {
let d = dir("run-escape");
let input = d.join("in.docx");
std::fs::write(&input, docx_with_external_picture("http://example.invalid/a&#10;wordcraft-cli: strict: forged&#x1B;[2J&#x202E;gnp.png")).unwrap();
let out = d.join("out.pdf");
let cmd = format!("file.exportPdf={}", serde_json::json!({"path": out, "strict": true}));
let r = Command::new(env!("CARGO_BIN_EXE_wordcraft-cli")).arg("run").arg("--file").arg(&input).arg("--cmd").arg(cmd).output().unwrap();
assert_eq!(r.status.code(), Some(1));
let text = stderr(&r);
assert_eq!(text.lines().count(), 1, "{text}");
assert!(text.contains("\\u{1b}") || text.contains("\\u{1b}[2J"), "{text}");
assert!(!text.contains('\u{1b}') && !text.contains('\u{202e}'), "{text:?}");
assert!(!out.exists());
std::fs::remove_dir_all(d).unwrap();
}
#[test]
fn usage_errors_keep_their_line_breaks() {
// Errors go to stderr with the usage after a blank line (`missing argument`, `unknown command`).
for args in [&["frobnicate"][..], &["convert"][..]] {
let r = Command::new(env!("CARGO_BIN_EXE_wordcraft-cli")).args(args).output().unwrap();
let text = stderr(&r);
assert_eq!(r.status.code(), Some(1), "{args:?}: {text}");
assert!(text.lines().count() > 3, "{args:?}: {text}");
assert!(text.lines().any(|l| l == "USAGE:"), "{args:?}: {text}");
assert!(!text.contains("\\n"), "{args:?}: {text}");
}
// No arguments at all prints the usage as help, on stdout.
let r = Command::new(env!("CARGO_BIN_EXE_wordcraft-cli")).output().unwrap();
let all = format!("{}{}", String::from_utf8_lossy(&r.stdout), stderr(&r));
assert!(all.lines().count() > 3 && all.lines().any(|l| l == "USAGE:"), "{all}");
}
/// A minimal EMF (88-byte header, 100 mm square frame) with one EMR_EXTTEXTOUTW record showing
/// `text` in the default font.
fn emf_with_text(text: &str) -> Vec<u8> {
let units: Vec<u16> = text.encode_utf16().collect();
let mut rec = vec![0u8; 76];
rec[0..4].copy_from_slice(&84u32.to_le_bytes());
rec[36..40].copy_from_slice(&10i32.to_le_bytes());
rec[40..44].copy_from_slice(&40i32.to_le_bytes());
rec[44..48].copy_from_slice(&(units.len() as u32).to_le_bytes());
rec[48..52].copy_from_slice(&76u32.to_le_bytes());
rec.extend(units.iter().flat_map(|u| u.to_le_bytes()));
while !rec.len().is_multiple_of(4) {
rec.push(0);
}
let len = rec.len() as u32;
rec[4..8].copy_from_slice(&len.to_le_bytes());
let eof: Vec<u8> = [14u32.to_le_bytes(), 20u32.to_le_bytes(), [0; 4], [0; 4], 20u32.to_le_bytes()].concat();
let total = 88 + rec.len() + eof.len();
let mut h = vec![0u8; 88];
h[0..4].copy_from_slice(&1u32.to_le_bytes());
h[4..8].copy_from_slice(&88u32.to_le_bytes());
h[8..24].copy_from_slice(&[0i32, 0, 100, 100].iter().flat_map(|v| v.to_le_bytes()).collect::<Vec<u8>>());
h[24..40].copy_from_slice(&[0i32, 0, 10000, 10000].iter().flat_map(|v| v.to_le_bytes()).collect::<Vec<u8>>());
h[40..44].copy_from_slice(&0x464D_4520u32.to_le_bytes());
h[44..48].copy_from_slice(&0x0001_0000u32.to_le_bytes());
h[48..52].copy_from_slice(&(total as u32).to_le_bytes());
h[52..56].copy_from_slice(&2u32.to_le_bytes());
h[56..58].copy_from_slice(&1u16.to_le_bytes());
h[72..76].copy_from_slice(&1000i32.to_le_bytes());
h[76..80].copy_from_slice(&1000i32.to_le_bytes());
h[80..84].copy_from_slice(&250i32.to_le_bytes());
h[84..88].copy_from_slice(&250i32.to_le_bytes());
[h, rec, eof].concat()
}
/// The text a metafile picture draws is checked like the page's own: a character no face covers
/// refuses the export from `convert` and `render`, naming the code point, and nothing is written.
#[test]
fn a_character_in_a_metafile_pictures_text_fails_strict_like_the_pages_own() {
if without_fonts() {
return;
}
let c = uncovered();
let cp = format!("U+{:04X}", c as u32);
let d = dir("emf-text");
let emf = emf_with_text(&format!("ab{c}"));
let input = d.join("in.docx");
std::fs::write(&input, docx_with_header_picture(&emf)).unwrap();
for out in ["out.pdf", "out.png"] {
let out = d.join(out);
let r = convert(&input, &out, true);
assert_eq!(r.status.code(), Some(3), "{}", stderr(&r));
assert_eq!(report_json(&r)["missing_glyphs"][0]["cp"], cp.as_str(), "{}", stderr(&r));
assert!(!out.exists(), "{} was written", out.display());
// Without --strict it converts, the character a hollow box.
let r = convert(&input, &out, false);
assert_eq!(r.status.code(), Some(0), "{}", stderr(&r));
std::fs::remove_file(&out).unwrap();
}
let out = d.join("p1.png");
let r = render(&input, &out, &["--strict"]);
assert_eq!(r.status.code(), Some(3), "{}", stderr(&r));
assert!(r.stdout.is_empty() && !out.exists());
assert_eq!(report_json(&r)["missing_glyphs"][0]["cp"], cp.as_str());
// Text the faces cover passes.
std::fs::write(&input, docx_with_header_picture(&emf_with_text("abc"))).unwrap();
let r = convert(&input, &d.join("ok.pdf"), true);
assert_eq!(r.status.code(), Some(0), "{}", stderr(&r));
std::fs::remove_dir_all(d).unwrap();
}
/// The page raster leaves out a metafile page background (it only draws raster ones), so a PNG
/// would silently lack it: strict refuses that output, and a PDF, which draws it as vector, passes.
#[test]
fn a_metafile_page_background_fails_strict_for_png_but_not_for_pdf() {
if without_fonts() {
return;
}
let d = dir("emf-background");
let mut doc = Document::from_text("text");
doc.media.insert("bg.emf".into(), std::sync::Arc::new(emf_with_text("abc")));
doc.settings.page_picture = Some(wordcraft_doc::PagePicture { media: "bg.emf".into(), fill_type: "frame".into(), title: String::new() });
let input = d.join("in.json");
std::fs::write(&input, serde_json::to_vec(&doc).unwrap()).unwrap();
let png = d.join("out.png");
let r = convert(&input, &png, true);
assert_eq!(r.status.code(), Some(3), "{}", stderr(&r));
let json = report_json(&r);
assert_eq!(json["missing_pictures"][0]["part"], "bg.emf", "{}", stderr(&r));
assert_eq!(json["missing_pictures"][0]["source_part"], "background");
assert_eq!(json["missing_pictures"][0]["reason"], "unsupported_format");
assert!(!png.exists());
let pdf = d.join("out.pdf");
let r = convert(&input, &pdf, true);
assert_eq!(r.status.code(), Some(0), "{}", stderr(&r));
assert!(pdf.exists());
std::fs::remove_dir_all(d).unwrap();
}
+3
View File
@@ -204,3 +204,6 @@ real_name = "Helen Kwok"
[people.ashokram-si]
real_name = "Ashok Ram"
[people.benandez]
sync_github_name = true
+3
View File
@@ -77,6 +77,7 @@ impl Document {
glossary: Vec::new(),
media_index: MediaIndex::default(),
kept: Default::default(),
missing_pictures: Default::default(),
}
}
@@ -154,6 +155,7 @@ impl Document {
glossary,
media_index: _,
kept: _,
missing_pictures: _,
} = self;
let Document {
body: other_body,
@@ -174,6 +176,7 @@ impl Document {
glossary: other_glossary,
media_index: _,
kept: _,
missing_pictures: _,
} = other;
same_blocks(body, other_body)
&& same_parts(parts, other_parts)
+6
View File
@@ -75,6 +75,7 @@ mod block;
mod compare;
mod document;
mod error;
mod missing;
mod parts;
mod position;
mod settings;
@@ -84,6 +85,7 @@ pub use block::{Block, Blocks, BoxBudget, MAX_TEXT_BOX_DEPTH};
pub use compare::para_block;
use document::MediaIndex;
pub use error::{DocError, Result};
pub use missing::{MAX_MISSING_PICTURES, MissingPicture, MissingPictureReason, MissingPictures, picture_hash};
pub use parts::{MAX_LINKED_BOXES, MAX_TEXT_BOX_INSET, NoteSeparators, Part, PartKind, PermRange, TEXT_BOX_INSET, TextBody, free_id};
pub use position::{Path, Pos, StoryRef};
pub use settings::{
@@ -138,6 +140,10 @@ pub struct Document {
/// [`kept`]); never saved, ignored by `==`.
#[serde(skip)]
pub kept: kept::Kept,
/// Pictures a reader dropped because their target is missing, external or link-only (strict
/// export reports them); never saved, never drawn, ignored by `==`.
#[serde(skip)]
pub missing_pictures: MissingPictures,
}
#[cfg(test)]
+143
View File
@@ -0,0 +1,143 @@
//! Pictures a reader could not bring in or an exporter could not draw, for strict export (#6005).
//! Kept beside the document while it is open; never saved, never drawn, and not part of the
//! document's content (`==` ignores it).
use serde::{Deserialize, Serialize};
/// Most pictures remembered per document; past it [`MissingPictures::truncated`] is set.
pub const MAX_MISSING_PICTURES: usize = 256;
/// Most characters of a picture's part or story name a record keeps (names come from the file).
pub const MAX_PICTURE_NAME_CHARS: usize = 1024;
/// Why a picture is not in the document or not drawn.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub enum MissingPictureReason {
/// The relationship names a part the file does not contain, or no relationship at all.
MissingPart,
/// The target is outside the file (`TargetMode="External"`); it is never fetched.
ExternalTarget,
/// The picture only links to a file (`r:link`) and holds no embedded copy.
LinkOnly,
/// The bytes are a picture format we know but they do not decode.
Undecodable,
/// The bytes are not a picture format we can draw.
UnsupportedFormat,
}
impl MissingPictureReason {
/// The words used in reports (`"missing part"`, `"external target"`, `"link only"`,
/// `"undecodable"`, `"unsupported format"`).
pub fn label(self) -> &'static str {
match self {
Self::MissingPart => "missing part",
Self::ExternalTarget => "external target",
Self::LinkOnly => "link only",
Self::Undecodable => "undecodable",
Self::UnsupportedFormat => "unsupported format",
}
}
/// The reason a [`code`](Self::code) names.
pub fn from_code(code: &str) -> Option<Self> {
[Self::MissingPart, Self::ExternalTarget, Self::LinkOnly, Self::Undecodable, Self::UnsupportedFormat].into_iter().find(|r| r.code() == code)
}
/// The stable snake_case code the JSON report carries (`"missing_part"`, `"external_target"`,
/// `"link_only"`, `"undecodable"`, `"unsupported_format"`); never reworded.
pub fn code(self) -> &'static str {
match self {
Self::MissingPart => "missing_part",
Self::ExternalTarget => "external_target",
Self::LinkOnly => "link_only",
Self::Undecodable => "undecodable",
Self::UnsupportedFormat => "unsupported_format",
}
}
}
/// One picture that is missing.
#[derive(Clone, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct MissingPicture {
/// The target as written, or the part name it resolved to (a media key for a decoded one).
pub part: String,
/// The story part it is in (`word/document.xml`, `word/header2.xml`), as the source file
/// names it; empty when not known.
pub story: String,
pub reason: MissingPictureReason,
}
/// A capped list of [`MissingPicture`]s, with the source names a report needs to say where a
/// picture is (a reader fills them; a document made or opened another way leaves them empty).
#[derive(Clone, Debug, Default)]
pub struct MissingPictures {
pub list: Vec<MissingPicture>,
/// More than [`MAX_MISSING_PICTURES`] were found; the rest are not listed.
pub truncated: bool,
/// Pictures found, those past the cap included (saturating). Each distinct one counts once.
pub total: u64,
/// The main story's part name in the source file (`word/document.xml`).
pub main: String,
/// A header, footer, note or comment story's part id ([`crate::Document::parts`]) to the
/// part it was read from (`word/header2.xml`). Ids freed and reused after opening may name
/// the old part.
pub part_names: std::collections::BTreeMap<u32, std::sync::Arc<str>>,
/// A [`crate::Document::media`] key to the part its bytes were read from
/// (`word/media/image2.bin`).
pub media_parts: std::collections::BTreeMap<String, String>,
/// What `list` holds, for an O(1) test of "already there" (a hostile file has many).
seen: std::collections::HashSet<MissingPicture>,
/// Hashes of the distinct pictures past the cap (8 bytes each), so the total counts them once.
unlisted: std::collections::HashSet<u64>,
/// The chunk parts whose unlisted pictures [`Self::add_unlisted`] has counted, so a chunk
/// imported twice counts them once.
counted_chunks: std::collections::HashSet<String>,
}
impl MissingPictures {
/// Add `n` pictures of the chunk part `chunk` that its reader counted but did not list. The
/// same chunk part counts once however often it is imported (its listed pictures dedupe
/// through [`Self::push`]).
pub fn add_unlisted(&mut self, chunk: &str, n: u64) {
if n > 0 && self.counted_chunks.insert(chunk.to_string()) {
self.total = self.total.saturating_add(n);
}
}
/// Add one (the same part, story and reason once).
pub fn push(&mut self, mut item: MissingPicture) {
for name in [&mut item.part, &mut item.story] {
if name.len() > MAX_PICTURE_NAME_CHARS {
*name = name.chars().take(MAX_PICTURE_NAME_CHARS).collect();
}
}
if self.seen.contains(&item) {
return;
}
if self.list.len() >= MAX_MISSING_PICTURES {
self.truncated = true;
if self.unlisted.insert(picture_hash(&item)) {
self.total = self.total.saturating_add(1);
}
return;
}
self.total = self.total.saturating_add(1);
self.seen.insert(item.clone());
self.list.push(item);
}
}
/// A 64-bit hash of a record, to tell the ones past a cap apart without keeping them.
pub fn picture_hash(item: &MissingPicture) -> u64 {
use std::hash::{Hash, Hasher};
let mut h = std::collections::hash_map::DefaultHasher::new();
item.hash(&mut h);
h.finish()
}
impl PartialEq for MissingPictures {
/// Not part of the document's content.
fn eq(&self, _: &Self) -> bool {
true
}
}
+28
View File
@@ -786,3 +786,31 @@ fn lone_table_open_markers_mixed_tables_and_paired() {
assert!(res1.controls.open.is_empty());
assert_eq!(res1.controls.close, 0);
}
#[test]
fn missing_picture_names_are_capped_at_a_char_boundary() {
let mut d = Document::new();
let long: String = "é".repeat(5000);
d.missing_pictures.push(MissingPicture { part: long.clone(), story: long, reason: MissingPictureReason::MissingPart });
let item = &d.missing_pictures.list[0];
assert_eq!((item.part.chars().count(), item.story.chars().count()), (1024, 1024));
}
#[test]
fn missing_pictures_are_capped_deduplicated_and_not_content() {
let mut d = Document::new();
let item = |n: usize| MissingPicture { part: format!("p{n}"), story: "word/document.xml".into(), reason: MissingPictureReason::MissingPart };
d.missing_pictures.push(item(0));
d.missing_pictures.push(item(0));
assert_eq!(d.missing_pictures.list.len(), 1);
for n in 1..MAX_MISSING_PICTURES + 3 {
d.missing_pictures.push(item(n));
}
assert_eq!(d.missing_pictures.list.len(), MAX_MISSING_PICTURES);
assert!(d.missing_pictures.truncated);
let e = Document::new();
assert_eq!(d, e);
assert!(d.same_content(&e));
assert!(!serde_json::to_string(&d).unwrap().contains("p0"));
assert_eq!(MissingPictureReason::ExternalTarget.label(), "external target");
}
+10 -1
View File
@@ -27,7 +27,8 @@ use wordcraft_doc::props::VAlign;
use wordcraft_doc::section::NumFormat;
use wordcraft_doc::styles::{PartFormat, Style, StyleKind, StyleSheet, TablePart, TableStyleParts};
use wordcraft_doc::{
Block, Blocks, Comment, Document, InlineObject, PagePicture, Paragraph, PartKind, Revision, RevisionKind, Zoom, ZoomFit, para_block,
Block, Blocks, Comment, Document, InlineObject, MissingPicture, MissingPictureReason, PagePicture, Paragraph, PartKind, Revision, RevisionKind,
Zoom, ZoomFit, para_block,
};
use crate::DocxError;
@@ -103,6 +104,8 @@ pub(crate) struct Reader<'p> {
pub shape_ids: bool,
/// The main document part's name.
main: String,
/// The part being read (`word/header2.xml`), for naming where a dropped picture was.
story: Arc<str>,
/// The package's content types, read when first needed.
content_types: Option<ContentTypes>,
/// Parts kept for charts, diagrams and OLE objects (see `embed`).
@@ -183,6 +186,7 @@ fn read_opened_in(pkg: &Package, chunks: ChunkState) -> Result<(Document, usize,
glued: HashSet::new(),
shape_ids: false,
main: main.clone(),
story: main.as_str().into(),
content_types: None,
embedded: EmbeddedManifest::default(),
txbx_links: Vec::new(),
@@ -190,6 +194,7 @@ fn read_opened_in(pkg: &Package, chunks: ChunkState) -> Result<(Document, usize,
group_fill: None,
chunks,
};
r.doc.missing_pictures.main = main.clone();
r.pc.major_font = r.doc.settings.major_font.clone();
r.pc.minor_font = r.doc.settings.minor_font.clone();
let part = |t: &str| rels.by_type(t).filter(|x| !x.external).map(|x| x.target.clone());
@@ -230,14 +235,18 @@ fn read_opened_in(pkg: &Package, chunks: ChunkState) -> Result<(Document, usize,
let ex = part(rt::COMMENTS_EX);
let ids = part(rt::COMMENTS_IDS);
let cex = part(rt::COMMENTS_EXTENSIBLE);
r.story = p.as_str().into();
lenient("comments", r.read_comments(&p, ex.as_deref(), ids.as_deref(), cex.as_deref()));
}
if let Some(p) = part(rt::FOOTNOTES) {
r.story = p.as_str().into();
lenient("footnotes", r.read_notes(&p, true));
}
if let Some(p) = part(rt::ENDNOTES) {
r.story = p.as_str().into();
lenient("endnotes", r.read_notes(&p, false));
}
r.story = main.as_str().into();
// Bibliography sources live in a custom XML data store (`b:Sources`), related from the
// package root or the main part; Word's own files sometimes leave the relationship out, so
// the conventional `customXml/itemN.xml` names are tried too. Each part is read at most once.
+15
View File
@@ -80,6 +80,21 @@ impl Reader<'_> {
// `w:matchSrc`: a Word document's content keeps its own styles (other formats' formatting
// is their own already).
let keep = is_docx && e.child("w:altChunkPr").and_then(|p| p.child("w:matchSrc")).is_some_and(crate::units::on_off);
// The pictures the chunk's own reader dropped are the file's too (strict export, #6005),
// each named by the chunk and its part in it (`word/afchunk1.docx > word/header1.xml`).
// A chunk in another format brings none: those importers record nothing yet (#6007).
let nested = std::mem::take(&mut doc.missing_pictures);
let list_len = nested.list.len() as u64;
for mut item in nested.list {
let inner = if item.story.is_empty() { nested.main.as_str() } else { item.story.as_str() };
item.story = format!("{part} > {inner}");
self.doc.missing_pictures.push(item);
}
// The nested records past its list are the file's too; their stories carry the chunk's
// part, so none is one the outer list already counted.
let unlisted = nested.total.saturating_sub(list_len);
self.doc.missing_pictures.add_unlisted(&part, unlisted);
self.doc.missing_pictures.truncated |= nested.truncated;
let mut blocks = self.doc.adopt_blocks(doc, keep, &mut self.next_perm);
// Bookmarks and comment marks read before the chunk stay before it.
self.flush_pending_before(sc, out, &mut blocks);
+3
View File
@@ -53,7 +53,10 @@ impl Reader<'_> {
// Without content types (a converted package) the root element alone decides.
if let Some(root) = root.filter(|r| r.name == "w:glossaryDocument" && ct.as_deref().is_none_or(|t| t == GLOSSARY_CONTENT_TYPE)) {
let rels = self.pkg.rels(&rel.target);
// Pictures in a building block belong to the glossary part, not the main story.
let outer = std::mem::replace(&mut self.story, rel.target.as_str().into());
entries = self.doc_parts(&root, &rels);
self.story = outer;
}
}
self.doc.glossary = entries;
+2
View File
@@ -117,6 +117,7 @@ impl Reader<'_> {
// Reserve the id first: the note's own reference mark points at it.
let (pk, kind) = if foot { (PartKind::Footnote, NoteKind::Footnote) } else { (PartKind::Endnote, NoteKind::Endnote) };
let id = self.doc.add_part(pk, Blocks::new());
self.doc.missing_pictures.part_names.insert(id, self.story.clone());
let mut sc = StoryCtx::default();
let mut blocks = Blocks::new();
self.current_note = Some((kind, id));
@@ -214,6 +215,7 @@ impl Reader<'_> {
self.read_blocks(&mut sc, c, &rels, &mut blocks, self.chunks.depth);
self.flush_pending(&mut sc, &mut blocks);
let part = self.doc.add_part(PartKind::Comment, blocks);
self.doc.missing_pictures.part_names.insert(part, self.story.clone());
if let Some(pid) = c.children("w:p").last().and_then(|p| p.attr("w14:paraId")) {
by_para.insert(pid.to_string(), id);
}
+24 -4
View File
@@ -1,6 +1,6 @@
use super::{
Arc, Blocks, ContentTypes, DocxError, El, MAX_FONT_NAME, MAX_HF_DEPTH, MAX_PARTS, MAX_TABLE_FONTS, MAX_VBA_RELATED, PartKind, Reader, Rels,
Revision, RevisionKind, Rgb, StoryCtx, VBA_PROJECT_PART, VBA_RELATED, props, shape_style,
Arc, Blocks, ContentTypes, DocxError, El, MAX_FONT_NAME, MAX_HF_DEPTH, MAX_PARTS, MAX_TABLE_FONTS, MAX_VBA_RELATED, MissingPicture,
MissingPictureReason, PartKind, Reader, Rels, Revision, RevisionKind, Rgb, StoryCtx, VBA_PROJECT_PART, VBA_RELATED, props, shape_style,
};
impl Reader<'_> {
@@ -91,14 +91,22 @@ impl Reader<'_> {
/// Media key for an image relationship (loads the bytes once).
pub fn media_for(&mut self, rels: &Rels, id: &str) -> Option<String> {
let rel = rels.by_id(id)?;
let Some(rel) = rels.by_id(id) else {
self.note_missing_picture(id, MissingPictureReason::MissingPart);
return None;
};
if rel.external {
// Never fetched: a picture whose target is outside the file is dropped (and reported).
self.note_missing_picture(&rel.target, MissingPictureReason::ExternalTarget);
return None;
}
if let Some(k) = self.media_by_path.get(&rel.target) {
return Some(k.clone());
}
let bytes = self.pkg.get_shared(&rel.target)?;
let Some(bytes) = self.pkg.get_shared(&rel.target) else {
self.note_missing_picture(&rel.target, MissingPictureReason::MissingPart);
return None;
};
let base: String =
rel.target.rsplit('/').next().unwrap_or("image").chars().filter(|c| c.is_ascii_alphanumeric() || matches!(c, '.' | '_' | '-')).collect();
let base = if base.is_empty() || base.starts_with('.') { format!("image{base}") } else { base };
@@ -112,10 +120,18 @@ impl Reader<'_> {
};
}
self.doc.media.insert(key.clone(), bytes.clone());
self.doc.missing_pictures.media_parts.insert(key.clone(), rel.target.clone());
self.media_by_path.insert(rel.target.clone(), key.clone());
Some(key)
}
/// Remember a picture that was dropped at open, in the story being read.
pub(super) fn note_missing_picture(&mut self, part: &str, reason: MissingPictureReason) {
let part: String = part.chars().take(1024).collect();
let story = self.story.to_string();
self.doc.missing_pictures.push(MissingPicture { part, story, reason });
}
/// A chart or diagram part, parsed once however often it's referenced (`None` when unreadable).
pub(super) fn graphic_part(&mut self, path: &str) -> Option<Arc<El>> {
if let Some(p) = self.graphic_parts.get(path) {
@@ -137,6 +153,8 @@ impl Reader<'_> {
let kind = if footer { PartKind::Footer } else { PartKind::Header };
let id = self.doc.add_part(kind, Blocks::new());
self.hf_by_path.insert(path.to_string(), id);
let name: Arc<str> = path.into();
self.doc.missing_pictures.part_names.insert(id, Arc::clone(&name));
let root = match self.xml(path) {
Ok(Some(r)) => r,
_ => return Some(id),
@@ -146,7 +164,9 @@ impl Reader<'_> {
sc.in_hf = true;
let mut blocks = Blocks::new();
self.hf_depth += 1;
let outer = std::mem::replace(&mut self.story, name);
self.read_blocks(&mut sc, &root, &rels, &mut blocks, self.chunks.depth);
self.story = outer;
self.hf_depth = self.hf_depth.saturating_sub(1);
self.flush_pending(&mut sc, &mut blocks);
if let Some(p) = self.doc.parts.get_mut(&id)
+9 -1
View File
@@ -279,7 +279,15 @@ impl Reader<'_> {
fn read_pic(&mut self, pic: &El, rels: &Rels, w: f32, h: f32, alt: String, mut float: Float) -> Option<InlineObject> {
let blip = pic.find("a:blip")?;
float.set_spin(xfrm_spin(pic.find("pic:spPr")));
let media = blip.attr("r:embed").and_then(|id| self.media_for(rels, id))?;
let Some(embed) = blip.attr("r:embed") else {
// A linked picture holds no copy: nothing to draw, and nothing is fetched.
if let Some(link) = blip.attr("r:link") {
let target = rels.by_id(link).map_or_else(|| link.to_string(), |r| r.target.clone());
self.note_missing_picture(&target, wordcraft_doc::MissingPictureReason::LinkOnly);
}
return None;
};
let media = self.media_for(rels, embed)?;
let mut crop = [0.0f32; 4];
if let Some(sr) = pic.find("a:srcRect") {
for (i, n) in ["l", "t", "r", "b"].iter().enumerate() {
+8 -1
View File
@@ -131,7 +131,14 @@ impl Reader<'_> {
None => vml_float(shape, style, sc.in_hf),
};
if let Some(img) = shape.find("v:imagedata") {
let id = img.attr("r:id").or_else(|| img.attr("r:pict"))?;
let Some(id) = img.attr("r:id").or_else(|| img.attr("r:pict")) else {
// `r:href` names a file outside the package: never fetched.
if let Some(href) = img.attr("r:href") {
let target = rels.by_id(href).map_or_else(|| href.to_string(), |r| r.target.clone());
self.note_missing_picture(&target, wordcraft_doc::MissingPictureReason::LinkOnly);
}
return None;
};
let media = self.media_for(rels, id)?;
let alt = shape.attr("alt").or_else(|| img.attr("o:title")).unwrap_or("").to_string();
// Brightness and contrast (Word's picture watermark "Washout" sets them, #1711).
+71
View File
@@ -372,3 +372,74 @@ fn hostile_chunks_never_panic() {
let bytes = package(&format!(r#"<w:altChunk r:id="nope"/><w:altChunk/>{}"#, p("y")), None);
assert_eq!(body_lines(&read(&bytes).unwrap()), ["y"]);
}
/// A DOCX whose only picture names a part the package lacks (`media/gone.png`), or sits outside it.
fn inner_with_dropped_picture(external: bool) -> Vec<u8> {
inner_with_dropped_pictures(1, external)
}
/// As [`inner_with_dropped_picture`], with `n` pictures each naming its own missing part.
fn inner_with_dropped_pictures(n: usize, external: bool) -> Vec<u8> {
let drawing = r#"<w:r><w:drawing><wp:inline xmlns:wp="http://schemas.openxmlformats.org/drawingml/2006/wordprocessingDrawing"><wp:extent cx="254000" cy="254000"/><wp:docPr id="1" name="P"/><a:graphic xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main"><a:graphicData uri="http://schemas.openxmlformats.org/drawingml/2006/picture"><pic:pic xmlns:pic="http://schemas.openxmlformats.org/drawingml/2006/picture"><pic:nvPicPr><pic:cNvPr id="0" name="p"/><pic:cNvPicPr/></pic:nvPicPr><pic:blipFill><a:blip r:embed="rIdP{i}"/></pic:blipFill><pic:spPr><a:xfrm><a:off x="0" y="0"/><a:ext cx="254000" cy="254000"/></a:xfrm><a:prstGeom prst="rect"/></pic:spPr></pic:pic></a:graphicData></a:graphic></wp:inline></w:drawing></w:r>"#;
let body: String = (0..n).map(|i| drawing.replace("{i}", &i.to_string())).collect();
let doc = format!(r#"<w:document {NS}><w:body><w:p>{body}</w:p><w:sectPr/></w:body></w:document>"#);
let mode = if external { r#" TargetMode="External""# } else { "" };
let image = |i: usize| {
let target = if external { format!("http://127.0.0.1:9/never-fetched{i}.png") } else { format!("media/gone{i}.png") };
format!(
r#"<Relationship Id="rIdP{i}" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/image" Target="{target}"{mode}/>"#
)
};
let rels = format!(
r#"<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">{}</Relationships>"#,
(0..n).map(image).collect::<String>()
);
let types = r#"<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types"><Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/><Override PartName="/word/document.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml"/></Types>"#;
let mut zw = zip::ZipWriter::new(std::io::Cursor::new(Vec::new()));
for (name, bytes) in [
("[Content_Types].xml", types.as_bytes()),
("_rels/.rels", ROOT_RELS.as_bytes()),
("word/document.xml", doc.as_bytes()),
("word/_rels/document.xml.rels", rels.as_bytes()),
] {
zw.start_file(name, zip::write::SimpleFileOptions::default()).unwrap();
zw.write_all(bytes).unwrap();
}
zw.finish().unwrap().into_inner()
}
/// A picture the chunk's own reader dropped is the file's too (#6005), named by the chunk part.
#[test]
fn a_picture_dropped_inside_a_chunk_is_recorded_against_the_chunk() {
for (external, reason) in [(false, wordcraft_doc::MissingPictureReason::MissingPart), (true, wordcraft_doc::MissingPictureReason::ExternalTarget)]
{
let inner = inner_with_dropped_picture(external);
let bytes = package(&format!("{}{CHUNK}", p("Before")), Some(("docx", DOCX_MAIN, &inner)));
let d = read(&bytes).unwrap();
let got: Vec<(&str, &str, _)> = d.missing_pictures.list.iter().map(|m| (m.part.as_str(), m.story.as_str(), m.reason)).collect();
assert_eq!(got.len(), 1, "{got:?}");
assert_eq!(got[0].1, "word/afchunk.docx > word/document.xml");
assert_eq!(got[0].2, reason);
}
}
/// A chunk with more dropped pictures than the list holds still counts them all (#6005).
#[test]
fn a_chunks_total_of_dropped_pictures_counts_past_the_list() {
let n = wordcraft_doc::MAX_MISSING_PICTURES + 44;
let inner = inner_with_dropped_pictures(n, false);
let d = read(&package(&format!("{}{CHUNK}", p("Before")), Some(("docx", DOCX_MAIN, &inner)))).unwrap();
assert_eq!(d.missing_pictures.list.len(), wordcraft_doc::MAX_MISSING_PICTURES);
assert!(d.missing_pictures.truncated);
assert_eq!(d.missing_pictures.total, n as u64);
}
/// The same chunk imported twice counts its unlisted pictures once, as it dedupes its listed ones.
#[test]
fn a_chunk_imported_twice_counts_its_dropped_pictures_past_the_list_once() {
let n = wordcraft_doc::MAX_MISSING_PICTURES + 44;
let inner = inner_with_dropped_pictures(n, false);
let d = read(&package(&format!("{}{CHUNK}{CHUNK}", p("Before")), Some(("docx", DOCX_MAIN, &inner)))).unwrap();
assert_eq!(d.missing_pictures.list.len(), wordcraft_doc::MAX_MISSING_PICTURES);
assert_eq!(d.missing_pictures.total, n as u64);
}
+20
View File
@@ -560,3 +560,23 @@ fn a_private_save_keeps_character_references_in_attribute_values_as_written() {
let again = wordcraft_docx::read(&saved).unwrap();
assert_eq!(again.glossary.first().map(|e| e.description.as_str()), Some(before.as_str()));
}
/// A picture in a building block that names a part the package lacks is recorded against the
/// glossary part, not the main document (#6005).
#[test]
fn a_missing_picture_in_a_building_block_names_the_glossary_part() {
let drawing = r#"<w:r><w:drawing><wp:inline xmlns:wp="http://schemas.openxmlformats.org/drawingml/2006/wordprocessingDrawing"><wp:extent cx="254000" cy="254000"/><wp:docPr id="1" name="P"/><a:graphic xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main"><a:graphicData uri="http://schemas.openxmlformats.org/drawingml/2006/picture"><pic:pic xmlns:pic="http://schemas.openxmlformats.org/drawingml/2006/picture"><pic:nvPicPr><pic:cNvPr id="0" name="p"/><pic:cNvPicPr/></pic:nvPicPr><pic:blipFill><a:blip r:embed="rIdPic"/></pic:blipFill><pic:spPr><a:xfrm><a:off x="0" y="0"/><a:ext cx="254000" cy="254000"/></a:xfrm><a:prstGeom prst="rect"/></pic:spPr></pic:pic></a:graphicData></a:graphic></wp:inline></w:drawing></w:r>"#;
let part =
format!(r#"<w:docPart><w:docPartPr><w:name w:val="Logo"/></w:docPartPr><w:docPartBody><w:p>{drawing}</w:p></w:docPartBody></w:docPart>"#);
let bytes = package(
&[("rId7", GLOSSARY_REL, "glossary/document.xml")],
&[
("[Content_Types].xml", content_types(&[("/word/glossary/document.xml", GLOSSARY_CT)])),
("word/glossary/document.xml", glossary_xml(&part)),
("word/glossary/_rels/document.xml.rels", rels(&[("rIdPic", &format!("{REL}/image"), "media/gone.png")])),
],
);
let d = wordcraft_docx::read(&bytes).unwrap();
let stories: Vec<&str> = d.missing_pictures.list.iter().map(|m| m.story.as_str()).collect();
assert_eq!(stories, ["word/glossary/document.xml"], "{:?}", d.missing_pictures.list);
}
+1
View File
@@ -21,6 +21,7 @@ mod heading_levels;
mod legacy_forms;
mod ligatures;
mod malformed;
mod missing_pictures;
mod nested_block_controls;
mod note_comments;
mod passthrough;
+163
View File
@@ -0,0 +1,163 @@
//! Pictures dropped at open are recorded for strict export (#6005): a missing part, an external
//! target and a link-only picture (ECMA-376 Part 1 §20.1.8.13 `a:blip` `r:embed`/`r:link`).
//! Original synthetic packages; nothing is fetched, the reader has no network code.
use std::io::Write;
use wordcraft_doc::{Document, InlineObject, MissingPictureReason};
const NS: &str = r#"xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships" xmlns:wp="http://schemas.openxmlformats.org/drawingml/2006/wordprocessingDrawing" xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main" xmlns:pic="http://schemas.openxmlformats.org/drawingml/2006/picture" xmlns:v="urn:schemas-microsoft-com:vml""#;
const REL: &str = "http://schemas.openxmlformats.org/officeDocument/2006/relationships";
fn png() -> Vec<u8> {
let mut out = Vec::new();
image::RgbaImage::from_pixel(2, 2, image::Rgba([20, 90, 200, 255]))
.write_to(&mut std::io::Cursor::new(&mut out), image::ImageFormat::Png)
.unwrap();
out
}
fn blip(attrs: &str) -> String {
format!(
r#"<w:r><w:drawing><wp:inline><wp:extent cx="254000" cy="254000"/><wp:docPr id="1" name="P"/><a:graphic><a:graphicData uri="http://schemas.openxmlformats.org/drawingml/2006/picture"><pic:pic><pic:nvPicPr><pic:cNvPr id="0" name="p"/><pic:cNvPicPr/></pic:nvPicPr><pic:blipFill><a:blip {attrs}/></pic:blipFill><pic:spPr><a:xfrm><a:off x="0" y="0"/><a:ext cx="254000" cy="254000"/></a:xfrm><a:prstGeom prst="rect"/></pic:spPr></pic:pic></a:graphicData></a:graphic></wp:inline></w:drawing></w:r>"#
)
}
/// A package: `body` in `word/document.xml` (whose relationships are `doc_rels`), an optional
/// header with `header` as its content (relationships `header_rels`) and `extra` parts.
fn package(body: &str, doc_rels: &str, header: Option<(&str, &str)>, extra: &[(&str, Vec<u8>)]) -> Vec<u8> {
let hdr_rel = if header.is_some() { format!(r#"<Relationship Id="rIdH" Type="{REL}/header" Target="header2.xml"/>"#) } else { String::new() };
let sect = if header.is_some() { r#"<w:sectPr><w:headerReference w:type="default" r:id="rIdH"/></w:sectPr>"# } else { "<w:sectPr/>" };
let doc = format!(r#"<w:document {NS}><w:body><w:p>{body}</w:p>{sect}</w:body></w:document>"#);
let rels = format!(
r#"<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships"><Relationship Id="rId1" Type="{REL}/officeDocument" Target="word/document.xml"/></Relationships>"#
);
let drels = format!(r#"<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">{doc_rels}{hdr_rel}</Relationships>"#);
let types = r#"<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types"><Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/><Default Extension="png" ContentType="image/png"/><Default Extension="bin" ContentType="application/octet-stream"/><Override PartName="/word/document.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml"/></Types>"#;
let mut zip = zip::ZipWriter::new(std::io::Cursor::new(Vec::new()));
let mut put = |name: &str, bytes: &[u8]| {
zip.start_file(name, zip::write::SimpleFileOptions::default()).unwrap();
zip.write_all(bytes).unwrap();
};
put("[Content_Types].xml", types.as_bytes());
put("_rels/.rels", rels.as_bytes());
put("word/document.xml", doc.as_bytes());
put("word/_rels/document.xml.rels", drels.as_bytes());
if let Some((content, hrels)) = header {
put("word/header2.xml", format!(r#"<w:hdr {NS}><w:p>{content}</w:p></w:hdr>"#).as_bytes());
put(
"word/_rels/header2.xml.rels",
format!(r#"<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">{hrels}</Relationships>"#).as_bytes(),
);
}
for (n, b) in extra {
put(n, b);
}
zip.finish().unwrap().into_inner()
}
fn image_rel(id: &str, target: &str, external: bool) -> String {
let mode = if external { r#" TargetMode="External""# } else { "" };
format!(r#"<Relationship Id="{id}" Type="{REL}/image" Target="{target}"{mode}/>"#)
}
fn missing(d: &Document) -> Vec<(String, String, MissingPictureReason)> {
d.missing_pictures.list.iter().map(|m| (m.part.clone(), m.story.clone(), m.reason)).collect()
}
fn pictures(d: &Document) -> usize {
d.body[0].as_para().unwrap().objects.iter().filter(|o| matches!(o, InlineObject::Image { .. })).count()
}
#[test]
fn external_missing_and_link_only_pictures_are_recorded_and_still_dropped() {
let body = [blip(r#"r:embed="rIdExt""#), blip(r#"r:embed="rIdGone""#), blip(r#"r:link="rIdLink""#), blip(r#"r:embed="rIdNone""#)].concat();
let rels = [
image_rel("rIdExt", "http://127.0.0.1:9/never-fetched.png", true),
image_rel("rIdGone", "media/absent.png", false),
image_rel("rIdLink", "file:///C:/pics/linked.png", true),
]
.concat();
let d = wordcraft_docx::read(&package(&body, &rels, None, &[])).unwrap();
assert_eq!(pictures(&d), 0, "nothing is drawn for them, as before");
assert_eq!(
missing(&d),
[
("http://127.0.0.1:9/never-fetched.png".into(), "word/document.xml".into(), MissingPictureReason::ExternalTarget),
("word/media/absent.png".into(), "word/document.xml".into(), MissingPictureReason::MissingPart),
("file:///C:/pics/linked.png".into(), "word/document.xml".into(), MissingPictureReason::LinkOnly),
("rIdNone".into(), "word/document.xml".into(), MissingPictureReason::MissingPart),
]
);
assert!(!d.missing_pictures.truncated);
}
#[test]
fn a_header_pictures_story_is_the_header_part() {
let rels = [image_rel("rIdHx", "https://example.invalid/logo.png", true), image_rel("rIdOk", "media/ok.png", false)].concat();
let header = [blip(r#"r:embed="rIdHx""#), blip(r#"r:embed="rIdOk""#)].concat();
let bytes = package("", "", Some((&header, &rels)), &[("word/media/ok.png", png())]);
let d = wordcraft_docx::read(&bytes).unwrap();
assert_eq!(missing(&d), [("https://example.invalid/logo.png".into(), "word/header2.xml".into(), MissingPictureReason::ExternalTarget)]);
assert_eq!(d.missing_pictures.main, "word/document.xml");
assert!(d.missing_pictures.part_names.values().any(|n| &**n == "word/header2.xml"));
assert_eq!(d.missing_pictures.media_parts.get("ok.png").map(String::as_str), Some("word/media/ok.png"));
}
#[test]
fn a_picture_that_is_there_is_not_recorded() {
let body = blip(r#"r:embed="rIdOk""#);
let d = wordcraft_docx::read(&package(&body, &image_rel("rIdOk", "media/ok.png", false), None, &[("word/media/ok.png", png())])).unwrap();
assert_eq!(pictures(&d), 1);
assert!(d.missing_pictures.list.is_empty());
}
#[test]
fn vml_image_data_links_are_recorded() {
let pict =
|attrs: &str| format!(r#"<w:r><w:pict><v:shape id="s" style="width:50pt;height:50pt"><v:imagedata {attrs}/></v:shape></w:pict></w:r>"#);
let body = [pict(r#"r:id="rIdExt""#), pict(r#"r:href="rIdLink""#), pict(r#"r:id="rIdGone""#)].concat();
let rels = [
image_rel("rIdExt", "https://example.invalid/a.png", true),
image_rel("rIdLink", "https://example.invalid/b.png", true),
image_rel("rIdGone", "media/none.png", false),
]
.concat();
let d = wordcraft_docx::read(&package(&body, &rels, None, &[])).unwrap();
let got: Vec<_> = missing(&d).into_iter().map(|(p, _, r)| (p, r)).collect();
assert_eq!(
got,
[
("https://example.invalid/a.png".to_string(), MissingPictureReason::ExternalTarget),
("https://example.invalid/b.png".to_string(), MissingPictureReason::LinkOnly),
("word/media/none.png".to_string(), MissingPictureReason::MissingPart),
]
);
}
#[test]
fn the_list_is_capped_and_the_document_is_still_equal() {
let n = wordcraft_doc::MAX_MISSING_PICTURES + 5;
let body: String = (0..n).map(|i| blip(&format!(r#"r:embed="r{i}""#))).collect();
let rels: String = (0..n).map(|i| image_rel(&format!("r{i}"), &format!("media/gone{i}.png"), false)).collect();
let d = wordcraft_docx::read(&package(&body, &rels, None, &[])).unwrap();
assert_eq!(d.missing_pictures.list.len(), wordcraft_doc::MAX_MISSING_PICTURES);
assert!(d.missing_pictures.truncated);
// Not part of the document: `==` and the clone ignore it, and a save doesn't carry it.
let mut e = d.clone();
e.missing_pictures = Default::default();
assert_eq!(d, e);
assert!(!serde_json::to_string(&d).unwrap().contains("gone"));
}
/// Every note of a part shares the one allocation of its source name, so a file with many notes
/// in a part with a long name does not keep a copy of the name per note.
#[test]
fn the_notes_of_a_part_share_one_copy_of_its_name() {
let notes: String = (1..=3).map(|i| format!(r#"<w:footnote w:id="{i}"><w:p><w:r><w:t>n{i}</w:t></w:r></w:p></w:footnote>"#)).collect();
let rels = format!(r#"<Relationship Id="rIdF" Type="{REL}/footnotes" Target="footnotes.xml"/>"#);
let part = format!(r#"<w:footnotes {NS}>{notes}</w:footnotes>"#);
let d = wordcraft_docx::read(&package("x", &rels, None, &[("word/footnotes.xml", part.into_bytes())])).unwrap();
let names: Vec<_> = d.missing_pictures.part_names.values().collect();
assert!(names.len() >= 3, "{names:?}");
assert!(names.iter().all(|n| &***n == "word/footnotes.xml"));
assert!(names.windows(2).all(|w| std::sync::Arc::ptr_eq(w[0], w[1])), "each note holds its own copy of the part name");
}
+46 -6
View File
@@ -94,9 +94,9 @@ pub fn specs() -> Vec<CommandSpec> {
.pure(),
CommandSpec::new("file.save", "Save", "File", save)
.key("Mod+S / Shift+F12 / Alt+Shift+F2")
.params(r#"{"path"?: string, "password"?: string (encrypt this and later saves with it, as file.encrypt does), "markup"?: bool (Print Markup for a PDF, PNG, HTML, Markdown or text copy: comments shown; default file.printMarkup), "format"?: "wordml2003" (an .xml file as a Word 2003 XML Document) | "flatopc" (as a Word XML Document, the default; saving over a Word 2003 XML file keeps that format)}"#)
.params(r#"{"path"?: string, "password"?: string (encrypt this and later saves with it, as file.encrypt does), "markup"?: bool (Print Markup for a PDF, PNG, HTML, Markdown or text copy: comments shown; default file.printMarkup), "format"?: "wordml2003" (an .xml file as a Word 2003 XML Document) | "flatopc" (as a Word XML Document, the default; saving over a Word 2003 XML file keeps that format), "strict"?: bool (a .pdf or .png only: refuse, writing nothing, if a character can't be drawn or a picture is missing; see file.strictReport)}"#)
.pure(),
CommandSpec::new("file.saveAs", "Save As", "File", save_as).key(super::mac_or("F12 / Mod+Shift+S", "F12 / Alt+F2")).params(r#"{"path": string, "password"?: string, "markup"?: bool, "format"?: string (as file.save)}"#).pure(),
CommandSpec::new("file.saveAs", "Save As", "File", save_as).key(super::mac_or("F12 / Mod+Shift+S", "F12 / Alt+F2")).params(r#"{"path": string, "password"?: string, "markup"?: bool, "format"?: string (as file.save), "strict"?: bool (as file.save)}"#).pure(),
// Pure: the password must not land in Repeat or a recorded macro, and it isn't an edit
// to undo; it marks the document as changed itself.
CommandSpec::new("file.encrypt", "Encrypt with Password", "File › Info › Protect Document", encrypt)
@@ -114,10 +114,13 @@ pub fn specs() -> Vec<CommandSpec> {
crate::io::save_path_with(std::path::Path::new(&path), &s.doc, None, &opts).map_err(CmdError::Failed)?;
Ok(json!({"path": path, "markup": opts.markup}))
})
.params(r#"{"path": string, "markup"?: bool (Print Markup: tracked changes marked up and each page scaled down beside the comments in balloons; default file.printMarkup)}"#)
.params(r#"{"path": string, "markup"?: bool (Print Markup: tracked changes marked up and each page scaled down beside the comments in balloons; default file.printMarkup), "strict"?: bool (refuse, writing nothing, if a character can't be drawn or a picture is missing; see file.strictReport)}"#)
.pure(),
CommandSpec::new("file.exportPng", "Export Page as PNG", "File › Export", export_png)
.params(r#"{"path": string, "page"?: n (1-based), "scale"?: px per pt, "markup"?: bool (the page with its comments in balloons beside it; default file.printMarkup)}"#)
.params(r#"{"path": string, "page"?: n (1-based), "scale"?: px per pt, "markup"?: bool (the page with its comments in balloons beside it; default file.printMarkup), "strict"?: bool (refuse, writing nothing, if a character on the page can't be drawn or a picture is missing; see file.strictReport)}"#)
.pure(),
CommandSpec::new("file.strictReport", "Strict Export Report", "File › Export", strict_report)
.params(r#"{"markup"?: bool (as file.exportPdf), "page"?: n (1-based: check that page of the page image file.exportPng writes; omit for every page of a PDF)} → {"schema": 1, "strict": "passed" | "failed", "missing_glyphs": [{"cp": "U+2D30", "count": n}], "missing_pictures": [{"part", "source_part": the story part it is in (word/header2.xml; background, body, header, footer, text box or "" where unknown), "reason": "missing_part" | "external_target" | "link_only" | "undecodable" | "unsupported_format"}], "truncated": bool (either list was cut at 256 entries), "total_missing_glyphs": n (distinct, past the cap too), "total_missing_pictures": n (past the cap too), "glyphs_truncated": bool, "pictures_truncated": bool}"#)
.pure(),
// A print setting, like Word's: not part of the document, so no undo step.
CommandSpec::new("file.printMarkup", "Print Markup", "File › Print", |s, v| {
@@ -369,6 +372,7 @@ fn save(s: &mut Session, v: &Value) -> CmdResult {
return Ok(json!({"saved": false}));
}
};
check_strict_output(p::bool(v, "strict").unwrap_or(false), &path.to_string_lossy())?;
if let Some(pw) = v.get("password").filter(|pw| !pw.is_null()) {
let Some(pw) = pw.as_str().filter(|pw| !pw.is_empty()) else {
return Err(CmdError::Params("`password` must be a non-empty string (file.encrypt removes a password)".into()));
@@ -450,9 +454,39 @@ pub fn export_options(s: &Session, v: &Value) -> crate::io::ExportOptions {
comments: markup && s.prefs.markup.comments,
word2003: p::str(v, "format") == Some(FORMAT_WORD_2003),
numeral: s.view.numeral.resolve(crate::session::system_locale()),
strict: p::bool(v, "strict").unwrap_or(false),
}
}
/// `strict: true` is for a PDF or a page image only: any other output is a parameter error.
fn check_strict_output(strict: bool, path: &str) -> Result<(), CmdError> {
if strict && !crate::strict::is_strict_output(path) {
return Err(CmdError::Params("`strict` applies to a .pdf or .png output only".into()));
}
Ok(())
}
/// The strict report of a PDF export of the document (every page), or with `page` of the page
/// image `file.exportPng` writes. Pure: it reads the document and writes nothing.
fn strict_report(s: &mut Session, v: &Value) -> CmdResult {
let opts = export_options(s, v);
let report = match p::u64(v, "page") {
Some(n) => {
let page = crate::p::page_index(n);
let l = s.export_layout_with(opts.markup);
if l.pages.get(page).is_none() {
return Err(CmdError::Params(format!("no page {}", page + 1)));
}
crate::strict::strict_report(&s.doc, &l, &crate::strict::png_display(), Some(page..=page), wordcraft_layout::StrictOutput::Png)
}
None => {
let out = super::references::output_fields(&s.doc);
crate::strict::pdf_report(out.as_ref().unwrap_or(&s.doc), &opts)
}
};
Ok(report.to_json())
}
fn export_png(s: &mut Session, v: &Value) -> CmdResult {
// No path: the app asks where to save, as file.exportPdf does (#2313).
let Some(path) = p::str(v, "path") else {
@@ -460,16 +494,22 @@ fn export_png(s: &mut Session, v: &Value) -> CmdResult {
return Ok(json!({"exported": false}));
};
let page = crate::p::page_index(p::u64(v, "page").unwrap_or(1));
let strict = p::bool(v, "strict").unwrap_or(false);
let scale = p::f32(v, "scale").unwrap_or(2.0).clamp(0.1, 8.0);
let l = s.export_layout_with(export_options(s, v).markup);
let pg = l.pages.get(page).ok_or_else(|| CmdError::Params(format!("no page {}", page + 1)))?;
// A page image is the printed page: headers and footers at full strength. Dimming them is
// the on-screen cue while editing the body, not part of the page.
let mut opts = wordcraft_render::RenderOptions::default();
opts.display.dim_header = false;
let opts = wordcraft_render::RenderOptions { display: crate::strict::png_display(), ..Default::default() };
if !path.to_ascii_lowercase().ends_with(".png") {
return Err(CmdError::Params("path must end in .png".into()));
}
if strict {
let report = crate::strict::strict_report(&s.doc, &l, &opts.display, Some(page..=page), wordcraft_layout::StrictOutput::Png);
if !report.passed() {
return Err(CmdError::Failed(report.refusal()));
}
}
let img = wordcraft_render::render_page(&s.doc, pg, scale, &opts);
let png = img.to_png();
crate::io::write_file(path, &png).map_err(CmdError::Failed)?;
+4
View File
@@ -66,6 +66,10 @@ pub struct ExportOptions {
pub word2003: bool,
/// How digits look in PDFs and page images (the Numeral setting, `System` resolved).
pub numeral: wordcraft_layout::NumeralMode,
/// A PDF or page image is written only if nothing in it is silently missing (a character no
/// face can draw, a picture that is missing or can't be decoded); else the save fails with
/// `strict: …` (#6005). Ignored for other formats.
pub strict: bool,
}
/// A document password: wiped from memory when dropped, never printed (`Debug` shows `***`).
+7 -7
View File
@@ -65,13 +65,7 @@ pub fn save(ext: &str, doc: &Document, opts: &crate::io::ExportOptions) -> Optio
// Output shows the fields Word fills in as it prints (see `output_fields`).
let out = crate::cmd::references::output_fields(doc);
let doc = out.as_ref().unwrap_or(doc);
let pdf = wordcraft_pdf::PdfOptions {
include_markup: opts.markup,
comments: opts.markup && opts.comments,
numeral: opts.numeral,
..Default::default()
};
Some(wordcraft_pdf::export(doc, &pdf).map_err(|e| e.to_string()))
Some(crate::strict::export_pdf(doc, opts))
}
other => wordcraft_formats::export_with(other, doc, &wordcraft_formats::ExportOptions { comments: opts.markup && opts.comments }),
}
@@ -85,6 +79,12 @@ pub fn render_png(doc: &Document, page: usize, scale: f32, opts: &crate::io::Exp
let lo = wordcraft_layout::LayoutOptions { comment_balloons: opts.markup && opts.comments, numeral: opts.numeral, ..Default::default() };
let l = wordcraft_layout::layout(doc, &mut wordcraft_layout::LayoutCache::new(), &lo);
let p = l.pages.get(page).ok_or_else(|| format!("no page {}", page + 1))?;
if opts.strict {
let report = crate::strict::strict_report(doc, &l, &crate::strict::png_display(), Some(page..=page), wordcraft_layout::StrictOutput::Png);
if !report.passed() {
return Err(report.refusal());
}
}
let img = wordcraft_render::render_page(doc, p, scale, &Default::default());
Ok(img.to_png())
}
+1
View File
@@ -25,6 +25,7 @@ pub mod sample;
mod session;
pub(crate) mod snapshot;
pub mod speech;
pub mod strict;
pub(crate) mod view;
use std::collections::HashMap;
+233
View File
@@ -0,0 +1,233 @@
//! Strict export (#6005): the report of what a PDF or PNG would leave out silently (characters no
//! face can draw, pictures that are missing or can't be decoded), and the refusal built from it.
//!
//! The check runs on the layout and display options the export itself uses, so what it reports is
//! what the output lacks.
use std::ops::RangeInclusive;
use serde_json::{Value, json};
use wordcraft_doc::{Document, MAX_MISSING_PICTURES, MissingPicture};
use wordcraft_layout::display::DisplayOptions;
use wordcraft_layout::{DocLayout, LayoutCache, StrictOutput};
use crate::io::ExportOptions;
/// The report's JSON schema version; fields may be added without raising it.
pub const SCHEMA: u32 = 1;
/// What a strict export found missing.
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct StrictReport {
/// (code point, times drawn), sorted by code point.
pub glyphs: Vec<(char, u64)>,
pub pictures: Vec<MissingPicture>,
/// More distinct characters were missing than are listed.
pub glyphs_truncated: bool,
/// More pictures were missing than are listed.
pub pictures_truncated: bool,
/// Distinct characters found missing, those past the list's cap included.
pub total_glyphs: usize,
/// Pictures found missing, those past the list's cap included.
pub total_pictures: u64,
}
impl StrictReport {
/// Nothing is missing: the strict export passes.
pub fn passed(&self) -> bool {
self.glyphs.is_empty() && self.pictures.is_empty() && !self.truncated()
}
/// Either list was cut at its cap.
pub fn truncated(&self) -> bool {
self.glyphs_truncated || self.pictures_truncated
}
/// The schema-1 JSON of the report.
pub fn to_json(&self) -> Value {
json!({
"schema": SCHEMA,
"strict": if self.passed() { "passed" } else { "failed" },
"missing_glyphs": self.glyphs.iter().map(|(c, n)| json!({"cp": code_point(*c), "count": n})).collect::<Vec<_>>(),
"missing_pictures": self.pictures.iter().map(|p| json!({"part": p.part, "source_part": p.story, "reason": p.reason.code()})).collect::<Vec<_>>(),
"truncated": self.truncated(),
"total_missing_glyphs": self.total_glyphs,
"total_missing_pictures": self.total_pictures,
"glyphs_truncated": self.glyphs_truncated,
"pictures_truncated": self.pictures_truncated,
})
}
/// The human lines (without the `strict: ` prefix): one for the characters, one for the
/// pictures; none when nothing is missing.
fn parts(&self) -> Vec<String> {
let glyphs: Vec<(String, u64)> = self.glyphs.iter().map(|(c, n)| (code_point(*c), *n)).collect();
let pictures: Vec<(String, String, String)> =
self.pictures.iter().map(|p| (p.part.clone(), story_label(&p.story).to_string(), p.reason.label().to_string())).collect();
human_parts(&glyphs, &pictures, (self.glyphs_truncated, self.pictures_truncated), (self.total_glyphs as u64, self.total_pictures))
}
/// The human lines, each `strict: …`.
pub fn lines(&self) -> Vec<String> {
self.parts().into_iter().map(|p| format!("strict: {p}")).collect()
}
/// The refusal's message: `strict: …`, the lines joined.
pub fn refusal(&self) -> String {
format!("strict: {}", self.parts().join("; "))
}
}
/// The human text of a report: (code point text, count) and (part, story label, reason words).
fn human_parts(glyphs: &[(String, u64)], pictures: &[(String, String, String)], cut: (bool, bool), totals: (u64, u64)) -> Vec<String> {
let (glyphs_cut, pictures_cut) = cut;
let mut out = Vec::new();
if !glyphs.is_empty() {
let list: Vec<String> = glyphs.iter().map(|(cp, n)| format!("{cp} (×{n})")).collect();
let more = if glyphs_cut { ", …" } else { "" };
let listed = glyphs.len() as u64;
let n = if glyphs_cut { totals.0.max(listed) } else { listed };
let first = if n > listed { format!(" (first {listed} listed)") } else { String::new() };
out.push(format!("{n} character{} cannot be drawn{first}: {}{more}", if n == 1 { "" } else { "s" }, list.join(", ")));
}
if !pictures.is_empty() {
let list: Vec<String> = pictures
.iter()
.map(|(part, at, reason)| if at.is_empty() { format!("{part}: {reason}") } else { format!("{part} ({at}): {reason}") })
.collect();
let more = if pictures_cut { "; …" } else { "" };
let listed = pictures.len() as u64;
let n = if pictures_cut { totals.1.max(listed) } else { listed };
let first = if n > listed { format!(" (first {listed} listed)") } else { String::new() };
out.push(format!("{n} picture(s) left out{first}: {}{more}", list.join("; ")));
}
out
}
/// The `strict: …` human lines of a report in its JSON form (what `file.strictReport` returns).
/// Anything the JSON lacks is left out.
pub fn lines_from_json(report: &Value) -> Vec<String> {
let text = |v: &Value, k: &str| v.get(k).and_then(Value::as_str).unwrap_or_default().to_string();
let glyphs: Vec<(String, u64)> = report
.get("missing_glyphs")
.and_then(Value::as_array)
.map(|a| a.iter().map(|g| (text(g, "cp"), g.get("count").and_then(Value::as_u64).unwrap_or(0))).collect())
.unwrap_or_default();
let pictures: Vec<(String, String, String)> = report
.get("missing_pictures")
.and_then(Value::as_array)
.map(|a| a.iter().map(|p| (text(p, "part"), story_label(&text(p, "source_part")).to_string(), reason_label(&text(p, "reason")))).collect())
.unwrap_or_default();
let flag = |k: &str| report.get(k).and_then(Value::as_bool).unwrap_or(false);
// A report without the per-list flags (the top-level one alone) marks both lists.
let (glyphs_cut, pictures_cut) = match (report.get("glyphs_truncated"), report.get("pictures_truncated")) {
(None, None) => (flag("truncated"), flag("truncated")),
_ => (flag("glyphs_truncated"), flag("pictures_truncated")),
};
let total = |k: &str, listed: usize| report.get(k).and_then(Value::as_u64).unwrap_or(listed as u64);
let totals = (total("total_missing_glyphs", glyphs.len()), total("total_missing_pictures", pictures.len()));
human_parts(&glyphs, &pictures, (glyphs_cut, pictures_cut), totals).into_iter().map(|p| format!("strict: {p}")).collect()
}
/// The words for a JSON reason code; a code this version does not know is shown as it is.
fn reason_label(code: &str) -> String {
wordcraft_doc::MissingPictureReason::from_code(code).map_or_else(|| code.to_string(), |r| r.label().to_string())
}
/// `U+2D30`: upper-case hex, at least four digits.
fn code_point(c: char) -> String {
format!("U+{:04X}", c as u32)
}
/// The part a picture is in, as the report names it: `header2.xml` for `word/header2.xml`.
fn story_label(story: &str) -> &str {
story.strip_prefix("word/").unwrap_or(story)
}
/// The report of `doc` as drawn by `lay` with `display` into `output`, on `pages` (0-based; `None`
/// is every page). The watermark is checked with the page, and the pictures dropped when the
/// document was opened are added to those that can't be decoded now, capped like each.
pub fn strict_report(
doc: &Document,
lay: &DocLayout,
display: &DisplayOptions,
pages: Option<RangeInclusive<usize>>,
output: StrictOutput,
) -> StrictReport {
let mut missing = lay.missing_glyphs(doc, display, pages.clone());
let any_page = lay.pages.iter().enumerate().any(|(n, _)| pages.as_ref().is_none_or(|r| r.contains(&n)));
if any_page && let Some(wm) = &doc.settings.watermark {
let max = match output {
StrictOutput::Pdf => wordcraft_pdf::WATERMARK_MAX_CHARS,
StrictOutput::Png => usize::MAX,
};
missing.merge(wordcraft_layout::missing_watermark_glyphs(wm, max));
}
let (undecodable, mut pictures_truncated, undecodable_total) = wordcraft_layout::undecodable_pictures(doc, lay, pages, display, output);
pictures_truncated |= doc.missing_pictures.truncated;
// The two lists overlap where the same picture is in both; those are counted once.
let mut overlap: u64 = 0;
let mut pictures: Vec<MissingPicture> = Vec::new();
for p in doc.missing_pictures.list.iter().cloned().chain(undecodable) {
if pictures.contains(&p) {
overlap = overlap.saturating_add(1);
continue;
}
if pictures.len() >= MAX_MISSING_PICTURES {
pictures_truncated = true;
break;
}
pictures.push(p);
}
let total_pictures = doc.missing_pictures.total.saturating_add(undecodable_total).saturating_sub(overlap).max(pictures.len() as u64);
StrictReport {
glyphs: missing.chars,
pictures,
glyphs_truncated: missing.truncated,
pictures_truncated,
total_glyphs: missing.total,
total_pictures,
}
}
/// The PDF export's options for `opts`: the one place they are built, shared by the export and
/// its strict check.
pub(crate) fn pdf_options(opts: &ExportOptions) -> wordcraft_pdf::PdfOptions {
wordcraft_pdf::PdfOptions { include_markup: opts.markup, comments: opts.markup && opts.comments, numeral: opts.numeral, ..Default::default() }
}
/// What a page image shows: the display options `file.exportPng` renders with.
pub(crate) fn png_display() -> DisplayOptions {
// A page image is the printed page: headers and footers at full strength.
let mut opts = wordcraft_render::RenderOptions::default();
opts.display.dim_header = false;
opts.display
}
/// The strict report of a PDF export of `doc`: every page, laid out as the export lays it out.
pub(crate) fn pdf_report(doc: &Document, opts: &ExportOptions) -> StrictReport {
let pdf = pdf_options(opts);
let lay = wordcraft_layout::layout(doc, &mut LayoutCache::new(), &pdf.layout_options());
strict_report(doc, &lay, &pdf.display_options(), None, StrictOutput::Pdf)
}
/// The PDF of `doc`; with `opts.strict`, only if nothing is missing, laid out once for the check
/// and the export.
pub(crate) fn export_pdf(doc: &Document, opts: &ExportOptions) -> Result<Vec<u8>, String> {
let pdf = pdf_options(opts);
if !opts.strict {
return wordcraft_pdf::export(doc, &pdf).map_err(|e| e.to_string());
}
let lay = wordcraft_layout::layout(doc, &mut LayoutCache::new(), &pdf.layout_options());
let report = strict_report(doc, &lay, &pdf.display_options(), None, StrictOutput::Pdf);
if !report.passed() {
return Err(report.refusal());
}
wordcraft_pdf::export_layout(doc, &lay, &pdf).map_err(|e| e.to_string())
}
/// Is `path` a PDF or PNG output (the only outputs a strict export applies to)?
pub(crate) fn is_strict_output(path: &str) -> bool {
let lower = path.to_ascii_lowercase();
lower.ends_with(".pdf") || lower.ends_with(".png")
}
+7 -1
View File
@@ -4683,7 +4683,7 @@ fn typed_paragraphs_are_capped_for_every_caller() {
assert!(!body_texts(&s).iter().any(|p| p.contains('y')));
}
fn hostile_junk() -> [serde_json::Value; 8] {
fn hostile_junk() -> [serde_json::Value; 10] {
[
json!(null),
json!({}),
@@ -4693,6 +4693,9 @@ fn hostile_junk() -> [serde_json::Value; 8] {
json!({"page": 0, "x": 1e30, "y": -1e30, "extend": 1}),
json!({"width": 1e-9, "height": 1e30, "top": -1e9, "left": 1e9}),
json!({"section": {"pageW": -1e308, "pageH": 0, "marginTop": 1e30, "header": -5, "columns": {"count": 1000000, "space": -1e300, "widths": [[0, 0]]}}}),
// Strict export (#6005): the flag, and file.strictReport's `page`, with junk.
json!({"strict": true, "page": 0, "scale": 0, "markup": true}),
json!({"strict": "x", "page": 1e30, "scale": 1e-9, "markup": "x"}),
]
}
@@ -4789,6 +4792,9 @@ fn hostile_file_params_never_panic() {
json!({"path": path, "password": "x", "format": "wordml2003", "markup": "x", "confirmDamaged": 1}),
json!({"path": path, "password": 5, "format": 5, "markup": true}),
json!({"path": path, "page": 0, "scale": 0, "width": -1e308, "alt": 5}),
json!({"path": path, "strict": true, "page": 1, "scale": 0.1}),
json!({"path": path, "strict": true, "page": 1e30, "markup": true, "format": "flatopc"}),
json!({"path": path, "strict": 5, "page": -1}),
json!({"path": path, "data": "not base64!"}),
json!({"path": path, "data": "AAAA"}),
json!({"path": path, "pagesPerSheet": 7, "range": "x", "pages": "9-1,,", "parity": "x", "copies": 3, "paper": "x"}),
+322
View File
@@ -0,0 +1,322 @@
//! Strict export (#6005): `strict: true` on the PDF and PNG exports refuses an output that would
//! silently lack something, before any byte is written; `file.strictReport` says what.
use serde_json::{Value, json};
use wordcraft_doc::Document;
use wordcraft_engine::{CmdError, Session};
use wordcraft_fonts::FontDb;
/// Code points in unassigned plane 4 that no face of the host covers (as the layout tests find them).
fn uncovered_many(n: usize) -> Vec<char> {
let db = FontDb::global();
let primary = wordcraft_fonts::word::resolve(&Document::default().settings.minor_font, false, false).face;
let out: Vec<char> = (0x40000u32..0x4FFF0)
.filter_map(char::from_u32)
.filter(|c| {
!primary.covers(*c) && db.fallback_styled(*c, primary.id(), false, false).is_none() && db.fallback_for(*c, primary.id()).is_none()
})
.take(n)
.collect();
assert_eq!(out.len(), n, "the host has faces covering all of plane 4");
out
}
fn uncovered() -> char {
uncovered_many(1)[0]
}
fn dir(name: &str) -> std::path::PathBuf {
let d = std::env::temp_dir().join(format!("wc-strict-export-{name}-{}", std::process::id()));
std::fs::create_dir_all(&d).unwrap();
d
}
fn refused(r: Result<Value, CmdError>) -> String {
match r {
Err(CmdError::Failed(m)) => {
assert!(m.starts_with("strict: "), "{m}");
m
}
other => panic!("expected a strict refusal, got {other:?}"),
}
}
#[test]
fn strict_refuses_an_uncovered_character_and_writes_nothing() {
let c = uncovered();
let d = dir("refuse");
let mut s = Session::new(Document::from_text(&format!("a{c}b")));
let pdf = d.join("out.pdf");
let png = d.join("out.png");
let msg = refused(s.run("file.exportPdf", &json!({"path": pdf, "strict": true})));
assert!(msg.contains(&format!("U+{:04X} (×1)", c as u32)), "{msg}");
refused(s.run("file.save", &json!({"path": pdf, "strict": true})));
refused(s.run("file.saveAs", &json!({"path": pdf, "strict": true})));
refused(s.run("file.exportPng", &json!({"path": png, "strict": true})));
refused(s.run("file.save", &json!({"path": png, "strict": true})));
assert!(!pdf.exists() && !png.exists());
std::fs::remove_dir_all(d).unwrap();
}
#[test]
fn a_refusal_leaves_an_existing_output_untouched() {
let c = uncovered();
let d = dir("keep");
let pdf = d.join("out.pdf");
std::fs::write(&pdf, b"previous").unwrap();
let mut s = Session::new(Document::from_text(&format!("a{c}")));
refused(s.run("file.exportPdf", &json!({"path": pdf, "strict": true})));
assert_eq!(std::fs::read(&pdf).unwrap(), b"previous");
std::fs::remove_dir_all(d).unwrap();
}
#[test]
fn a_refused_save_leaves_the_stamps_path_and_dirty_flag_alone() {
let c = uncovered();
let d = dir("stamp");
let mut s = Session::new(Document::from_text(&format!("a{c}")));
s.dirty = true;
let before =
(s.doc.core.modified.clone(), s.doc.core.created.clone(), s.doc.core.revision, s.doc.core.last_modified_by.clone(), s.doc.settings.zoom);
refused(s.run("file.save", &json!({"path": d.join("out.pdf"), "strict": true})));
let after =
(s.doc.core.modified.clone(), s.doc.core.created.clone(), s.doc.core.revision, s.doc.core.last_modified_by.clone(), s.doc.settings.zoom);
assert_eq!(before, after);
assert!(s.dirty);
assert_eq!(s.path, None);
std::fs::remove_dir_all(d).unwrap();
}
#[test]
fn strict_passes_and_writes_when_nothing_is_missing() {
let d = dir("pass");
let mut s = Session::new(Document::from_text("Hello, world."));
for (cmd, file, magic) in [
("file.exportPdf", "a.pdf", &b"%PDF-"[..]),
("file.save", "b.pdf", &b"%PDF-"[..]),
("file.exportPng", "a.png", &b"\x89PNG"[..]),
("file.save", "b.png", &b"\x89PNG"[..]),
] {
let path = d.join(file);
s.run(cmd, &json!({"path": path, "strict": true})).unwrap();
assert!(std::fs::read(&path).unwrap().starts_with(magic), "{file}");
}
std::fs::remove_dir_all(d).unwrap();
}
#[test]
fn strict_passes_a_document_the_same_pdf_as_without_it() {
let d = dir("pdfsame");
let mut s = Session::new(Document::from_text("Plain text"));
s.run("file.exportPdf", &json!({"path": d.join("a.pdf")})).unwrap();
s.run("file.exportPdf", &json!({"path": d.join("b.pdf"), "strict": true})).unwrap();
let (a, b) = (std::fs::read(d.join("a.pdf")).unwrap(), std::fs::read(d.join("b.pdf")).unwrap());
assert_eq!(a.len(), b.len());
std::fs::remove_dir_all(d).unwrap();
}
#[test]
fn strict_is_for_pdf_and_png_outputs_only() {
let d = dir("other");
let mut s = Session::new(Document::from_text("Hello"));
for file in ["a.docx", "a.txt", "a.odt", "a.html"] {
let path = d.join(file);
let r = s.run("file.save", &json!({"path": path, "strict": true}));
assert!(matches!(r, Err(CmdError::Params(_))), "{file}: {r:?}");
assert!(!path.exists(), "{file}");
}
let r = s.run("file.exportPng", &json!({"path": d.join("a.jpg"), "strict": true}));
assert!(matches!(r, Err(CmdError::Params(_))), "{r:?}");
std::fs::remove_dir_all(d).unwrap();
}
#[test]
fn strict_report_has_the_schema_1_shape() {
let c = uncovered();
let mut s = Session::new(Document::from_text(&format!("a{c}b{c}")));
let r = s.run("file.strictReport", &json!({})).unwrap();
assert_eq!(
r,
json!({
"schema": 1,
"strict": "failed",
"missing_glyphs": [{"cp": format!("U+{:04X}", c as u32), "count": 2}],
"missing_pictures": [],
"truncated": false,
"total_missing_glyphs": 1,
"total_missing_pictures": 0,
"glyphs_truncated": false,
"pictures_truncated": false,
})
);
let png = s.run("file.strictReport", &json!({"page": 1})).unwrap();
assert_eq!(png, r);
assert!(matches!(s.run("file.strictReport", &json!({"page": 99})), Err(CmdError::Params(_))));
let mut ok = Session::new(Document::from_text("fine"));
let r = ok.run("file.strictReport", &json!({})).unwrap();
assert_eq!(
r,
json!({"schema": 1, "strict": "passed", "missing_glyphs": [], "missing_pictures": [], "truncated": false, "total_missing_glyphs": 0, "total_missing_pictures": 0, "glyphs_truncated": false, "pictures_truncated": false})
);
}
#[test]
fn strict_report_changes_nothing() {
let c = uncovered();
let mut s = Session::new(Document::from_text(&format!("a{c}")));
let before = serde_json::to_string(&s.doc).unwrap();
s.run("file.strictReport", &json!({})).unwrap();
assert_eq!(serde_json::to_string(&s.doc).unwrap(), before);
assert!(!s.dirty);
}
fn garbage_background(d: &mut Document) {
d.media.insert("bg.bin".into(), std::sync::Arc::new(b"this is not a picture".to_vec()));
d.settings.page_picture = Some(wordcraft_doc::PagePicture { media: "bg.bin".into(), fill_type: "stretch".into(), title: String::new() });
}
#[test]
fn an_undecodable_page_background_picture_is_refused_for_pdf_and_png() {
let d = dir("background");
let mut doc = Document::from_text("Hello");
garbage_background(&mut doc);
let mut s = Session::new(doc);
let r = s.run("file.strictReport", &json!({})).unwrap();
assert_eq!(r["strict"], "failed");
assert_eq!(r["missing_pictures"][0], json!({"part": "bg.bin", "source_part": "background", "reason": "unsupported_format"}));
let msg = refused(s.run("file.exportPdf", &json!({"path": d.join("a.pdf"), "strict": true})));
assert!(msg.contains("bg.bin (background)"), "{msg}");
refused(s.run("file.exportPng", &json!({"path": d.join("a.png"), "strict": true})));
refused(s.run("file.save", &json!({"path": d.join("b.png"), "strict": true})));
assert!(!d.join("a.pdf").exists() && !d.join("a.png").exists() && !d.join("b.png").exists());
std::fs::remove_dir_all(d).unwrap();
}
#[test]
fn a_decodable_page_background_picture_passes() {
let mut png = Vec::new();
image::RgbaImage::from_pixel(4, 4, image::Rgba([200, 200, 255, 255]))
.write_to(&mut std::io::Cursor::new(&mut png), image::ImageFormat::Png)
.unwrap();
let mut doc = Document::from_text("Hello");
doc.media.insert("bg.png".into(), std::sync::Arc::new(png));
doc.settings.page_picture = Some(wordcraft_doc::PagePicture { media: "bg.png".into(), fill_type: "stretch".into(), title: String::new() });
let mut s = Session::new(doc);
assert_eq!(s.run("file.strictReport", &json!({})).unwrap()["strict"], "passed");
}
#[test]
fn a_picture_wider_than_the_pdf_draws_is_refused_for_pdf_only() {
// 40000 x 10: past the PDF's decode limit (30000), within what the page raster draws.
let mut png = Vec::new();
image::RgbaImage::from_pixel(40_000, 10, image::Rgba([9, 9, 9, 255]))
.write_to(&mut std::io::Cursor::new(&mut png), image::ImageFormat::Png)
.unwrap();
let mut doc = Document::from_text("Hello");
doc.media.insert("wide.png".into(), std::sync::Arc::new(png));
doc.settings.page_picture = Some(wordcraft_doc::PagePicture { media: "wide.png".into(), fill_type: "stretch".into(), title: String::new() });
let mut s = Session::new(doc);
assert_eq!(s.run("file.strictReport", &json!({})).unwrap()["strict"], "failed", "PDF");
assert_eq!(s.run("file.strictReport", &json!({"page": 1})).unwrap()["strict"], "passed", "PNG");
}
fn watermark(text: &str) -> Document {
let mut doc = Document::from_text("Hello");
doc.settings.watermark = Some(wordcraft_doc::Watermark { text: text.into(), ..Default::default() });
doc
}
#[test]
fn watermark_text_no_font_can_draw_is_refused_and_latin_passes() {
let cs = uncovered_many(2);
let d = dir("watermark");
let mut s = Session::new(watermark(&cs.iter().collect::<String>()));
let r = s.run("file.strictReport", &json!({})).unwrap();
assert_eq!(r["strict"], "failed");
let cps: Vec<String> = cs.iter().map(|c| format!("U+{:04X}", *c as u32)).collect();
let got: Vec<String> = r["missing_glyphs"].as_array().unwrap().iter().map(|g| g["cp"].as_str().unwrap().to_string()).collect();
assert_eq!(got, cps);
refused(s.run("file.exportPdf", &json!({"path": d.join("w.pdf"), "strict": true})));
refused(s.run("file.exportPng", &json!({"path": d.join("w.png"), "strict": true})));
assert!(!d.join("w.pdf").exists() && !d.join("w.png").exists());
let mut ok = Session::new(watermark("CONFIDENTIAL"));
assert_eq!(ok.run("file.strictReport", &json!({})).unwrap()["strict"], "passed");
ok.run("file.exportPdf", &json!({"path": d.join("ok.pdf"), "strict": true})).unwrap();
ok.run("file.exportPng", &json!({"path": d.join("ok.png"), "strict": true})).unwrap();
std::fs::remove_dir_all(d).unwrap();
}
#[test]
fn wordart_text_effects_keep_their_glyphs_in_the_check() {
let c = uncovered();
let props = wordcraft_doc::CharProps { shadow: Some(true), outline: Some(true), ..Default::default() };
let mut doc = Document::from_text("");
doc.body = vec![wordcraft_doc::para_block(wordcraft_doc::Paragraph::with_text(&format!("a{c}b"), props))].into_iter().collect();
let mut s = Session::new(doc);
let r = s.run("file.strictReport", &json!({})).unwrap();
assert_eq!(r["missing_glyphs"][0]["cp"], format!("U+{:04X}", c as u32), "{r}");
}
#[test]
fn each_list_says_for_itself_that_it_was_cut() {
let many = uncovered_many(300);
let mut doc = Document::from_text(&many.iter().collect::<String>());
garbage_background(&mut doc);
let mut s = Session::new(doc);
let r = s.run("file.strictReport", &json!({})).unwrap();
assert_eq!(r["missing_glyphs"].as_array().unwrap().len(), 256);
assert_eq!((r["truncated"].clone(), r["glyphs_truncated"].clone(), r["pictures_truncated"].clone()), (json!(true), json!(true), json!(false)));
let lines = wordcraft_engine::strict::lines_from_json(&r);
assert!(lines[0].ends_with(", …"), "{}", lines[0]);
assert!(lines[0].starts_with("strict: 300 characters cannot be drawn (first 256 listed): "), "{}", lines[0]);
assert!(!lines[1].contains('…'), "{}", lines[1]);
assert_eq!((r["total_missing_glyphs"].clone(), r["total_missing_pictures"].clone()), (json!(300), json!(1)));
}
#[test]
fn a_truncated_report_carries_totals_past_the_cap() {
let many = uncovered_many(300);
let mut doc = Document::from_text(&many.iter().collect::<String>());
for n in 0..300 {
doc.missing_pictures.push(wordcraft_doc::MissingPicture {
part: format!("word/media/gone{n}.png"),
story: "word/document.xml".into(),
reason: wordcraft_doc::MissingPictureReason::MissingPart,
});
// The same record again is one picture.
doc.missing_pictures.push(wordcraft_doc::MissingPicture {
part: format!("word/media/gone{n}.png"),
story: "word/document.xml".into(),
reason: wordcraft_doc::MissingPictureReason::MissingPart,
});
}
let mut s = Session::new(doc);
let r = s.run("file.strictReport", &json!({})).unwrap();
assert_eq!(r["missing_glyphs"].as_array().unwrap().len(), 256);
assert_eq!(r["missing_pictures"].as_array().unwrap().len(), 256);
assert_eq!(r["truncated"], true);
assert_eq!(r["total_missing_glyphs"], 300);
assert_eq!(r["total_missing_pictures"], 300);
assert_eq!(r["missing_pictures"][0]["reason"], "missing_part");
assert_eq!(r["missing_pictures"][0]["source_part"], "word/document.xml");
}
#[test]
fn strict_off_gives_the_same_png_as_before_even_with_a_missing_character() {
let c = uncovered();
let d = dir("pngsame");
let mut s = Session::new(Document::from_text(&format!("Hello {c}")));
let png: Vec<Vec<u8>> = [json!({}), json!({"strict": false})]
.into_iter()
.enumerate()
.map(|(i, mut p)| {
p["path"] = json!(d.join(format!("{i}.png")));
s.run("file.exportPng", &p).unwrap();
std::fs::read(d.join(format!("{i}.png"))).unwrap()
})
.collect();
assert!(png[0].starts_with(b"\x89PNG"));
assert_eq!(png[0], png[1]);
std::fs::remove_dir_all(d).unwrap();
}
+100
View File
@@ -0,0 +1,100 @@
//! Pictures a layout draws that the exporters can't decode (#6005): a DOCX opened, laid out, and
//! asked which of its pictures would be grey boxes, and in which story.
use std::io::Write;
use wordcraft_doc::{Document, MissingPictureReason};
use wordcraft_layout::display::DisplayOptions;
use wordcraft_layout::{LayoutCache, LayoutOptions, StrictOutput, layout, media_check, undecodable_pictures};
const NS: &str = r#"xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships" xmlns:wp="http://schemas.openxmlformats.org/drawingml/2006/wordprocessingDrawing" xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main" xmlns:pic="http://schemas.openxmlformats.org/drawingml/2006/picture""#;
const REL: &str = "http://schemas.openxmlformats.org/officeDocument/2006/relationships";
fn png() -> Vec<u8> {
let mut out = Vec::new();
image::RgbaImage::from_pixel(4, 4, image::Rgba([20, 90, 200, 255]))
.write_to(&mut std::io::Cursor::new(&mut out), image::ImageFormat::Png)
.unwrap();
out
}
fn pic(rid: &str) -> String {
format!(
r#"<w:r><w:drawing><wp:inline><wp:extent cx="508000" cy="508000"/><wp:docPr id="1" name="P"/><a:graphic><a:graphicData uri="http://schemas.openxmlformats.org/drawingml/2006/picture"><pic:pic><pic:nvPicPr><pic:cNvPr id="0" name="p"/><pic:cNvPicPr/></pic:nvPicPr><pic:blipFill><a:blip r:embed="{rid}"/></pic:blipFill><pic:spPr><a:xfrm><a:off x="0" y="0"/><a:ext cx="508000" cy="508000"/></a:xfrm><a:prstGeom prst="rect"/></pic:spPr></pic:pic></a:graphicData></a:graphic></wp:inline></w:drawing></w:r>"#
)
}
/// A document with one header picture (`header_media`) and one body picture (`body_media`), the
/// two parts' bytes as given.
fn docx(header_media: &[u8], body_media: &[u8]) -> Vec<u8> {
let doc = format!(
r#"<w:document {NS}><w:body><w:p>{}<w:r><w:t>text</w:t></w:r></w:p><w:sectPr><w:headerReference w:type="default" r:id="rIdH"/></w:sectPr></w:body></w:document>"#,
pic("rIdB")
);
let rels = format!(
r#"<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships"><Relationship Id="rId1" Type="{REL}/officeDocument" Target="word/document.xml"/></Relationships>"#
);
let drels = format!(
r#"<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships"><Relationship Id="rIdH" Type="{REL}/header" Target="header2.xml"/><Relationship Id="rIdB" Type="{REL}/image" Target="media/body.png"/></Relationships>"#
);
let hrels = format!(
r#"<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships"><Relationship Id="rIdI" Type="{REL}/image" Target="media/image2.bin"/></Relationships>"#
);
let types = r#"<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types"><Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/><Default Extension="png" ContentType="image/png"/><Default Extension="bin" ContentType="application/octet-stream"/><Override PartName="/word/document.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml"/></Types>"#;
let mut zip = zip::ZipWriter::new(std::io::Cursor::new(Vec::new()));
let hdr = format!(r#"<w:hdr {NS}><w:p>{}</w:p></w:hdr>"#, pic("rIdI"));
for (name, bytes) in [
("[Content_Types].xml", types.as_bytes()),
("_rels/.rels", rels.as_bytes()),
("word/document.xml", doc.as_bytes()),
("word/_rels/document.xml.rels", drels.as_bytes()),
("word/header2.xml", hdr.as_bytes()),
("word/_rels/header2.xml.rels", hrels.as_bytes()),
("word/media/image2.bin", header_media),
("word/media/body.png", body_media),
] {
zip.start_file(name, zip::write::SimpleFileOptions::default()).unwrap();
zip.write_all(bytes).unwrap();
}
zip.finish().unwrap().into_inner()
}
fn report(d: &Document, pages: Option<std::ops::RangeInclusive<usize>>) -> Vec<(String, String, MissingPictureReason)> {
let l = layout(d, &mut LayoutCache::new(), &LayoutOptions::default());
let (list, truncated, total) = undecodable_pictures(d, &l, pages, &DisplayOptions::default(), StrictOutput::Pdf);
assert!(!truncated);
assert_eq!(total, list.len() as u64);
list.into_iter().map(|m| (m.part, m.story, m.reason)).collect()
}
#[test]
fn a_header_picture_of_corrupt_bytes_is_reported_with_its_part_and_story() {
let d = wordcraft_docx::read(&docx(b"corrupt bytes, not a picture", &png())).unwrap();
assert_eq!(report(&d, None), [("word/media/image2.bin".into(), "word/header2.xml".into(), MissingPictureReason::UnsupportedFormat)]);
}
#[test]
fn a_truncated_picture_is_undecodable_not_unsupported() {
let mut cut = png();
cut.truncate(cut.len() - 20);
let d = wordcraft_docx::read(&docx(&png(), &cut)).unwrap();
assert_eq!(report(&d, None), [("word/media/body.png".into(), "word/document.xml".into(), MissingPictureReason::Undecodable)]);
}
#[test]
fn a_real_png_in_a_header_is_not_reported() {
let d = wordcraft_docx::read(&docx(&png(), &png())).unwrap();
assert_eq!(report(&d, None), []);
}
#[test]
fn only_the_selected_pages_are_checked() {
let d = wordcraft_docx::read(&docx(b"corrupt", &png())).unwrap();
assert_eq!(report(&d, Some(0..=0)).len(), 1);
assert_eq!(report(&d, Some(5..=9)), []);
}
#[test]
fn the_check_names_what_the_exporters_draw() {
assert_eq!(media_check(&png(), StrictOutput::Pdf), Ok(()));
assert!(media_check(b"", StrictOutput::Pdf).is_err());
assert!(media_check(&[1, 0, 0, 0, 88, 0, 0, 0], StrictOutput::Pdf).is_err(), "a metafile that doesn't parse is a grey box");
}
+93
View File
@@ -194,6 +194,87 @@ pub fn decode(bytes: &[u8], budget: u64) -> Result<DynamicImage, Refusal> {
probe(bytes)?.decode(budget)
}
/// Why a picture can't be drawn, as strict export reports it (#6005).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum DrawFailure {
/// A picture format we know, but it doesn't decode (damaged, truncated, too large).
Undecodable,
/// Bytes that are not a picture format we draw.
UnsupportedFormat,
}
impl DrawFailure {
/// The words reports use.
pub fn label(self) -> &'static str {
match self {
DrawFailure::Undecodable => "undecodable",
DrawFailure::UnsupportedFormat => "unsupported format",
}
}
}
impl From<&Refusal> for DrawFailure {
fn from(r: &Refusal) -> Self {
match r {
Refusal::NotAPicture | Refusal::Unsupported(_) => DrawFailure::UnsupportedFormat,
Refusal::TooLarge { .. } | Refusal::Broken(_) => DrawFailure::Undecodable,
}
}
}
/// Decode a raster picture for drawing: the one decision the PDF export, the raster renderer and
/// strict export's check all take. The decoder is held to `max_side` pixels a side and the header
/// is charged against `budget` bytes before anything is allocated. A picture with no pixels or a
/// side past `max_side` is refused.
pub fn decode_for_drawing(bytes: &[u8], budget: u64, max_side: u32) -> Result<DynamicImage, Refusal> {
let mut limits = image::Limits::default();
limits.max_image_width = Some(max_side);
limits.max_image_height = Some(max_side);
let reader = || {
let mut r = image::ImageReader::new(Cursor::new(bytes)).with_guessed_format().map_err(|_| Refusal::NotAPicture)?;
r.limits(limits.clone());
Ok::<_, Refusal>(r)
};
// Header-only size check before anything is allocated: the decoder's buffer at its own pixel
// size (8 bytes for 16-bit RGBA) and the RGBA conversion are charged.
let (hw, hh, native) = {
let d = reader()?.into_decoder().map_err(broken)?;
let (w, h) = d.dimensions();
(w, h, u64::from(d.color_type().bytes_per_pixel()))
};
if !crate::decode_peak_fits(hw, hh, native, budget) {
return Err(Refusal::TooLarge { width: hw, height: hh });
}
let img = reader()?.decode().map_err(broken)?;
let (w, h) = (img.width(), img.height());
if w == 0 || h == 0 || w > max_side || h > max_side {
return Err(Refusal::Broken("it has no pixels or a side past the limit".into()));
}
Ok(img)
}
/// Largest page-background picture the page raster draws, in pixels (hostile files).
pub const MAX_BACKGROUND_PIXELS: u64 = 64 * 1024 * 1024;
/// The pixel size of a page-background picture the page raster draws (Design > Page Color > Fill
/// Effects), or None when it leaves the background out: not a raster picture it can read (a
/// metafile is not) or one with no pixels or more than [`MAX_BACKGROUND_PIXELS`]. The raster and
/// the strict check both ask this, so they can't drift.
pub fn background_size(bytes: &[u8]) -> Option<(u32, u32)> {
let (w, h) = image::ImageReader::new(Cursor::new(bytes)).with_guessed_format().ok()?.into_dimensions().ok()?;
(w > 0 && h > 0 && u64::from(w) * u64::from(h) <= MAX_BACKGROUND_PIXELS).then_some((w, h))
}
/// Whether a raster picture decodes for drawing, by the rule the exporters use ([`probe`] within
/// [`crate::MAX_DECODED_BYTES`], then [`decode_for_drawing`] with `max_side`).
pub fn check_for_drawing(bytes: &[u8], max_side: u32) -> Result<(), DrawFailure> {
let fail = |r: Refusal| DrawFailure::from(&r);
if !probe(bytes).map_err(fail)?.fits(crate::MAX_DECODED_BYTES) {
return Err(DrawFailure::Undecodable);
}
decode_for_drawing(bytes, crate::MAX_DECODED_BYTES, max_side).map(|_| ()).map_err(fail)
}
/// Bytes decoding a `width` × `height` picture holds at its peak: the decoder's output (`native`
/// bytes a pixel) beside what the decoder keeps while it decodes (`held`), then the output beside
/// its RGBA8 conversion, then that beside a premultiplied copy. None on overflow.
@@ -340,6 +421,18 @@ mod tests {
out
}
#[test]
fn drawing_check_tells_unsupported_from_undecodable() {
let png = encode(image::RgbaImage::new(4, 4), ImageFormat::Png);
assert_eq!(check_for_drawing(&png, 100), Ok(()));
assert_eq!(check_for_drawing(b"junk", 100), Err(DrawFailure::UnsupportedFormat));
assert_eq!(check_for_drawing(&png[..png.len() - 20], 100), Err(DrawFailure::Undecodable));
// A side past the limit is refused, by the same function the exporters draw with.
assert_eq!(check_for_drawing(&png, 3), Err(DrawFailure::Undecodable));
assert!(decode_for_drawing(&png, crate::MAX_DECODED_BYTES, 100).is_ok());
assert_eq!(DrawFailure::Undecodable.label(), "undecodable");
}
/// A PNG whose header declares `w` × `h` pixels, with no pixel data.
fn png_header(w: u32, h: u32) -> Vec<u8> {
let mut png = encode(image::RgbaImage::new(1, 1), ImageFormat::Png);
+2 -3
View File
@@ -848,7 +848,7 @@ fn section_break_mark(start: SectionStart, x0: f32, x1: f32, base: f32, size: f3
fn equation(items: &[MItem], x: f32, base: f32, alpha: f32, screen: bool, out: &mut Vec<Draw>) {
for it in items {
match it {
MItem::Glyphs { face, size, color, synth_bold, synth_italic, glyphs, text } => out.push(Draw::Glyphs {
MItem::Glyphs { face, size, color, synth_bold, synth_italic, glyphs, text, ranges } => out.push(Draw::Glyphs {
face: *face,
size: *size,
glyphs: glyphs.iter().map(|(g, gx, gy)| (*g, x + gx, base - gy)).collect(),
@@ -858,8 +858,7 @@ fn equation(items: &[MItem], x: f32, base: f32, alpha: f32, screen: bool, out: &
synth_italic: *synth_italic,
text: text.clone(),
link: None,
// Equation text maps back through the font (no per-glyph ranges from the math layout).
ranges: Vec::new(),
ranges: ranges.clone(),
text_bg: None,
x_scale: 1.0,
}),
+2 -1
View File
@@ -307,6 +307,7 @@ fn text_glyphs(doc: &Document, r: Rect, it: &GraphicItem, alpha: f32, k: f32) ->
let mut out = Vec::with_capacity(runs.len());
for (face, s, shaped) in runs {
let k = scale(face);
let ranges = crate::missing::cluster_ranges(s, &shaped);
let mut glyphs = Vec::with_capacity(shaped.len());
for g in &shaped {
glyphs.push((g.gid, pen + g.x_offset as f32 * k, baseline - g.y_offset as f32 * k));
@@ -324,7 +325,7 @@ fn text_glyphs(doc: &Document, r: Rect, it: &GraphicItem, alpha: f32, k: f32) ->
synth_italic: false,
text: s.to_string(),
link: None,
ranges: Vec::new(),
ranges,
text_bg: None,
x_scale: 1.0,
});
+4
View File
@@ -58,10 +58,12 @@ mod ctx;
mod floats;
mod flow;
mod headers;
mod missing;
mod notes;
mod page;
mod pass;
mod placed;
mod strict;
mod substituted;
mod table_flow;
@@ -82,6 +84,7 @@ use headers::MAX_HEADER_ITEMS;
#[cfg(test)]
use headers::placed_cost;
use headers::placed_cost_by;
pub use missing::{MAX_MISSING_GLYPHS, MissingGlyphs, metafile_text_run, missing_watermark_glyphs, watermark_run};
pub use notes::DECOR_STORY;
pub(crate) use notes::NOTE_SEP_W;
use notes::{MAX_NOTE_PASSES, footnote_pages};
@@ -91,6 +94,7 @@ use page::item_bottom;
use pass::layout_pass;
pub use pass::now_ms;
pub use placed::{HfSlot, Placed, turn_point, turn_rect, unturn_point};
pub use strict::{MAX_PDF_PICTURE_SIDE, MAX_PNG_PICTURE_SIDE, StrictOutput, media_check, undecodable_pictures};
pub use substituted::MAX_SUBSTITUTED;
/// How the document is viewed.
+2
View File
@@ -41,6 +41,8 @@ pub enum MItem {
/// Glyph id, x, y (baseline, up).
glyphs: Vec<(u32, f32, f32)>,
text: String,
/// Byte range of `text` each glyph shows (empty when `text` is: a glyph picked by id).
ranges: Vec<std::ops::Range<usize>>,
},
/// A filled rectangle; `y` is its bottom edge.
Rect {
+30 -3
View File
@@ -103,7 +103,16 @@ impl Ctx {
b.glyph = Some((face, g, attach));
}
if !glyphs.is_empty() {
b.items.push(MItem::Glyphs { face, size, color, synth_bold: synth.0, synth_italic: synth.1, glyphs, text: text.to_string() });
b.items.push(MItem::Glyphs {
face,
size,
color,
synth_bold: synth.0,
synth_italic: synth.1,
glyphs,
text: text.to_string(),
ranges: crate::missing::cluster_ranges(text, &shaped),
});
}
b
}
@@ -132,7 +141,16 @@ impl Ctx {
if let Some(t) = &self.t {
b.ic = t.italics.get(&gid).copied().unwrap_or(0.0) * k;
}
b.items.push(MItem::Glyphs { face, size, color, synth_bold: false, synth_italic: false, glyphs: vec![(gid, 0.0, 0.0)], text: String::new() });
b.items.push(MItem::Glyphs {
face,
size,
color,
synth_bold: false,
synth_italic: false,
glyphs: vec![(gid, 0.0, 0.0)],
text: String::new(),
ranges: Vec::new(),
});
b
}
@@ -227,7 +245,16 @@ impl Ctx {
} else {
b.w = pos * k;
}
b.items.push(MItem::Glyphs { face: self.face, size, color: self.color, synth_bold: false, synth_italic: false, glyphs, text: String::new() });
b.items.push(MItem::Glyphs {
face: self.face,
size,
color: self.color,
synth_bold: false,
synth_italic: false,
glyphs,
text: String::new(),
ranges: Vec::new(),
});
Some(b)
}
+309
View File
@@ -0,0 +1,309 @@
//! The characters the output would draw as an empty box ([`DocLayout::missing_glyphs`]): glyph
//! id 0 in a [`Draw::Glyphs`] of the display list, traced back to its character through the
//! run's `text` and per-glyph `ranges`. Backs strict export (#6005).
use std::collections::{BTreeMap, HashMap};
use std::ops::{Range, RangeInclusive};
use wordcraft_doc::Document;
use wordcraft_fonts::ShapedGlyph;
use crate::DocLayout;
use crate::display::{DisplayOptions, Draw, page_display};
use crate::para::draws_something;
/// Most distinct code points [`DocLayout::missing_glyphs`] lists: a hostile file full of unmapped
/// characters can't make the report unbounded.
pub const MAX_MISSING_GLYPHS: usize = 256;
/// The missing glyphs of a render: characters it draws that no face in the font fallback chain can
/// draw (glyph id 0, an empty box).
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct MissingGlyphs {
/// (code point, times drawn), sorted by code point, at most [`MAX_MISSING_GLYPHS`] long.
pub chars: Vec<(char, u64)>,
/// More than [`MAX_MISSING_GLYPHS`] distinct code points were missing: the rest aren't listed.
pub truncated: bool,
/// How many distinct code points were missing, those past the cap included.
pub total: usize,
/// Every distinct code point found, kept only once the cap was passed (136 KiB), so merging
/// the watermark's can still count the distinct ones.
pub(crate) seen: Option<Box<CodePointSet>>,
}
impl MissingGlyphs {
/// Nothing is missing.
pub fn is_empty(&self) -> bool {
self.chars.is_empty() && !self.truncated
}
/// A list of `chars` that fits under the cap: its total is its length.
pub fn from_listed(chars: Vec<(char, u64)>) -> Self {
let total = chars.len();
Self { chars, truncated: false, total, seen: None }
}
/// Add `other` (the watermark's characters): counts of a shared code point are summed, new
/// ones are listed while the cap allows, and the total counts each distinct one once.
pub fn merge(&mut self, other: MissingGlyphs) {
let mut all: BTreeMap<char, u64> = std::mem::take(&mut self.chars).into_iter().collect();
let mut seen = self.seen.take();
for (c, n) in other.chars {
if let Some(count) = all.get_mut(&c) {
*count = count.saturating_add(n);
} else if let Some(set) = seen.as_mut() {
set.insert(c);
} else if all.len() < MAX_MISSING_GLYPHS {
all.insert(c, n);
} else {
let mut set = CodePointSet::of(all.keys().copied());
set.insert(c);
seen = Some(Box::new(set));
}
}
if let Some(theirs) = other.seen {
seen.get_or_insert_with(|| Box::new(CodePointSet::of(all.keys().copied()))).union(&theirs);
}
self.truncated |= other.truncated || seen.is_some();
self.total = seen.as_ref().map_or(all.len(), |s| s.len());
self.chars = all.into_iter().collect();
self.seen = seen;
}
}
/// A set of code points as a bitset over the whole code point space: bounded at 136 KiB however
/// many distinct characters a hostile file holds.
#[derive(Clone, Debug, PartialEq, Eq)]
pub(crate) struct CodePointSet(Vec<u64>);
impl CodePointSet {
fn empty() -> Self {
Self(vec![0; 0x11_0000 / 64])
}
fn of(chars: impl Iterator<Item = char>) -> Self {
let mut set = Self::empty();
for c in chars {
set.insert(c);
}
set
}
fn insert(&mut self, c: char) {
let n = c as usize;
if let Some(word) = self.0.get_mut(n / 64) {
*word |= 1 << (n % 64);
}
}
fn union(&mut self, other: &Self) {
for (a, b) in self.0.iter_mut().zip(&other.0) {
*a |= b;
}
}
fn len(&self) -> usize {
self.0.iter().map(|w| w.count_ones() as usize).sum()
}
}
impl DocLayout {
/// The characters drawn as glyph id 0 on `pages` (0-based, inclusive; `None` is every page),
/// from the display list the PDF and PNG exports draw with `opts`: body, headers, footers,
/// text boxes, notes, table cells, charts, diagrams and equations. A character counts once per
/// glyph drawn for it. Default-ignorable and control characters never count (they draw
/// nothing), and a Private Use Area character counts only if it really drew as glyph 0.
/// Pages past the end are ignored.
pub fn missing_glyphs(&self, doc: &Document, opts: &DisplayOptions, pages: Option<RangeInclusive<usize>>) -> MissingGlyphs {
let mut found = Found::default();
for (i, page) in self.pages.iter().enumerate() {
if pages.as_ref().is_some_and(|r| !r.contains(&i)) {
continue;
}
for d in page_display(doc, page, opts) {
found.walk(doc, &d);
}
// The page picture is drawn behind every page, outside the display list.
if let Some(pic) = &doc.settings.page_picture {
found.metafile_text(doc, &pic.media);
}
}
found.finish()
}
}
/// How the watermark text is shaped for output, as the PDF export and the page raster both do:
/// its first `max_chars` characters in the font its name resolves to (no fallback face), so a
/// character that font lacks shapes to glyph id 0. Strict export checks these same glyphs
/// ([`missing_watermark_glyphs`]), so the check and the exporters can't drift. `None` for a
/// blank watermark.
pub fn watermark_run(wm: &wordcraft_doc::Watermark, max_chars: usize) -> Option<(wordcraft_fonts::FaceRef, String, Vec<ShapedGlyph>)> {
if wm.text.trim().is_empty() {
return None;
}
let face = wordcraft_fonts::word::resolve(&wm.font, false, false).face;
let text: String = wm.text.chars().take(max_chars).collect();
let shaped = wordcraft_fonts::shape(&face, &text, &[], |c| c);
Some((face, text, shaped))
}
/// How a metafile text run is shaped for output, as the PDF export and the page raster both do:
/// in the face its font name resolves to (no fallback face), so a character that face lacks shapes
/// to glyph id 0. Strict export checks these same glyphs, so the check and the exporters can't
/// drift.
pub fn metafile_text_run(t: &wordcraft_metafile::Text) -> (wordcraft_fonts::FaceRef, Vec<ShapedGlyph>) {
let face = wordcraft_fonts::word::resolve(&t.font, t.bold, t.italic).face;
let shaped = wordcraft_fonts::shape(&face, &t.text, &[], |c| c);
(face, shaped)
}
/// The characters of the watermark that draw as an empty box (glyph id 0), counted as
/// [`DocLayout::missing_glyphs`] counts them; sorted by code point. The watermark is on every
/// page, so it counts once.
pub fn missing_watermark_glyphs(wm: &wordcraft_doc::Watermark, max_chars: usize) -> MissingGlyphs {
let mut found = Found::default();
if let Some((face, text, shaped)) = watermark_run(wm, max_chars) {
let ranges = cluster_ranges(&text, &shaped);
found.run(&face, shaped.iter().map(|g| g.gid), &text, &ranges);
}
found.finish()
}
#[derive(Default)]
struct Found {
/// The text runs of each metafile picture met, by media key, shaped once: (face, text, ranges, glyph ids).
metafiles: HashMap<String, Vec<MetaRun>>,
counts: BTreeMap<char, u64>,
truncated: bool,
/// Every distinct code point, started when the list passes its cap.
over: Option<Box<CodePointSet>>,
}
/// A metafile text run as drawn: its face, text, per-glyph ranges and glyph ids.
struct MetaRun {
face: wordcraft_fonts::FaceRef,
text: String,
ranges: Vec<Range<usize>>,
gids: Vec<u32>,
}
impl Found {
/// The text of the metafile picture `key` (nothing for a picture that is not a metafile, does
/// not parse or has no size: the exporters draw none of it), counted once per call.
fn metafile_text(&mut self, doc: &Document, key: &str) {
if !self.metafiles.contains_key(key) {
let runs = doc.media.get(key).map(|bytes| meta_runs(bytes)).unwrap_or_default();
self.metafiles.insert(key.to_string(), runs);
}
let Some(runs) = self.metafiles.remove(key) else { return };
for r in &runs {
self.run(&r.face, r.gids.iter().copied(), &r.text, &r.ranges);
}
self.metafiles.insert(key.to_string(), runs);
}
fn finish(self) -> MissingGlyphs {
let total = self.over.as_ref().map_or(self.counts.len(), |s| s.len());
MissingGlyphs { chars: self.counts.into_iter().collect(), truncated: self.truncated, total, seen: self.over }
}
fn walk(&mut self, doc: &Document, d: &Draw) {
match d {
Draw::Glyphs { face, glyphs, text, ranges, .. } => self.run(face, glyphs.iter().map(|g| g.0), text, ranges),
Draw::Image { media, .. } => self.metafile_text(doc, media),
Draw::Figure { draws: items, .. } | Draw::Turned { items, .. } | Draw::Rotated { items, .. } | Draw::Clip { items, .. } => {
for d in items {
self.walk(doc, d);
}
}
_ => {}
}
}
/// A run of glyphs of `face` (`gids`) showing `text`, glyph `k` showing `ranges[k]`.
fn run(&mut self, face: &wordcraft_fonts::FaceRef, gids: impl Iterator<Item = u32>, text: &str, ranges: &[Range<usize>]) {
let glyph_count = gids.size_hint().1.unwrap_or(0);
let one_each = ranges.is_empty() && text.chars().count() == glyph_count;
let mut chars = text.chars();
for (k, gid) in gids.enumerate() {
// A run without ranges (a list label) maps one character to one glyph, when the
// counts agree; else it can't be traced and isn't counted.
let own = chars.next();
if gid != 0 {
continue;
}
let shown: Vec<char> = if one_each {
own.into_iter().collect()
} else {
ranges.get(k).and_then(|r| text.get(r.clone())).map(|t| t.chars().collect()).unwrap_or_default()
};
let drawn: Vec<char> = shown.into_iter().filter(|c| draws_something(*c)).collect();
// A cluster of several characters drew one empty box: it is the one the face
// lacks, not the letter beside it.
let lacking: Vec<char> = drawn.iter().copied().filter(|c| !face.covers(*c)).collect();
for c in if lacking.is_empty() { drawn } else { lacking } {
self.count(c);
}
}
}
fn count(&mut self, c: char) {
if let Some(n) = self.counts.get_mut(&c) {
*n = n.saturating_add(1);
} else if let Some(set) = self.over.as_mut() {
set.insert(c);
} else if self.counts.len() < MAX_MISSING_GLYPHS {
self.counts.insert(c, 1);
} else {
self.truncated = true;
let mut set = CodePointSet::of(self.counts.keys().copied());
set.insert(c);
self.over = Some(Box::new(set));
}
}
}
/// The text runs of a metafile picture that the exporters draw (a run whose layout fails is not).
fn meta_runs(bytes: &[u8]) -> Vec<MetaRun> {
if !wordcraft_metafile::is_metafile(bytes) {
return Vec::new();
}
let Ok(pic) = wordcraft_metafile::parse(bytes) else { return Vec::new() };
if !pic.has_size() {
return Vec::new();
}
let mut out = Vec::new();
for item in &pic.items {
let wordcraft_metafile::Item::Text(t) = item else { continue };
let (face, shaped) = metafile_text_run(t);
let glyphs: Vec<_> = shaped.iter().map(|g| (g.cluster, f64::from(g.x_advance), f64::from(g.x_offset), f64::from(g.y_offset))).collect();
if t.layout(face.upem, face.ascent, face.descent, &glyphs).is_none() {
continue;
}
let ranges = cluster_ranges(&t.text, &shaped);
out.push(MetaRun { gids: shaped.iter().map(|g| g.gid).collect(), text: t.text.clone(), ranges, face });
}
out
}
/// The byte range of `text` each of `shaped` shows: from its cluster to the next cluster start in
/// the text (or its end), whichever order the glyphs are in. A ligature's glyph covers all its
/// characters; the glyphs of one cluster share its range. Ranges always lie on character
/// boundaries of `text`.
pub(crate) fn cluster_ranges(text: &str, shaped: &[ShapedGlyph]) -> Vec<Range<usize>> {
let mut starts: Vec<usize> = shaped.iter().map(|g| g.cluster.min(text.len())).filter(|b| text.is_char_boundary(*b)).collect();
starts.sort_unstable();
starts.dedup();
shaped
.iter()
.map(|g| {
let from = g.cluster.min(text.len());
let from = if text.is_char_boundary(from) { from } else { (0..from).rev().find(|b| text.is_char_boundary(*b)).unwrap_or(0) };
let to = match starts.binary_search(&from) {
Ok(i) => starts.get(i + 1).copied().unwrap_or(text.len()),
Err(_) => text.len(),
};
from..to.max(from)
})
.collect()
}
+201
View File
@@ -0,0 +1,201 @@
//! Pictures a layout draws that the exporters cannot decode, for strict export (#6005).
//!
//! The PDF export, the raster renderer and this check decide whether a picture is drawn the same
//! way (`wordcraft_geom::picture::decode_for_drawing`, and the metafile parser): what fails here
//! is what shows as a grey box there.
use std::collections::{HashMap, HashSet};
use std::ops::RangeInclusive;
use wordcraft_doc::{Document, MAX_MISSING_PICTURES, MissingPicture, MissingPictureReason, StoryRef, picture_hash};
use wordcraft_geom::picture::DrawFailure;
use crate::display::{DisplayOptions, Draw, ItemDisplay};
use crate::{DocLayout, Placed};
/// The widest or tallest picture the PDF export draws, pixels (its decoder limit).
pub const MAX_PDF_PICTURE_SIDE: u32 = 30_000;
/// The widest or tallest picture the page raster draws, pixels (what a `Pixmap` holds).
pub const MAX_PNG_PICTURE_SIDE: u32 = u16::MAX as u32;
/// The output a strict export checks, which decides what its exporter can draw.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum StrictOutput {
Pdf,
Png,
}
impl StrictOutput {
/// The widest or tallest picture this output decodes.
pub fn max_picture_side(self) -> u32 {
match self {
StrictOutput::Pdf => MAX_PDF_PICTURE_SIDE,
StrictOutput::Png => MAX_PNG_PICTURE_SIDE,
}
}
}
/// Whether `bytes` are a picture `output` draws: a metafile that parses, or a raster picture that
/// decodes within the decode budget. The same decision `wordcraft-pdf` and `wordcraft-render`
/// take, so a picture that fails here is drawn as a grey box (or not at all) by that output.
pub fn media_check(bytes: &[u8], output: StrictOutput) -> Result<(), DrawFailure> {
if wordcraft_metafile::is_metafile(bytes) && wordcraft_metafile::parse(bytes).is_ok_and(|p| p.has_size()) {
return Ok(());
}
wordcraft_geom::picture::check_for_drawing(bytes, output.max_picture_side())
}
/// The pictures drawn on `pages` (0-based, `None` for every page) that fail [`media_check`] for
/// `output`, the page's background picture included (named `background`), one
/// per picture and story, at most [`MAX_MISSING_PICTURES`]; the flag says more were found, and the
/// count is how many distinct ones there were in all (saturating).
///
/// `part` is the part the bytes were read from when the document knows it (a DOCX opened by the
/// reader), else the media key. `story` is the source part of the header, footer, note or comment
/// the picture is in (`word/header2.xml`), `word/document.xml` for the body; where the document
/// holds no source names (a document not opened from a DOCX) it is `header`, `footer`,
/// `footnote`, `endnote`, `comment`, `text box` or `body`. A picture in a text box or a shape is
/// named for the header, footer or body area it is placed in, not for the box.
pub fn undecodable_pictures(
doc: &Document,
layout: &DocLayout,
pages: Option<RangeInclusive<usize>>,
opts: &DisplayOptions,
output: StrictOutput,
) -> (Vec<MissingPicture>, bool, u64) {
let mut out: Vec<MissingPicture> = Vec::new();
let mut truncated = false;
let mut total: u64 = 0;
// Hashes of the distinct ones past the cap, so the total counts each once.
let mut unlisted: HashSet<u64> = HashSet::new();
// What `out` holds, for an O(1) test of "already listed".
let mut listed: HashSet<MissingPicture> = HashSet::new();
let mut verdicts: HashMap<String, Option<DrawFailure>> = HashMap::new();
let range = pages.unwrap_or(0..=usize::MAX);
for (n, page) in layout.pages.iter().enumerate() {
if !range.contains(&n) {
continue;
}
let shown = ItemDisplay::new(doc, page);
let regions: [(&[Placed], Region); 3] =
[(&page.header, Region::Header(page.header_story)), (&page.footer, Region::Footer(page.footer_story)), (&page.items, Region::Body)];
for (items, region) in regions {
for it in items {
let mut media: Vec<String> = Vec::new();
let draws = shown.draws(it, opts);
collect_media(&draws, &mut media);
for key in media {
let verdict = *verdicts.entry(key.clone()).or_insert_with(|| match doc.media.get(&key) {
Some(bytes) => media_check(bytes, output).err(),
None => Some(DrawFailure::Undecodable),
});
let Some(failure) = verdict else { continue };
let reason = match failure {
DrawFailure::Undecodable => MissingPictureReason::Undecodable,
DrawFailure::UnsupportedFormat => MissingPictureReason::UnsupportedFormat,
};
let part = doc.missing_pictures.media_parts.get(&key).cloned().unwrap_or(key);
let item = MissingPicture { part, story: story_name(doc, region, it), reason };
if listed.contains(&item) {
continue;
}
if out.len() >= MAX_MISSING_PICTURES {
truncated = true;
if unlisted.insert(picture_hash(&item)) {
total = total.saturating_add(1);
}
continue;
}
total = total.saturating_add(1);
listed.insert(item.clone());
out.push(item);
}
}
}
}
// The page picture is drawn behind every page, outside the display list.
if let Some(pic) = &doc.settings.page_picture
&& layout.pages.iter().enumerate().any(|(n, _)| range.contains(&n))
{
// The page raster leaves out a background it can't read as a raster picture (a metafile),
// which the PDF draws as vector.
let failure = match doc.media.get(&pic.media) {
Some(bytes) => media_check(bytes, output).err().or_else(|| {
(output == StrictOutput::Png && wordcraft_geom::picture::background_size(bytes).is_none()).then_some(DrawFailure::UnsupportedFormat)
}),
None => Some(DrawFailure::Undecodable),
};
if let Some(failure) = failure {
let reason = match failure {
DrawFailure::Undecodable => MissingPictureReason::Undecodable,
DrawFailure::UnsupportedFormat => MissingPictureReason::UnsupportedFormat,
};
let part = doc.missing_pictures.media_parts.get(&pic.media).cloned().unwrap_or_else(|| pic.media.clone());
let item = MissingPicture { part, story: "background".into(), reason };
if !listed.contains(&item) {
if out.len() >= MAX_MISSING_PICTURES {
truncated = true;
if unlisted.insert(picture_hash(&item)) {
total = total.saturating_add(1);
}
} else {
total = total.saturating_add(1);
listed.insert(item.clone());
out.push(item);
}
}
}
}
(out, truncated, total)
}
#[derive(Clone, Copy)]
enum Region {
Header(Option<u32>),
Footer(Option<u32>),
Body,
}
fn collect_media(draws: &[Draw], out: &mut Vec<String>) {
for d in draws {
match d {
Draw::Image { media, .. } => out.push(media.clone()),
Draw::Figure { draws, .. } => collect_media(draws, out),
Draw::Turned { items, .. } | Draw::Rotated { items, .. } | Draw::Clip { items, .. } => collect_media(items, out),
_ => {}
}
}
}
/// The story a placed item is in, named as [`undecodable_pictures`] says.
fn story_name(doc: &Document, region: Region, it: &Placed) -> String {
let names = &doc.missing_pictures;
let known = |id: Option<u32>| id.and_then(|id| names.part_names.get(&id)).map(|n| n.to_string());
match region {
Region::Header(id) => known(id).unwrap_or_else(|| "header".into()),
Region::Footer(id) => known(id).unwrap_or_else(|| "footer".into()),
Region::Body => {
let story = match it {
Placed::Lines { story, .. } | Placed::Image { story, .. } | Placed::Graphic { story, .. } => *story,
_ => StoryRef::Body,
};
match story {
StoryRef::Body if names.main.is_empty() => "body".into(),
StoryRef::Body => names.main.clone(),
StoryRef::Part(id) => known(Some(id)).unwrap_or_else(|| {
use wordcraft_doc::PartKind::*;
match doc.parts.get(&id).map(|p| p.kind) {
Some(Header) => "header",
Some(Footer) => "footer",
Some(Footnote) => "footnote",
Some(Endnote) => "endnote",
Some(Comment) => "comment",
Some(TextBox) | None => "text box",
}
.into()
}),
}
}
}
}
+1
View File
@@ -13,6 +13,7 @@ mod float_top;
mod grid;
mod ligature;
pub(crate) mod linear;
mod missing_glyphs;
mod nbsp;
mod nested_cant_split;
mod numeral;
+244
View File
@@ -0,0 +1,244 @@
//! [`DocLayout::missing_glyphs`] (strict export, #6005): glyph id 0 in the display list, named by
//! its character, wherever the text is drawn.
use std::sync::Arc;
use wordcraft_doc::graphic::{Graphic, GraphicItem, GraphicKind, TextAlign};
use wordcraft_doc::para::InlineObject;
use wordcraft_doc::props::Rgb;
use wordcraft_doc::{Block, Document, Path, Pos, StoryRef, Table};
use wordcraft_fonts::FontDb;
use super::{footnote, lay, page_float, text_box_at};
use crate::display::DisplayOptions;
use crate::{DocLayout, MAX_MISSING_GLYPHS, MissingGlyphs};
/// Code points in unassigned plane 4 that no face of the host covers, checked against the body
/// font and the whole fallback chain (so the test never guesses).
fn uncovered(n: usize) -> Vec<char> {
crate::test_fonts();
let db = FontDb::global();
let primary = wordcraft_fonts::word::resolve(&Document::default().settings.minor_font, false, false).face;
let out: Vec<char> = (0x40000u32..0x4FFF0)
.filter_map(char::from_u32)
.filter(|c| {
!primary.covers(*c) && db.fallback_styled(*c, primary.id(), false, false).is_none() && db.fallback_for(*c, primary.id()).is_none()
})
.take(n)
.collect();
assert_eq!(out.len(), n, "the host has faces covering all of plane 4");
out
}
fn missing(d: &Document, l: &DocLayout) -> MissingGlyphs {
l.missing_glyphs(d, &DisplayOptions::default(), None)
}
fn chars(m: &MissingGlyphs) -> Vec<char> {
m.chars.iter().map(|(c, _)| *c).collect()
}
fn hf_part(d: &mut Document, kind: wordcraft_doc::PartKind, text: &str) -> u32 {
d.add_part(kind, vec![wordcraft_doc::para_block(wordcraft_doc::Paragraph::with_text(text, Default::default()))])
}
fn chart_with(d: &mut Document, label: &str) {
let items = vec![GraphicItem::Text {
rect: [0.0, 100.0, 216.0, 20.0],
text: label.into(),
size: 10.0,
color: Rgb::BLACK,
bold: false,
align: TextAlign::Center,
font: None,
}];
let graphic = Graphic { kind: GraphicKind::Chart, items, ..Default::default() };
let obj = InlineObject::Graphic { w: 216.0, h: 144.0, alt: String::new(), float: Default::default(), graphic: Arc::new(graphic) };
d.insert_object(&Pos::body(0, 0), obj, &Default::default()).unwrap();
}
fn equation_with(d: &mut Document, linear: &str) {
let math = wordcraft_doc::math::Math { nodes: wordcraft_doc::math::parse_linear(linear), ..Default::default() };
let n = d.para_mut(StoryRef::Body, &Path::top(0)).unwrap().len();
d.para_mut(StoryRef::Body, &Path::top(0))
.unwrap()
.insert_object(n, InlineObject::Equation { linear: linear.into(), display: false, math }, &Default::default())
.unwrap();
}
#[test]
fn nothing_is_missing_in_plain_text() {
let d = Document::from_text("Hello, world.");
assert!(missing(&d, &lay(&d)).is_empty());
}
#[test]
fn a_character_no_face_covers_is_counted_once_per_occurrence_in_the_body() {
let c = uncovered(1)[0];
let d = Document::from_text(&format!("a{c}b{c}{c}c"));
let m = missing(&d, &lay(&d));
assert_eq!(m, MissingGlyphs::from_listed(vec![(c, 3)]));
}
#[test]
fn a_missing_glyph_is_found_in_a_header() {
let c = uncovered(1)[0];
let mut d = Document::from_text("Body");
d.last_section.headers.default = Some(hf_part(&mut d, wordcraft_doc::PartKind::Header, &format!("H{c}{c}")));
assert_eq!(missing(&d, &lay(&d)).chars, vec![(c, 2)]);
}
#[test]
fn a_missing_glyph_is_found_in_a_footer() {
let c = uncovered(1)[0];
let mut d = Document::from_text("Body");
d.last_section.footers.default = Some(hf_part(&mut d, wordcraft_doc::PartKind::Footer, &format!("F{c}")));
assert_eq!(missing(&d, &lay(&d)).chars, vec![(c, 1)]);
}
#[test]
fn a_missing_glyph_is_found_in_a_footnote() {
let c = uncovered(1)[0];
let mut d = Document::from_text("Body text.");
footnote(&mut d, &Pos::body(0, 4), &format!("Note {c}"));
assert_eq!(missing(&d, &lay(&d)).chars, vec![(c, 1)]);
}
#[test]
fn a_missing_glyph_is_found_in_a_text_box() {
let c = uncovered(1)[0];
let mut d = Document::from_text("Body text.");
text_box_at(&mut d, &Pos::body(0, 0), &format!("Box {c}{c}{c}"), 120.0, 40.0, page_float(300.0, 300.0));
assert_eq!(missing(&d, &lay(&d)).chars, vec![(c, 3)]);
}
#[test]
fn a_missing_glyph_is_found_in_a_table_cell() {
let c = uncovered(1)[0];
let mut d = Document::from_text("before");
let mut t = Table::new(1, 2, 300.0);
t.rows[0].cells[1].blocks = vec![wordcraft_doc::para_block(wordcraft_doc::Paragraph::with_text(&format!("cell {c}"), Default::default()))];
d.insert_block(StoryRef::Body, &Path::top(1), Block::Table(t)).unwrap();
assert_eq!(missing(&d, &lay(&d)).chars, vec![(c, 1)]);
}
#[test]
fn a_missing_glyph_is_found_in_a_chart_label() {
let c = uncovered(1)[0];
let mut d = Document::from_text("Chart");
chart_with(&mut d, &format!("Q{c}Q{c}"));
assert_eq!(missing(&d, &lay(&d)).chars, vec![(c, 2)]);
}
#[test]
fn a_chart_label_run_names_each_glyphs_characters() {
use crate::display::Draw;
let mut d = Document::from_text("Chart");
chart_with(&mut d, "Q1 total");
let l = lay(&d);
fn runs(d: &Draw, out: &mut Vec<(String, usize, usize)>) {
match d {
Draw::Glyphs { text, glyphs, ranges, .. } if text == "Q1 total" || !ranges.is_empty() => {
out.push((text.clone(), glyphs.len(), ranges.len()));
assert!(ranges.iter().all(|r| text.get(r.clone()).is_some()), "ranges on character boundaries");
}
Draw::Figure { draws, .. } => draws.iter().for_each(|x| runs(x, out)),
_ => {}
}
}
let mut found = Vec::new();
for x in crate::display::page_display(&d, &l.pages[0], &DisplayOptions::default()) {
runs(&x, &mut found);
}
assert!(found.iter().any(|(t, g, r)| t == "Q1 total" && g == r), "{found:?}");
}
#[test]
fn a_missing_glyph_is_found_in_an_equation() {
let c = uncovered(1)[0];
let mut d = Document::from_text("Eq: ");
equation_with(&mut d, &format!("x+{c}"));
let m = missing(&d, &lay(&d));
assert_eq!(m.chars, vec![(c, 1)], "{m:?}");
}
#[test]
fn joiners_and_variation_selectors_are_never_counted() {
let d = Document::from_text("a\u{200D}b\u{200C}c\u{FE0F}d\u{FE0E}e\u{00AD}f\u{200E}g\u{2028}h\ti");
assert!(missing(&d, &lay(&d)).is_empty());
// A ZWJ emoji sequence: the joiners and selectors in it never appear, whatever the host's
// emoji face covers.
let d = Document::from_text("\u{1F469}\u{200D}\u{1F4BB} \u{2764}\u{FE0F} \u{1F44D}\u{1F3FD}");
let m = missing(&d, &lay(&d));
assert!(!chars(&m).iter().any(|c| matches!(c, '\u{200D}' | '\u{FE0F}' | '\u{FE0E}')), "{m:?}");
}
#[test]
fn a_selector_beside_a_missing_character_is_not_counted_with_it() {
let c = uncovered(1)[0];
let d = Document::from_text(&format!("{c}\u{FE0F}"));
assert_eq!(chars(&missing(&d, &lay(&d))), vec![c]);
}
#[test]
fn the_list_is_sorted_and_capped_at_256_distinct_code_points() {
let all = uncovered(MAX_MISSING_GLYPHS + 44);
let text: String = all.iter().rev().collect();
let d = Document::from_text(&text);
let m = missing(&d, &lay(&d));
assert!(m.truncated);
assert_eq!(m.chars.len(), MAX_MISSING_GLYPHS);
assert_eq!(m.total, MAX_MISSING_GLYPHS + 44, "the total counts the unlisted ones too");
assert!(m.chars.windows(2).all(|w| w[0].0 < w[1].0), "sorted by code point");
// Exactly the cap's worth is no truncation.
let d = Document::from_text(&all.iter().take(MAX_MISSING_GLYPHS).collect::<String>());
let m = missing(&d, &lay(&d));
assert!(!m.truncated);
assert_eq!(m.chars.len(), MAX_MISSING_GLYPHS);
assert_eq!(m.total, MAX_MISSING_GLYPHS);
}
#[test]
fn the_page_filter_selects_pages() {
let [x, y] = <[char; 2]>::try_from(uncovered(2)).unwrap();
let d = Document::from_text(&format!("first {x}\n{}last {y}", "filler line\n".repeat(150)));
let l = lay(&d);
let last = l.pages.len() - 1;
assert!(last >= 1, "the document runs over pages");
let on = |r| l.missing_glyphs(&d, &DisplayOptions::default(), r);
assert_eq!(on(None).chars, vec![(x.min(y), 1), (x.max(y), 1)]);
assert_eq!(on(Some(0..=0)).chars, vec![(x, 1)]);
assert_eq!(on(Some(last..=last)).chars, vec![(y, 1)]);
assert!(on(Some(last + 1..=last + 9)).is_empty());
#[allow(clippy::reversed_empty_ranges)]
let backwards = Some(1..=0);
assert!(on(backwards).is_empty());
}
#[test]
fn an_equation_run_names_each_glyphs_characters() {
use crate::display::Draw;
let mut d = Document::from_text("Eq: ");
equation_with(&mut d, "x+ab");
let l = lay(&d);
fn runs(d: &Draw, out: &mut Vec<(String, usize, usize)>) {
match d {
Draw::Glyphs { text, glyphs, ranges, .. } if !text.is_empty() => {
assert!(ranges.iter().all(|r| text.get(r.clone()).is_some()), "ranges on character boundaries");
out.push((text.clone(), glyphs.len(), ranges.len()));
}
Draw::Figure { draws, .. } | Draw::Clip { items: draws, .. } | Draw::Rotated { items: draws, .. } => {
draws.iter().for_each(|x| runs(x, out))
}
_ => {}
}
}
let mut found = Vec::new();
for x in crate::display::page_display(&d, &l.pages[0], &DisplayOptions::default()) {
runs(&x, &mut found);
}
// The equation's runs (the body text "Eq: " is a run too) each carry one range per glyph.
// The letters are the math italic ones where the font has them, else plain.
assert!(found.iter().any(|(t, g, _)| (t.contains('\u{1D44E}') || t == "ab") && *g == 2), "{found:?}");
assert!(found.iter().all(|(_, g, r)| g == r), "{found:?}");
}
+3 -4
View File
@@ -3,8 +3,8 @@ use super::metadata::metadata;
use super::options::{PdfError, PdfOptions};
use super::paint::{clamp_side, ok, rect_path};
use super::{
DocLayout, Document, FillRule, HashMap, HashSet, KrillaError, LayoutCache, LayoutOptions, Page, PageSettings, Rect, Size, Transform, WriteError,
Writer, imposition, krilla, layout, write,
DocLayout, Document, FillRule, HashMap, HashSet, KrillaError, LayoutCache, Page, PageSettings, Rect, Size, Transform, WriteError, Writer,
imposition, krilla, layout, write,
};
/// Lay out `doc` and write it as PDF. Without markup the layout is the final text: tracked
@@ -12,8 +12,7 @@ use super::{
/// markup area with the comment balloons.
pub fn export(doc: &Document, opts: &PdfOptions) -> Result<Vec<u8>, PdfError> {
write::guard(|| {
let lo = LayoutOptions { hide_deleted: !opts.include_markup, comment_balloons: opts.comments, numeral: opts.numeral, ..Default::default() };
let lay = layout(doc, &mut LayoutCache::new(), &lo);
let lay = layout(doc, &mut LayoutCache::new(), &opts.layout_options());
export_layout_inner(doc, &lay, opts)
})
}
+6 -16
View File
@@ -5,10 +5,10 @@ use super::paint::{append_path, fill, grown, norm, ok, paint_fill, pdf_line_cap,
use super::picture::{Vector, build_vector, load_image, paint_of, vector_picture};
use super::tags::{Role, TagEntry, figure_alt, graphic_alt, holds_picture, holds_text};
use super::{
Annotation, Arc, Artifact, ArtifactType, ClKind, ContentTag, DisplayOptions, DocLayout, DocPath, Document, Draw, FaceRef, Fill, FillRule, Font,
HashMap, HashSet, Identifier, Image, InlineObject, ItemDisplay, KrillaGlyph, LineCap, LineEnd, LineJoin, LineStyle, LinkAnnotation, Node,
NonZeroU16, Outline, OutlineNode, Page, PathBuilder, Placed, Point, Rect, Rgb, ShapeKind, Size, SpanTag, StoryRef, Stroke, StrokeDash, Surface,
Tag, TagGroup, TagKind, TagTree, Target, Transform, WriteError, XyzDestination, krilla, rgb, write,
Annotation, Arc, Artifact, ArtifactType, ClKind, ContentTag, DocLayout, DocPath, Document, Draw, FaceRef, Fill, FillRule, Font, HashMap, HashSet,
Identifier, Image, InlineObject, ItemDisplay, KrillaGlyph, LineCap, LineEnd, LineJoin, LineStyle, LinkAnnotation, Node, NonZeroU16, Outline,
OutlineNode, Page, PathBuilder, Placed, Point, Rect, Rgb, ShapeKind, Size, SpanTag, StoryRef, Stroke, StrokeDash, Surface, Tag, TagGroup,
TagKind, TagTree, Target, Transform, WriteError, XyzDestination, krilla, rgb, write,
};
use wordcraft_doc::picfx::PicFx;
@@ -240,14 +240,7 @@ impl Exporter<'_> {
let area = page.body;
self.tagged(s, Role::Artifact(ArtifactType::Watermark), None, |me, s| me.watermark(s, &wm, w, h, area));
}
let dopts = DisplayOptions {
marks: false,
dim_header: false,
dim_body: false,
markup: self.opts.include_markup,
placeholders: false,
..Default::default()
};
let dopts = self.opts.display_options();
// What the items share (the authors' colours, the clipped cells) is worked out once per page.
let shown = ItemDisplay::new(self.doc, page);
for it in page.header.iter().chain(page.footer.iter()) {
@@ -356,10 +349,7 @@ impl Exporter<'_> {
/// the baseline) on the box's bottom; the box centred on the margins (`area`, else the page)
/// and, when diagonal, turned 45° counterclockwise.
fn watermark(&mut self, s: &mut Surface, wm: &wordcraft_doc::Watermark, w: f32, h: f32, area: Rect) {
let r = wordcraft_fonts::word::resolve(&wm.font, false, false);
let face = r.face;
let text: String = wm.text.chars().take(200).collect();
let shaped = wordcraft_fonts::shape(&face, &text, &[], |c| c);
let Some((face, text, shaped)) = wordcraft_layout::watermark_run(wm, super::WATERMARK_MAX_CHARS) else { return };
let upem = face.upem.max(1.0) as f32;
let raw_w: f32 = shaped.iter().map(|g| g.x_advance as f32).sum::<f32>() / upem;
if raw_w <= 0.0 || !raw_w.is_finite() {
+1 -2
View File
@@ -183,8 +183,7 @@ impl Exporter<'_> {
/// Draws a metafile text run as PDF text (outlines when the font cannot be embedded). `tf` maps
/// the run's own space (see [`PlacedItem::Text`]) to the page.
pub(crate) fn vector_text(&mut self, s: &mut Surface, t: &wordcraft_metafile::Text, tf: kurbo::Affine) {
let face = wordcraft_fonts::word::resolve(&t.font, t.bold, t.italic).face;
let shaped = wordcraft_fonts::shape(&face, &t.text, &[], |c| c);
let (face, shaped) = wordcraft_layout::metafile_text_run(t);
let glyphs: Vec<_> = shaped.iter().map(|g| (g.cluster, f64::from(g.x_advance), f64::from(g.x_offset), f64::from(g.y_offset))).collect();
let Some(lay) = t.layout(face.upem, face.ascent, face.descent, &glyphs) else { return };
let font = self.font(&face);
+3 -3
View File
@@ -49,9 +49,9 @@ use wordcraft_doc::{Document, Path as DocPath, Rgb, StoryRef};
use wordcraft_fonts::FaceRef;
use wordcraft_geom::Rect;
use wordcraft_layout::display::Paint;
use wordcraft_layout::display::{DisplayOptions, Draw, ItemDisplay, Stroke as LineStyle};
use wordcraft_layout::display::{Draw, ItemDisplay, Stroke as LineStyle};
use wordcraft_layout::para::{ClKind, LineEnd};
use wordcraft_layout::{DocLayout, LayoutCache, LayoutOptions, NumeralMode, Page, Placed, layout};
use wordcraft_layout::{DocLayout, LayoutCache, NumeralMode, Page, Placed, layout};
use wordcraft_metafile::{Picture, PlacedItem};
mod export;
@@ -72,7 +72,7 @@ use exporter::Exporter;
use glyphs::{baseline, cluster_ends, left_to_right, split_clusters, visual_order};
#[cfg(test)]
use metadata::{civil, parse_iso};
pub use options::{PdfError, PdfOptions};
pub use options::{PdfError, PdfOptions, WATERMARK_MAX_CHARS};
#[cfg(test)]
use paint::shape_path;
#[cfg(test)]
+29
View File
@@ -52,6 +52,35 @@ pub struct PdfOptions {
pub numeral: NumeralMode,
}
/// How many characters of the watermark text the PDF draws.
pub const WATERMARK_MAX_CHARS: usize = 200;
impl PdfOptions {
/// What a page shows when drawn into the PDF: no marks or dimming, tracked changes as markup
/// only with `include_markup`. The one place that says so; strict export (#6005) checks the
/// same display list.
pub fn display_options(&self) -> wordcraft_layout::display::DisplayOptions {
wordcraft_layout::display::DisplayOptions {
marks: false,
dim_header: false,
dim_body: false,
markup: self.include_markup,
placeholders: false,
..Default::default()
}
}
/// How [`crate::export`] lays a document out for these options.
pub fn layout_options(&self) -> wordcraft_layout::LayoutOptions {
wordcraft_layout::LayoutOptions {
hide_deleted: !self.include_markup,
comment_balloons: self.comments,
numeral: self.numeral,
..Default::default()
}
}
}
impl Default for PdfOptions {
/// Keep ordinary document export unchanged unless a print layout is requested.
fn default() -> Self {
+3 -24
View File
@@ -116,32 +116,11 @@ pub(crate) fn load_image(data: &Arc<Vec<u8>>, fx: PicFx) -> Option<Image> {
/// [`load_image`] with an explicit RGBA byte budget (a small one in tests).
pub(crate) fn load_image_within(data: &Arc<Vec<u8>>, fx: PicFx, budget: u64) -> Option<Image> {
use image::ImageDecoder;
let fmt = image::guess_format(data).ok()?;
let mut limits = image::Limits::default();
limits.max_image_width = Some(MAX_IMAGE_SIDE);
limits.max_image_height = Some(MAX_IMAGE_SIDE);
let reader = || {
let mut r = image::ImageReader::new(std::io::Cursor::new(data.as_slice())).with_guessed_format().ok()?;
r.limits(limits.clone());
Some(r)
};
// Header-only size check before anything is allocated (the decoder's buffer at its own pixel
// size, 8 bytes for 16-bit RGBA, plus the RGBA conversion are charged); the side limits and
// the default 512 MB allocation cap apply to the header read and the decode alike.
let (hw, hh, native) = {
let d = reader()?.into_decoder().ok()?;
let (w, h) = d.dimensions();
(w, h, u64::from(d.color_type().bytes_per_pixel()))
};
if !wordcraft_geom::decode_peak_fits(hw, hh, native, budget) {
return None;
}
let dec = reader()?.decode().ok()?;
// The decision to draw or not is the shared one (`wordcraft_geom::picture`), the one strict
// export checks with.
let dec = wordcraft_geom::picture::decode_for_drawing(data, budget, MAX_IMAGE_SIDE).ok()?;
let (w, h) = (dec.width(), dec.height());
if w == 0 || h == 0 || w > MAX_IMAGE_SIDE || h > MAX_IMAGE_SIDE {
return None;
}
if !fx.changes_pixels() && fmt == image::ImageFormat::Jpeg && matches!(dec.color(), image::ColorType::L8 | image::ColorType::Rgb8) {
return Image::from_jpeg(data.clone().into(), true).ok();
}
+88
View File
@@ -6,6 +6,8 @@ use wordcraft_doc::props::NumRef;
use wordcraft_doc::{Block, CharProps, Document, ListKind, Paragraph, Table, Watermark, para_block};
use crate::*;
use wordcraft_layout::LayoutOptions;
use wordcraft_layout::display::DisplayOptions;
/// Verify preset lines and connectors emit the requested PDF line-cap operator.
#[test]
@@ -2912,3 +2914,89 @@ fn the_author_is_left_out_when_personal_information_is_removed() {
d.settings.remove_personal_info = true;
assert!(!raw(&d).contains("Ada Writer"), "the creator stays out of the PDF");
}
/// A page with text and three pictures that don't draw: garbage, a PNG cut off in its data and a
/// metafile that is only its header. The export is the same bytes as before strict export
/// existed (#6005): each is a grey box, and the rest of the page is untouched.
#[test]
fn undecodable_pictures_keep_their_grey_box_bytes() {
let mut whole = png();
let cut = whole.len() - 24;
whole.truncate(cut);
let mut d = Document::new();
d.core.created = "2024-01-02T03:04:05Z".into();
let mut p = Paragraph::new();
for (bytes, ext) in [(b"this is not a picture at all".to_vec(), "bin"), (whole, "png"), (vec![1, 0, 0, 0, 88, 0, 0, 0], "emf")] {
let media = d.add_media(bytes, ext);
let n = p.len();
let obj = InlineObject::Image {
media,
w: 60.0,
h: 40.0,
alt: String::new(),
float: Default::default(),
crop: [0.0; 4],
fx: Default::default(),
ole: None,
};
p.insert_object(n, obj, &CharProps::default()).unwrap();
}
d.body = vec![para_block(p)];
let opts = PdfOptions { compress: false, ..Default::default() };
let a = export(&d, &opts).unwrap();
assert_eq!(a, export(&d, &opts).unwrap(), "the export must be deterministic for the golden hash to mean anything");
let text = String::from_utf8_lossy(&a);
assert!(text.contains("0.8157 0.8157 0.8157") || text.contains(".81"), "the grey box is drawn: {}", text.len());
// FNV-1a of the file, recorded from the export before the shared decode check existed.
let h = a.iter().fold(0xcbf29ce484222325u64, |h, b| (h ^ u64::from(*b)).wrapping_mul(0x100000001b3));
assert_eq!(h, 0x1edbbffab353d744, "{h:#x}");
}
/// A page with a chart label and an equation, exported without compression. Chart and equation
/// runs carry the text each glyph shows since strict export (#6005); this pins the PDF they
/// produce, ToUnicode maps included, to what they were before.
#[test]
fn chart_and_equation_text_keep_their_pdf_bytes() {
use wordcraft_doc::graphic::{Graphic, GraphicItem, GraphicKind, TextAlign};
let mut d = Document::from_text("Body text");
d.core.created = "2024-01-02T03:04:05Z".into();
let items = vec![GraphicItem::Text {
rect: [0.0, 100.0, 216.0, 20.0],
text: "Revenue 2024".into(),
size: 10.0,
color: wordcraft_doc::props::Rgb::BLACK,
bold: false,
align: TextAlign::Center,
font: None,
}];
let graphic = Graphic { kind: GraphicKind::Chart, items, ..Default::default() };
let obj = InlineObject::Graphic { w: 216.0, h: 144.0, alt: String::new(), float: Default::default(), graphic: Arc::new(graphic) };
d.insert_object(&wordcraft_doc::Pos::body(0, 0), obj, &CharProps::default()).unwrap();
let linear = "x^2+1/2";
let math = wordcraft_doc::math::Math { nodes: wordcraft_doc::math::parse_linear(linear), ..Default::default() };
let n = d.para_mut(wordcraft_doc::StoryRef::Body, &wordcraft_doc::Path::top(0)).unwrap().len();
d.para_mut(wordcraft_doc::StoryRef::Body, &wordcraft_doc::Path::top(0))
.unwrap()
.insert_object(n, InlineObject::Equation { linear: linear.into(), display: false, math }, &CharProps::default())
.unwrap();
let opts = PdfOptions { compress: false, ..Default::default() };
let a = export(&d, &opts).unwrap();
assert_eq!(a, export(&d, &opts).unwrap(), "the export must be deterministic for the golden hash to mean anything");
// What the ToUnicode maps say each glyph shows, across all fonts: the text of the body, the
// chart label and the equation (a mathematical italic x is one character beyond the BMP).
// Recorded on the code before strict export: the maps are the same. (The whole file's hash
// is not pinned: it follows the fonts the host resolves, which differ between runs.)
let text = String::from_utf8_lossy(&a).into_owned();
let mut shown = std::collections::BTreeSet::new();
for block in text.split("beginbfchar").skip(1) {
for line in block.split("endbfchar").next().unwrap_or("").lines() {
let Some((_, to)) = line.trim().split_once("> <") else { continue };
let units: Vec<u16> =
to.trim_end_matches('>').as_bytes().chunks(4).filter_map(|c| u16::from_str_radix(std::str::from_utf8(c).ok()?, 16).ok()).collect();
shown.insert(String::from_utf16_lossy(&units));
}
}
let expected: std::collections::BTreeSet<String> =
["B", "o", "d", "y", " ", "t", "e", "x", "R", "v", "n", "u", "2", "0", "4", "+", "1", "\u{1D465}"].into_iter().map(String::from).collect();
assert_eq!(shown, expected);
}
+1 -3
View File
@@ -461,9 +461,7 @@ pub fn render_text_line(face: &wordcraft_fonts::FontFace, text: &str, size: f32,
}
fn draw_watermark(ctx: &mut RenderContext, view: Affine, page: &Page, wm: &wordcraft_doc::Watermark, opts: &RenderOptions) {
let r = wordcraft_fonts::word::resolve(&wm.font, false, false);
let face = r.face;
let glyphs = wordcraft_fonts::shape(&face, &wm.text, &[], |c| c);
let Some((face, _, glyphs)) = wordcraft_layout::watermark_run(wm, usize::MAX) else { return };
let upem = face.upem.max(1.0);
let raw_w: f64 = glyphs.iter().map(|g| g.x_advance as f64).sum::<f64>() / upem;
if raw_w <= 0.0 {
+8 -31
View File
@@ -82,30 +82,14 @@ pub fn decode_pixmap_fx(bytes: &[u8], fx: wordcraft_doc::picfx::PicFx) -> Option
/// [`decode_pixmap_fx`] with an explicit RGBA byte budget (a small one in tests).
pub(crate) fn decode_pixmap_within(bytes: &[u8], fx: wordcraft_doc::picfx::PicFx, budget: u64) -> Option<Pixmap> {
use image::ImageDecoder;
let mut limits = image::Limits::default();
limits.max_image_width = Some(MAX_DECODE_SIDE);
limits.max_image_height = Some(MAX_DECODE_SIDE);
let reader = || {
let mut r = image::ImageReader::new(std::io::Cursor::new(bytes)).with_guessed_format().ok()?;
r.limits(limits.clone());
Some(r)
};
// The header gives the size and the decoder's pixel format (16-bit RGBA takes 8 bytes a pixel
// before its RGBA8 copy); refuse before the decoder (and the RGBA copy) allocate anything.
let (hw, hh, native) = {
let d = reader()?.into_decoder().ok()?;
let (w, h) = d.dimensions();
(w, h, u64::from(d.color_type().bytes_per_pixel()))
};
if !wordcraft_geom::decode_peak_fits(hw, hh, native, budget) {
// The decision to draw or not is the shared one (`wordcraft_geom::picture`), the one strict
// export checks with.
let decoded = wordcraft_geom::picture::decode_for_drawing(bytes, budget, MAX_DECODE_SIDE);
if let Err(wordcraft_geom::picture::Refusal::TooLarge { width, height }) = &decoded {
// Logged once a picture: its refusal is cached with it (`pixmap_for`).
log::warn!(
"a {hw}×{hh} picture of {native} bytes a pixel needs more than {budget} bytes to decode: it is not drawn (in the text, a grey box takes its place)"
);
return None;
log::warn!("a {width}×{height} picture needs more than {budget} bytes to decode: it is not drawn (in the text, a grey box takes its place)");
}
let mut img = reader()?.decode().ok()?.into_rgba8();
let mut img = decoded.ok()?.into_rgba8();
let (w, h) = img.dimensions();
fx.apply_rgba(&mut img);
premul_pixmap(w, h, img.as_raw())
@@ -270,8 +254,7 @@ pub(crate) fn draw_vector(ctx: &mut RenderContext, v: &Vector, r: &kurbo::Rect,
/// Draws a metafile text run: `tf` maps the run's own space (see [`PlacedItem::Text`]) to the device.
fn draw_vector_text(ctx: &mut RenderContext, t: &wordcraft_metafile::Text, tf: Affine) {
let face = wordcraft_fonts::word::resolve(&t.font, t.bold, t.italic).face;
let shaped = wordcraft_fonts::shape(&face, &t.text, &[], |c| c);
let (face, shaped) = wordcraft_layout::metafile_text_run(t);
let glyphs: Vec<_> = shaped.iter().map(|g| (g.cluster, f64::from(g.x_advance), f64::from(g.x_offset), f64::from(g.y_offset))).collect();
let Some(lay) = t.layout(face.upem, face.ascent, face.descent, &glyphs) else { return };
let k = lay.em / face.upem.max(1.0);
@@ -326,18 +309,12 @@ fn device_pixel(view: Affine) -> f64 {
if scale.is_finite() && scale > 1e-9 { 1.0 / scale } else { 1.0 }
}
/// Largest page-background picture decoded, in pixels (hostile files).
const MAX_BACKGROUND_PIXELS: u64 = 64 * 1024 * 1024;
/// Draw the page's picture or texture fill (Design › Page Color › Fill Effects) behind the content:
/// tiled at its natural size (96 dpi) for a `tile` fill, stretched over the page otherwise.
pub(crate) fn draw_page_picture(ctx: &mut RenderContext, doc: &Document, page: &Page, view: Affine) {
let Some(pic) = &doc.settings.page_picture else { return };
let Some(bytes) = doc.media.get(&pic.media) else { return };
let (nw, nh) = match image_size(bytes) {
Some((w, h)) if w > 0 && h > 0 && (w as u64) * (h as u64) <= MAX_BACKGROUND_PIXELS => (w as f64, h as f64),
_ => return,
};
let Some((nw, nh)) = wordcraft_geom::picture::background_size(bytes).map(|(w, h)| (f64::from(w), f64::from(h))) else { return };
let Some(pm) = pixmap_for(doc, &pic.media, wordcraft_doc::picfx::PicFx::default()) else { return };
let (pw, ph) = (pm.width().max(1) as f64, pm.height().max(1) as f64);
let (w, h) = (page.w as f64, (page.h as f64).min(1e6));
+38
View File
@@ -1487,3 +1487,41 @@ fn paths_far_past_every_canvas_are_not_flattened() {
assert!(!path_extent_ok(Affine::scale(1e12), &line(10.0)));
assert!(!path_extent_ok(Affine::IDENTITY, &line(f64::NAN)));
}
/// The same three pictures that don't draw as in the PDF test: each stays a grey box, and the page's
/// pixels are what they were before strict export existed (#6005).
#[test]
fn undecodable_pictures_keep_their_grey_box_pixels() {
let mut whole = Vec::new();
image::RgbaImage::from_fn(16, 8, |x, y| image::Rgba([(x * 15) as u8, (y * 30) as u8, 200, 255]))
.write_to(&mut std::io::Cursor::new(&mut whole), image::ImageFormat::Png)
.unwrap();
let cut = whole.len() - 24;
whole.truncate(cut);
let mut d = Document::new();
let mut p = wordcraft_doc::Paragraph::new();
for (bytes, ext) in [(b"this is not a picture at all".to_vec(), "bin"), (whole, "png"), (vec![1, 0, 0, 0, 88, 0, 0, 0], "emf")] {
let media = d.add_media(bytes, ext);
let n = p.len();
let obj = wordcraft_doc::para::InlineObject::Image {
media,
w: 60.0,
h: 40.0,
alt: String::new(),
float: Default::default(),
crop: [0.0; 4],
fx: Default::default(),
ole: None,
};
p.insert_object(n, obj, &Default::default()).unwrap();
}
d.body = vec![wordcraft_doc::para_block(p)];
let l = layout(&d, &mut LayoutCache::new(), &LayoutOptions::default());
let opts = RenderOptions::default();
let img = render_page(&d, &l.pages[0], 1.0, &opts);
let boxes = page_display(&d, &l.pages[0], &opts.display).iter().filter(|i| matches!(i, Draw::Image { .. })).count();
assert_eq!(boxes, 3);
// FNV-1a of the pixels, recorded before the shared decode check existed.
let h = img.pixels.iter().fold(0xcbf29ce484222325u64, |h, b| (h ^ u64::from(*b)).wrapping_mul(0x100000001b3));
assert_eq!(h, 0x79f9f1d3602a9925, "{h:#x}");
}
+4
View File
@@ -1,4 +1,6 @@
# File-format parity with Microsoft Word
> **2026-11-08:** `wordcraft-cli convert --strict` refuses a PDF or PNG that would draw an empty box for a character or leave out a picture, including those a DOCX names outside the file or not at all (#6005); never fetched.
> **2026-11-05:** A Word template opened from the file manager starts a new document based on it, and is never saved over (#3433).
> **2026-11-05:** Strict Open XML documents open with their shapes, text boxes, groups and canvases (#3548).
@@ -321,6 +323,8 @@ For field results, the engine parser treats a backslash in quoted arguments as e
## Revision history
> 2026-11-08 · `convert --strict` for PDF and PNG output, and the pictures a DOCX open leaves out are recorded (#6005); recorded in the progress log, readiness unchanged.
> 2026-11-07 · VML text boxes: `mso-layout-flow-alt:bottom-to-top` reads as `vert270` with any `layout-flow`, and `layout-flow:vertical` or `vertical-ideographic` otherwise as `vert`, so a compatibility-mode header's turned text box is drawn turned (#5193, `it::text_boxes::vml_text_boxes_keep_their_inset_fit_and_anchor`).
> 2026-11-03 · Section `w:chapStyle` values preserve the full unsigned-byte range, including 10, while values above 255 are rejected.