Skip to content

bannertopdf: support Unicode text with embedded fallback fonts - #256

Open
oubayashi wants to merge 2 commits into
OpenPrinting:masterfrom
oubayashi:bannertopdf-unicode
Open

oubayashi wants to merge 2 commits into
OpenPrinting:masterfrom
oubayashi:bannertopdf-unicode

Conversation

@oubayashi

@oubayashi oubayashi commented Sep 23, 2026 •

Copy link
Copy Markdown

Summary

Fix garbled non-ASCII text in bannertopdf output.

Banner fields were previously written as raw bytes using the PDF standard
Courier font, which only works reliably for single-byte text. Printer names,
locations, job titles, user names, and other fields containing UTF-8 text
could therefore appear garbled or lose characters.

This pull request also fixes an existing page-resource handling issue exposed
while testing Unicode font embedding. Banner templates with an indirect
/Resources dictionary could lose their image XObjects when banner fonts were
added.

Changes

Unicode banner text

  • Add strict UTF-8 decoding with validation for malformed, overlong, surrogate,
    and out-of-range sequences.
  • Use bit masking to validate UTF-8 leading and continuation byte patterns.
  • Use Fontconfig to find scalable outline fonts covering non-ASCII characters.
  • Embed selected fonts as Unicode CID fonts through PDFio.
  • Support per-character font fallback when a line contains multiple writing
    systems.
  • Cache and reuse up to 32 embedded fonts.
  • Keep the existing Courier path for ASCII text.
  • Escape parentheses, backslashes, and control characters in PDF literal
    strings.
  • Replace unsupported characters safely and emit warnings instead of producing
    invalid font references.
  • Apply the Unicode path consistently to printer information, job metadata,
    user names, host names, billing information, model names, and localized
    times.

Preserve template resources

  • Detect whether the page /Resources entry is a direct dictionary or an
    indirect object.
  • Resolve and copy indirect resource dictionaries before adding banner fonts.
  • Preserve existing /XObject, /ExtGState, /Font, and other template
    resources.
  • Prevent images from default-testpage.pdf from disappearing when the page
    resources are updated.

Regression Test

Add an end-to-end regression test following the existing
test-pdftopdf-inherited-mediabox test pattern.

The test runs a mixed ASCII and Unicode banner through the existing
testfilters harness, then reopens the generated PDF with PDFio and verifies
that:

  • The output is a valid single-page PDF.
  • The original Courier resource remains available for ASCII text.
  • A Unicode Type 0 fallback font resource is created.
  • The Unicode font contains a ToUnicode map.
  • The fallback font file is embedded in the PDF.
  • Unicode text is emitted using the embedded font.
  • Parentheses and backslashes are correctly escaped in PDF literal strings.
  • The template image resources /x6, /x7, and /x8 are preserved.
  • The corresponding /x6 Do, /x7 Do, and /x8 Do drawing commands remain
    in the page content stream.

The testfilters harness now supports selecting bannertopdf explicitly for
focused regression testing.

The regression test is skipped when Fontconfig, PDFio, or a font covering the
test character is unavailable.

The initial regression-test implementation was prepared with assistance from
OpenAI Codex and reviewed against the existing project test patterns.

The changes were built and tested successfully on Ubuntu 24.04. Testing
included generating a banner with mixed ASCII and Unicode text, confirming
that the fallback font was embedded, and verifying that the text and template
images were rendered correctly in the generated PDF.

Compatibility

  • Pure ASCII output keeps the existing behavior and layout.
  • Unicode font fallback is enabled only when Fontconfig is available.
  • Without Fontconfig, unsupported non-ASCII characters fall back to ?.
  • Supplementary-plane characters above U+FFFF currently fall back to
    U+FFFD, because PDFio 1.6 uses a BMP-sized CID-to-GID mapping.
  • Complex-script shaping and bidirectional text are outside the scope of this
    change.

Commit Structure

This pull request is split into two commits:

  1. Add UTF-8 decoding, Fontconfig fallback, embedded Unicode CID fonts, PDF
    string escaping, and an end-to-end Unicode banner regression test.
  2. Preserve indirect page resource dictionaries when adding banner fonts and
    extend the regression test to cover template image resources.

Comment thread cupsfilters/bannertopdf.c Outdated
Decode banner fields as UTF-8 instead of writing raw input bytes to
Courier strings.

Use Fontconfig to find outline fonts covering non-ASCII characters,
cache matching fonts, and embed them as Unicode CID fonts through
PDFio. Switch font resources as needed when a line contains characters
from multiple writing systems while keeping the existing Courier path
for ASCII text.

Validate UTF-8 sequences, escape special and control characters in PDF
literal strings, and gracefully replace characters when no embeddable
font is available. Supplementary-plane characters are currently
replaced because PDFio 1.6 uses a BMP-sized CIDToGID mapping.

This fixes garbled printer names, locations, job titles, user names,
and other banner fields containing non-ASCII text.
Resolve and copy an indirect page Resources dictionary before adding the
banner font instead of replacing it with an empty dictionary.

This preserves existing XObject, ExtGState, Font, and other resources from
the banner template. Extend the Unicode banner regression test to verify
that the template image resources and drawing commands remain present.
@oubayashi

Copy link
Copy Markdown
Author

I added a separate follow-up commit for the missing template images found
during testing.

The default test-page template stores /Resources as an indirect object.
The existing font setup path treated it as missing and replaced it with a new
dictionary, dropping the template's image XObjects. The follow-up commit now
copies and preserves indirect resources before adding banner fonts.

The regression test has also been extended to verify the /x6, /x7, and
/x8 image resources and their drawing commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants