Testing on real public documents, in 23 languages

PDF Info stopped working on our Hebrew, Arabic, Japanese, Korean and Chinese pages as soon as those languages went live. We found it in the October stress run, and it was also visible on pdfresult.com in Hebrew. The bug was ours. After building the report, the code compared the report title with a list of titles per language, and that list had been written before those languages existed. On a Hebrew page the title matched nothing, so the tool reported a failure.

None of our ordinary tests caught it. The regular browser tests run a short sample on each page, and the samples are chosen to finish in 1 to 3 seconds. The stress run is there to find what those samples miss.

What we set out to do

The last language group (Japanese, Korean and Chinese) went in on 3 October. After that we fed difficult files to every engine in every language, on the Laboratory build only. A very dirty file did not have to be read. It had to be refused honestly, or returned with “read partially, check the result”. A tool must not return garbage as if it were the answer.

Each finding went into one of two groups. Our own bugs we fixed. Engine limits we described on the tool pages.

The run covered the 63 tasks of the PDF workspace, the 18 engine pages (OCR, Office converters, PDF/A, the EPUB checker and others) and 23 page languages. This is what we fed them.

Input	Examples	What counted as a pass
Generated text PDFs	one line mixing the page language with Latin names, digits, brackets and quotes; dates, amounts and phone numbers inside right-to-left and CJK sentences; a 2,500-character line running off the page	every key word and number present in the output
Generated tables	merged title row, many empty cells, one table split over two pages with 24 rows, a right-to-left table	rows and the total come out; column positions survive
Generated scans	rotated pages, an A4 page at 100 DPI, grain and salt-and-pepper noise, two columns, text over a coloured picture	at least 50–60% of the key words, or a warning
Very dirty scan	12 degrees, heavy noise, blur, low contrast	refusal, or a partial result with a warning
Damaged and protected files	a PDF cut off at 60%, password-protected files	repair says what it recovered; passwords handled
Real public files	about 270 files from GovDocs1, Common Crawl PDFs (SafeDocs), the PDF.js test files, UN documents, Project Gutenberg and the W3C EPUB tests	no crash, no hang, no broken output, every message in the page language

The generated files came from scripts/stress/generate.mjs, one set per language. The real-file set came from scripts/stress/real-corpus.ts, which picks files from the public archives and detects their language from the text. Real files were capped at 20 MB each. The SafeDocs archive is about 1.3 GB unpacked, and the script kept at most four PDFs per detected language from it, ten for English. Real files have no answer key, so for them we looked for crashes, hangs, broken files and messages in the wrong language. Playwright drove the Laboratory build through tests/stress/. We deleted the raw archives after the run and kept a curated subset.

15,209 cases ran on 3–4 October 2026. 14,387 passed. 805 were skipped because the file had no data for that task, for example a table task on a file without a table. 17 failed on engine limits that we could not fix.

What we fixed

PDF Info came first. The report check no longer depends on the page language, and a test now runs it in all 23 languages. Control characters in metadata are dropped from the report.

PDFs with print or copy restrictions needed a decision. These files open without a password, but they are encrypted to restrict printing, copying or editing. On 3 October we decided to treat them the way large PDF services do. The workspace now removes the restrictions on selection with QPDF, processes the unrestricted copy and shows a notice that the result may be used only with permission. The QPDF program is about 3 MB and loads only for such files. A PDF that needs a password to open is still refused.

The Office converter had two bad messages. Our fonts have no emoji, so the converter turned emoji into blank space without a word; now it says so. When it stopped in the middle of a document, the page said it “could not start, check your connection”. It now says the document could not be read.

Smaller fixes went into the same release. Protect, Unlock, Repair, web optimization, OCR, the Office converters and the letterhead tool now keep the input name, for example report-unlocked.pdf. PDF OCR and the image OCR comparison add a warning when Tesseract’s mean confidence for a page is below 60. “Check the file and the limits” no longer appears under errors about the settings. PDFOxide mapped some list markers to the control character U+0007, which ended up in the Markdown output; those characters are now dropped.

All fixes went out as one release on 4 October. We checked them in production the same day on all six sites: PDF Info in seven languages, and a restricted PDF processed with the notice in English, Russian and Arabic.

What we left as it is

The engine limits are written on the tool pages. Text over coloured pictures is recognized only in part. The Greek and Hebrew OCR models read “P1” in a Latin code as “ΡΊ” with high confidence, so no warning appears. The Arabic OCR model loses words on a scan rotated 7 degrees. In Arabic Office output the visible text is correct, but the PDF text layer stores lam-alef ligatures and right-to-left dates in visual order, so search and copy see them reversed. Office layouts keep slide text boxes that were placed outside the slide. PDF.js and PDFOxide return Hebrew and Arabic text runs in visual order.

We also kept the “Try a sample” files as they were. Samples are there to show a result in 1 to 3 seconds, and a stress file would make them slow.

If one of our tools reads your file wrongly, the “Needs work” button under the tool tells us which tool and language had the problem, without sending the file.

PDF Info is in the Laboratory’s PDF workspace on laboratoru.com; switch the page to Hebrew or Japanese to try the fix.