Currently, if a document has an underlying text layer, we do not graft the text back into the PDF because it causes memory issues. Unfortunately, this also means that under the current implementation that page text positions are also no-oped.
This results in documents that have their page text replaced, but not their positions. This should be avoided.
We should set the page position text (not an operation that causes memory issues) before we check for underlying text and no-op the grafting.
- Write the position JSON files always
- Gate only the PDF text-layer graft on visible_text as it is now. That's the part that is experiencing memory issues which we can tackle in another issue.
This decouple of grafting and setting page text positions is something I originally brought up on Slack in April 2025: https://muckrock.slack.com/archives/C030N6P6DFX/p1744854634016639?thread_ts=1744834425.939369&cid=C030N6P6DFX
This is affecting legitimate workflows from our partners at Disclose. When I call the API to update the page text from an Add-On, I expect that the page text positions get updated too.
Currently, if a document has an underlying text layer, we do not graft the text back into the PDF because it causes memory issues. Unfortunately, this also means that under the current implementation that page text positions are also no-oped.
documentcloud/documentcloud/documents/models/document.py
Line 498 in 885b897
This results in documents that have their page text replaced, but not their positions. This should be avoided.
We should set the page position text (not an operation that causes memory issues) before we check for underlying text and no-op the grafting.
This decouple of grafting and setting page text positions is something I originally brought up on Slack in April 2025: https://muckrock.slack.com/archives/C030N6P6DFX/p1744854634016639?thread_ts=1744834425.939369&cid=C030N6P6DFX
This is affecting legitimate workflows from our partners at Disclose. When I call the API to update the page text from an Add-On, I expect that the page text positions get updated too.