Let's say I'm updating the page text on a large document like 100 pages in size. Currently, the patching of these pages must happen in chunks like the Azure Add-On does:
https://github.com/MuckRock/documentcloud-azure-document-intelligence-ocr-addon/blob/0a80e6bf6d922fb080b09563279da7de54b6278e/main.py#L105
Currently, on the last call to setting the page text 3 three things can happen:
- The API call to set the page text happens -> a task is created -> quickly picked up document goes from status success/pending/success and the document is back to success status, before the API call to set the tag is called. The call to tag is made and sticks. Document is in correct condition.
- The API call to set the page text happens -> a task is created -> it takes a while to process, the document is in readable status, the Add-On sees that the document is pending, so it doesn't call tag. The Add-On waits until it is success again and applies the tag. This is special handling in the Add-On that shouldn't have to happen.
- The API call to set the page text happens -> a task is created -> it takes a while for this to get picked up -> the document is in success status. The Azure Add-On sees the status is success, so it thinks the page text is good, so it makes the API call to add the key/value pair. This call succeeds and the document momentarily has the data saved. The page text task finally finishes and does a bare save(), except the version of the document it has in memory is one that is already out of date. It doesn't have the key/value pair applied. It gets saved, overwriting the key/value pair we just applied.
The Add-On's status-polling can't reliably avoid this: the task sets status = success inside the same finally block as the clobbering save, so observing success from outside doesn't guarantee the save has happened. With multiple page-text chunks (multiple tasks), a success reading can also come from a later chunk while an earlier chunk's save is still pending.
Proposed fix:
Scope both save() calls in set_page_text to the fields the task actually intends to change.
Let's say I'm updating the page text on a large document like 100 pages in size. Currently, the patching of these pages must happen in chunks like the Azure Add-On does:
https://github.com/MuckRock/documentcloud-azure-document-intelligence-ocr-addon/blob/0a80e6bf6d922fb080b09563279da7de54b6278e/main.py#L105
Currently, on the last call to setting the page text 3 three things can happen:
The Add-On's status-polling can't reliably avoid this: the task sets status = success inside the same finally block as the clobbering save, so observing success from outside doesn't guarantee the save has happened. With multiple page-text chunks (multiple tasks), a success reading can also come from a later chunk while an earlier chunk's save is still pending.
Proposed fix:
Scope both save() calls in set_page_text to the fields the task actually intends to change.