When a batch job hits a file it cannot read, it has two options: end the run, or make the failure part of the output. We picked the second one, and that changed where the work went.
The problem is familiar. You point a tool at a directory, it processes two hundred files, and one of them is a half-written export with a broken byte sequence in the middle. If reading throws, you lose the other one hundred ninety-nine results. If you wrap the read in a try and move on, you get all your results and quietly lose the knowledge that anything was skipped. The second failure is worse, because nobody notices it.
In proseprobe, our open-source text-scoring project, the batch entry points walk a tree, keep what the filesystem confirms is a file, sort the result, and hand the list to one shared reader. The reader works out the document name before it tries to read anything: relative to the base directory when there is one, bare filename when the caller named individual paths. Then the read happens inside a try, and two exception types are caught together, the OS refusing the file and the decoder refusing its contents. Different causes, identical consequence: there is no text to score.
The caught failure gets appended as a record carrying the name, the path, and the text of the error. The loop continues. Only a successful read reaches the scorer.
The useful part is that the report then holds two lists side by side, scored documents and unread files. The headline states how many files could not be read. The closing notes name them as not scored. The machine-readable summary carries them as structured records, so a downstream job can see the gap instead of inferring it from a count that looks low. And when reports are written to disk, unreadable files get no report file of their own, but the summary file is still written. One more honest detail: the sibling function that scores prose already held in memory passes an empty unread list, because nothing was read and nothing could fail.
The trade-off is real. Partial success means callers have to check two fields instead of trusting one exit code, and a run full of unreadable files looks like a success until you read the headline. We took that over a pipeline that stops on file one hundred and forty-three.
If you run batch jobs in production, it is worth checking whether your failed items end up in the output artifact or only in the logs.