A torn corner, a worn-off label, a chipped edge. Which object did it come from?
Enrol the whole object once. Later, show Residue whatever survived. It finds where the piece sits on the enrolled surface, returns the original record, and says cannot tell when the evidence is thin or the pieces disagree. The same memory tracks how the object's condition changes over time against a fixed original.
From a fragment, on real cards
Crops cut from real Library of Congress scans, searched against a gallery of look-alike cards from the same print runs. A 32 by 32 patch of a card normalised to 288 by 512 pixels is about 0.7 percent of its surface.
Top: a 32 by 32 crop found its source card by fine-detail comparison, score 0.924 against a frozen threshold of 0.845, a gap of 0.469 to the runner-up. Middle: the whole-picture method accepted an unrelated card from a 128 by 128 crop. That is why Residue does not use it. Bottom: an 8 by 8 crop is refused before any search.
| Method | Fragments of a known card | Correct | Wrong | Refused |
|---|---|---|---|---|
| Whole-picture comparison | 36 crops | 26 | 0 | 10 |
| Fine-detail comparison | 36 crops | 35 | 0 | 1 |
| Fine detail plus three-region agreement | 36 crops | 36 | 0 | 0 |
The same test also sent in 36 crops from cards that were never enrolled but share the design. Every method accepted 6 of them. On real scans, recognising a known fragment works; rejecting a never-seen look-alike is not yet solved by fine detail alone. The guided method below is the fix, and so far it has only been run on synthetic surfaces.
Guided recovery: ask for the right second piece
Photograph the object three times at enrolment. Learn which small areas keep looking like themselves. When a damaged piece comes back unclear, ask to see the most useful surviving area next. Then require every piece to independently support the same object.
| Method | Known reads recovered, of 432 | Unknown objects accepted, of 360 | Mixed-origin pieces accepted, of 216 |
|---|---|---|---|
| One ordinary image patch | 1 | 0 | — |
| One patch, lighting corrected, shared design removed | 113 | 2 | — |
| Two patches, fixed second place | 168 | 0 | 39 |
| Two patches, stable and distinguishing second place | 211 | 0 | 35 |
| Same, and both pieces must agree on their own | 51 | 0 | 0 |
72 simulated objects, three held-out galleries, thresholds frozen on different objects before scoring. Picking a stable second patch beat a fixed one in all three galleries. Demanding that both pieces agree on their own kills the forgery that glues one authentic scrap onto something else, at the price of many more "cannot tell" answers. That trade is the product: a hold is cheap, a wrong record is not.
What the record shows after a recovery: only the pixels actually observed are coloured. Nothing is filled in, completed or inferred.
Four pieces, one rule
Each piece is located on the enrolled surface and scored on its own. Pieces that point at different objects produce a refusal, not an average. Duplicate pieces do not count twice. Unknown pieces stay unassigned. Coverage reports only what was seen. When two candidates stay close, the system names the unobserved region whose detail would separate them.
Condition tracking against a fixed original
Identity is enrolled once and never rewritten. Condition baselines are appended after a human review. Every later observation is compared with both, so a change is tracked without weakening the identity it is tracked against.
identity reference fixed
changed area: 30 × 12 px region
changed fraction: 0.15 percent
diagnosis: not assessed
vs original: change observed
vs approved baseline: no change
identity still version 1
The system reports where intensity changed and by how much, in normalised units. It does not call a change a scratch, a repair or a defect; that stays with the reviewer. A different capture setup is refused for comparison rather than compared badly. History is append-only with a hash chain.
More than one sense, and a challenge an old photo cannot answer
Backlight
A separate enrolled channel through the material. When the ordinary surface is unreadable, a supported backlight read can carry the identity on its own. Conflicting channels are never averaged into a match.
Registered CT volume
Already-registered 3D volumes compared on local detail against every compatible candidate, with an ambiguity gap. Known volume resolved, unknown refused, incompatible grid refused.
One-use lighting challenge
Three to eight light-response images at enrolment. A random 16-frame sequence bound to the item and the moment, consumed once. An old sequence under a new challenge fails: temporal correlation 0.11, refused.