From a pile of 1,554 unreadable scans to a searchable, reconciled archive
A case study in making a scanned records collection searchable, and checking it against its own inventory, without touching the originals
At a glance
| Situation | A government field office holds 1,554 scanned land-record folders, about 15 GB, with no way to search them and an inventory spreadsheet nobody had checked against what was actually scanned |
| Approach | A nine-stage, repeatable pipeline that inventories, matches, reads (OCR), classifies, groups and maps the collection, working read-only on the source files |
| Window | 22 July to 6 August 2026, from first commit to a management presentation and a set of recommendations |
| Output | 28,892 pages with searchable text, a full-text search tool, a reconciliation against the official inventory, a per-property catalogue, a map, an aggregate-only management report and eight recommendations |
The situation
The collection is a set of scanned folders, one per parcel or claim, spread across 72 boxes and folders on disk. A hand-kept spreadsheet of 5,392 rows was supposed to be the official index. Before this work there was no reliable way to answer basic questions: what is in the pile, does it match the index, and where are the records for a given property?
Four features made it harder than it sounds:
- Every file is an image. None of the 1,554 PDFs has a text layer, so nothing could be searched by content.
- Three naming schemes. Boxes are labelled by number, by a different code that matches the spreadsheet, or by municipality, and a catch-all folder holds items pulled out at some point. This reflects how the physical boxes were labelled over time, so it is not an error to fix.
- Hand-typed filenames. Names follow a rough pattern (parties, lot, survey number, place) but vary: disputes between parties, "heirs of", "part 1" and "part 2", and missing words.
- Personal data. The folders contain real people's names and property records, so the work had to be careful about what leaves the machine.
The goal was modest: know what is in the pile, find the gaps against the official index, and make each document findable by name, lot, location or content. It was not to edit documents or manage land records.
How the work was run
The working rule was read-only on the originals. Nothing in the source folders was moved, renamed or changed, and everything the pipeline learned went into a separate database.
- Inventory. Walk every file and record its path, box, size, page count, a content hash and whether it has a text layer. Re-runs skip unchanged files, so the same step doubles as a "what is new" scan.
- Parse filenames. Pull party names, lot, survey number and location out of the hand-typed names. This works on about 98% of files for the lot number and 97% for the survey number, and is treated as an aid, not as truth.
- Reconcile with the official spreadsheet. Match each spreadsheet row to a scanned file using the lot as a pre-filter and fuzzy name matching with an accept threshold. The spreadsheet itself needed correcting on the fly, because in about 79% of rows the survey number had been typed into the lot column.
- Read every page (OCR). Each page is rasterised and run through open-source OCR into a database, resumable by file and by page, because this is the one slow stage.
- Build a search index. Filenames and page text go into a full-text index, queried from a small command-line tool that supports lot, box and boolean searches.
- Classify documents. Rules over the OCR text label each file's lead document type, such as certification, indorsement or certificate of title.
- Group by property. Files are grouped to one row per property by lot plus survey number, since a lot number alone repeats across unrelated parcels.
- Map it. Properties are placed by barangay and municipality on a map. There is no survey geometry in the collection, so the map shows concentrations, not parcel boundaries.
- Report. An executive summary with no names and no per-parcel detail, safe to present beyond the small group who need row-level data.
Every stage can be re-run end to end. The inventory, filename parsing, reconciliation and indexing stages take under two minutes in total over the whole collection, and OCR is the one slow stage.
What it found
| Finding | Result |
|---|---|
| Scale | 1,554 files, 28,892 pages, 15.4 GB, all OCR'd |
| Duplicates | None, checked by content hash rather than filename |
| Scans that match the official spreadsheet | 1,263 of 1,554 (81.3%) |
| Spreadsheet rows that match a scan | 1,314 of 5,392 (24.4%) |
| Distinct properties | 1,402, from 1,495 groupable files |
| Properties with more than one scanned file | 73, of which 43 are split across different boxes |
| Files too blank or illegible to read | 45 (2.9%) |
| Files that could not be grouped to a property | 59 (3.8%) |
| Map units placed | 1,453 of 1,461 (99.5%), where a unit is a property or an ungrouped file; 58% have a municipality but no barangay, so barangay counts are a floor |
Three results shaped the recommendations:
- The asymmetry is the finding. The scans map well to the spreadsheet, but the spreadsheet describes about 3.5 times more records than have been scanned. That is either a large body of unscanned paper, or an index that covers a wider scope than this batch. The report says to settle the scope before treating the gap as a to-do list.
- Most files are supporting paperwork. Only 63 files (4.1%) are an actual certificate of title. Indorsements, certifications and affidavits make up most of the rest. Title-bearing documents in all (titles, deeds of sale and similar) total 95, a short list for anyone verifying ownership.
- The collection is concentrated. One city accounts for the large majority of properties, and a single barangay in it holds the most, which tells a follow-up field effort where to start.
Where it could have gone wrong, and how it was handled
- Heuristics are not facts. Filename parsing, fuzzy matching and document classification are all pattern matching. The work says so in the deck and in the project notes, and every finding traces back to specific files so any figure can be spot-checked.
- Unmatched does not mean missing. The 291 scans with no spreadsheet row are concentrated in the catch-all folder (about half) and in areas that appear to fall outside the spreadsheet's coverage. They are reported as a scoping question, not as lost or unauthorised records.
- Lot numbers repeat. Early grouping by lot alone would have merged unrelated parcels, for example one lot number that appears on six different parcels. Grouping by lot plus survey number fixed that, and files without a usable pair were left ungrouped instead of being forced into a match.
- Classification looks at the lead document only. Each PDF is a compiled dossier, so the label names the dominant document type, not everything inside.
- Personal data stayed local. The database and every per-parcel output are treated as containing personal data and kept off any remote. Only the aggregate management report was designed for wider circulation.
What was left open
- The scope of the official spreadsheet, and therefore how much remains to be scanned, still needs an answer from the people who maintain it.
- The 45 unreadable scans need a physical check for a better copy.
- The 59 ungrouped files need a short manual pass to add the missing identifier.
- The map shows barangay-level concentration only, because there is no cadastral data in the collection.
Recommendations delivered
- Scope and reconcile the inventory-versus-scans gap first, since it sizes any further digitisation.
- Fix the lot and survey column mix-up at the source, with a validation rule.
- Physically reunite the 43 properties split across boxes.
- Rescan the 45 blank or illegible files.
- Backfill identifiers for the 59 ungrouped files.
- Verify the 95 title-bearing documents first if capacity is limited.
- Put the search index into day-to-day use, and re-run the pipeline whenever new scans arrive.
- Start any field work in the area that holds most of the records.
Takeaways
- Never touch the originals. Working read-only made every later step safe to repeat and every result easy to defend.
- Make the slow step resumable. OCR over nearly 29,000 pages was designed to run in batches and pick up where it stopped, so the long stage never blocked the quick ones.
- Reconcile in both directions. Matching scans to the spreadsheet and the spreadsheet to scans gave two different answers, and the difference between them was the main insight.
- Treat pattern matching as triage. Saying plainly that parsing, matching and classification are aids made the results more credible, and produced short, concrete work lists instead of a vague claim of completeness.
- Keep an aggregate-only version. A report with no names and no per-parcel detail let the findings be presented widely while the sensitive data stayed put.
About these numbers
Counts come from the project's own inventory database and the figures in its management presentation, which were produced in early August 2026. Matching rates depend on a fuzzy threshold and are not legal findings. Document types are assigned by rules over OCR text and are not authoritative. Place names, party names and the agency are deliberately left out of this write-up.