Case Study · Document AI & Trust Infrastructure
Attest — Lease Intelligence with a Verifiable Trust Layer
A commercial lease abstraction tool where every extracted field is grounded, verified, and cited back to its source page before it's shown as fact — built to close the gap between AI adoption and AI trust in commercial real estate.
Next.js 16TypeScriptClaudeSQLite · Drizzlepdf.js
01
The problem & requirements
Functional
- Extract ~18 lease-economics fields across six groups: parties & premises, term, rent & escalation, options & notice, expenses, risk clauses
- Derive critical dates (notice windows, expiration) and risk flags from verified extraction data — never from a raw model guess
- Every field click-to-source: one click from any value to its exact page and cited passage in the source PDF
Non-functional
- Groundedness: no extracted value shown as fact without a citation verified against the source page
- Refuse, don't guess: a derived date on an unverified input field blocks with a stated reason rather than computing anyway
- Reproducibility: every extraction versioned by
run_idandprompt_version, so any two eval runs are diffable field by field
02
Scale & constraints
Built and scored against real, publicly filed commercial leases — not synthetic data.
10
real commercial office leases seeded from SEC EDGAR EX-10 exhibits, processed end to end
126
hand-labeled gold fields across 7 documents — the ground truth the eval harness scores against
57
passing tests, including pure-function unit coverage on the date-derivation engine
03
Pipeline design
// six stages, each independently callable and testable
ingest(pdf) → pages[] // pdf.js text + coordinates, no OCR needed
extract(pages, fieldGroup) → candidate[] // two-pass: route pages, then extract per group
verify(candidate) → {value, confidence, evidence} // grounding + verifier pass
persist(verified, runId, promptVersion) → extraction // immutable, versioned rows
derive(extractions) → {dates[], flags[]} // pure functions, no LLM call, blocks on low confidence
surface(document) → reviewUI // PDF + extraction side by side, click-to-source
04
Data model
| Entity | Key fields |
|---|---|
| documents | id, slug, type, status, source_path |
| pages | document_id, page_number, text, item_index |
| extractions | field, value, evidence_text, page, confidence, run_id, prompt_version |
| derived_dates | document_id, kind, date, status (ok / blocked), reason |
| risk_flags | document_id, flag, present, source_extraction_id |
| gold_labels / eval_runs | field, expected_value, run_id, field_score, date_score |
run_id + prompt_version on every extraction is what makes a score regression traceable to the exact prompt change that caused it — without it, "accuracy dropped" has no owner.
05
Architecture
A document either seeds from fixtures or is uploaded; both paths run the same six-stage pipeline. The eval harness reads through the same persistence layer as the app — never a parallel scoring path.
⇧
Ingest
pdf.js → text + coords
→
⇄
Extract
Claude, two-pass
→
✓
Verify
grounding + verifier pass
↓
▤
Persist
SQLite, versioned
→
ƒ
Derive
dates + risk flags
→
⧉
Surface
review UI
↳ eval harness reads Persist directly — 20-doc gold set, field + derived-date accuracy, diffed by run
Fig. 5a — six-stage pipeline; the trust layer (green) is where grounding and verification gate every field before it reaches the UI.
06
Key decisions & trade-offs
Risk flags, not a composite score. A weighted risk score implies validated weights nobody — this build included — can actually justify. Individually-cited presence/absence flags let the reviewer judge risk instead of trusting an unverifiable number.
Owner/asset-manager point of view, stated explicitly. Clauses like co-tenancy read as risk to one party and protection to the other. A perspective-agnostic tool isn't a neutral default — it's roughly double the scope. Named the lens in the UI rather than silently picking one while claiming objectivity.
Block, don't guess, on derived dates. If an input field feeding a critical-date calculation falls below confidence threshold, the date is
blocked with a stated reason — not computed. Date errors compound silently across multiple fields; refusing is safer than guessing.Direct Anthropic API, not Bedrock. Production would run on Bedrock for VPC posture and data residency. This build optimizes for a reviewer running it without provisioning AWS — a deliberate trade made for the audience, not a technical limitation.
SQLite, not Postgres. Zero setup — the seeded database ships in the repo so the app runs with no external dependency and no API key required to browse existing documents.
Single document type: office leases only. No retail, no industrial, no amendment-chain resolution. Narrowed scope to put the majority of build time into the verification layer rather than extraction breadth.
07
Lessons learned
💡 What actually held up
- Grounding catches fabrication cheaply. String-matching evidence text against the source page is deterministic and free — it rejects a real class of hallucination before a value is ever persisted.
- The honest-gap split proved the thesis. Parties & premises scored 94% grounded; risk clauses scored 0% — not random error, but the system correctly refusing to assert an absent clause as a confirmed fact.
- Two-pass extraction paid for itself. Routing pages to field groups before extracting sharpened accuracy and cut cost versus one long-context prompt over the full document.
↺What I'd do differently
- Ship self-consistency scoring, not defer it. One of three planned trust signals was cut once grounding and the verifier pass caught most failures — a real gap that should close before this handles higher-stakes fields.
- Give extraction a way to say "confirmed absent." The current schema can't distinguish "not found in routed pages" from "clause confirmed absent" — the direct mechanism behind the 0% risk-clause number.
- Sequence backend before UI more strictly. Building review-workspace screens ahead of the wired backend was a deliberate, logged deviation — it worked, but it's a sequencing risk I'd tighten next time.
🔁If I started this over: I'd design the negative-claim extraction path and lock the confidence threshold before writing a single prompt — both were discovered mid-build instead of decided up front.
System
Attest — Lease Intelligence
Primary services
Claude · pdf.js · Drizzle
Status
Built — portfolio demo
Type
Document intelligence pipeline