BFSI prototype
The prototype that became the spec
A policy-management prototype for banks where every trust-critical decision was moved out of the model and into architecture, which is why there's no confidence score in the codebase.
Same question, different person, different answer
A staff member asks the assistant for the personal-loan policy and gets it. They ask what a colleague earns and get not permitted. An admin asks the same two questions and gets two answers. That's the entire product in one interaction, and it's the moat to validate against the horizontal players, because a general-purpose company assistant that can't do this can't be sold to a bank.
The product is a policy-management platform for banks; the prototype's job was to prove the permission-faithful version of it was worth building. It had one month, one constraint that shaped everything (it had to run, live, at a URL a prospect could click), and a decision budget spent almost entirely on a single question: what is the model allowed to be wrong about?
Figure 1. One question, two personas, two answers. The refusal on the right is not the model declining — the policy text was never in its context window, and the chip underneath names the scope it fell outside of.
The prototype running locally against a regenerated org: fictional company, fictional staff, addresses on a reserved domain. The access rules, the refusal copy and the citation chips are the prototype's own, unedited.
- 8,067
- lines, zero dependencies, no build step
- 15
- views registered on a three-line module system
- 5
- personas, each seeing a different product
- 11
- models across 6 providers, none preselected
No framework, and no framework rationale either
There is no comment in the codebase explaining why it's vanilla JS, and I'm not going to invent one retroactively. The argument is in the structure. The module system is three lines: a views object and a register function. Every view is a pure function from state to an HTML string. There is no package manifest, no lockfile, no build; deployment is pointing a static host at the folder.
What that bought was the thing the month actually needed: the PRD links to the prototype as its own spec. Open this to see the target UI is a sentence a document can only say when the prototype deploys anywhere instantly and breaks for no environmental reason. The roadmap treated "prototype link frozen" as a hard milestone, which inverts the usual order: the prototype came first, and the document was written to describe it.
In place of a test framework there's a 258-line headless harness that stubs a minimal DOM, loads every view, and renders each one for every persona — the cross-product that matters for a product whose thesis is "the answer changes by persona."
Figure 2. One of the fifteen views, rendered by a function that returns a string. Nine policies here because the viewer is an administrator; the same route returns four rows for the staff user in Figure 1.
The prototype running locally. Policy names, categories and version numbers are the prototype's own seed; every owner and every identity is regenerated.
Where the model is allowed to be wrong
The prototype connects to eleven models across six providers, bring-your-own-key, none preselected. And nowhere in the codebase is there a confidence score, an output validator, or a retry-on-malformed. That absence is the design.
Figure 3. Where the gate is. The filter runs before the prompt, not after the answer — so the model's context contains only what this persona may read, and a hallucination has nothing out-of-scope to hallucinate from. Note the second path: every number the bank would act on is routed around the model entirely.
The model can't leak what it never received
Permission filtering happens before the prompt, not after the answer. The context builder assembles only the policies the signed-in persona may see, then tells the model how many others exist and that they are out of scope. A hallucinating model still cannot reveal an out-of-scope policy, because that text was never in its window.
- →The prompt's one rule — answer only from the context; if something is absent, say the user is not permitted to see it — never guess
- →The product claim — log in as staff and the model literally can't answer about a policy they're not on
Numbers come from deterministic code
Every quantitative claim — approval rates, NPA projections, who flips under a rule change — comes from the simulator, which is plain arithmetic over a labelled test cohort. The model is never asked to compute anything, so it cannot be confidently wrong about a number a bank would act on.
A human always decides
Everything terminal routes to maker–checker. A simulated policy change becomes a pending change request, not an applied edit. The regulatory reviewer downloads the revised policy PDF to sign through their own approval workflow — deliberately not an in-app approval, because the tool refusing to own the final yes is what lets a compliance team adopt it.
The one surface left to the model
Wording, summaries, and the trailing source-citation line the UI scrapes into chips. If the model formats sources wrong, chips don't render and nothing else degrades. The blast radius of a bad completion is cosmetic — which is the whole point of the other three walls.
Figure 4. Three hard walls, one soft surface. The gate is architectural, not statistical.
There's also a third state between demo and live that turned out to be a sales feature: picking a model persists and shows in the header without a key. A prospect can see their preferred provider in the chrome during a demo; adding a key upgrades the same choice to live answers. Separating chosen from keyed cost one boolean and removed a whole category of demo awkwardness.
Citations correct by construction
The prototype's answers cite pages, clause 3.2, page 2, with a chip that opens the document at that page, clause highlighted. The honest engineering behind that is an inversion worth stating plainly: there are no real PDFs in the prototype. Getting from a real document to a reliable page number needs a parsing pipeline the month didn't have. So the prototype goes the other way: it synthesizes a paginated document from the policy's own structured data, which means every page number is correct by construction, because the pagination and the citation come from the same source.
The header comment in that file says exactly what it is (no real PDFs in the prototype; we synthesize a paginated document from a policy's own content), and that sentence is why the trick is honest. Real ingestion is scoped for the build phase. The prototype's job was to prove the citation experience, and it proved it without pretending to a parsing capability that didn't exist.
The simulator: evidence you attach to a change request
The most sellable idea in the prototype is small: before a bank changes a credit-policy threshold, show them who flips. The simulator re-runs a policy's eligibility rules over a fixed 220-applicant test cohort with ground-truth default labels, under both the live and the proposed parameters, and reports the delta on the book.
Figure 5. The simulator as it ships, with the CIBIL floor moved from 700 to 720: approval down 4.1 points, projected NPA down 2.6, nine applicants reclassified. Note the button in the bottom right — it is there because nine people moved. At 700, the same panel renders without it.
The prototype running locally. The eligibility engine, the metrics and the 220-applicant test cohort are the prototype's own; the cohort carries no PII by construction.
- Approval rate
- 16.8% → 16.8%flat
- Projected NPA
- 10.8% → 10.8%flat
- Reclassified
- 00 lose approval · 0 gain it
Approved, by CIBIL band
No one flips at these settings, so the prototype shows no Propose button — a change request with no impact attached is noise in an approver's queue.
Figure 6. The same engine, ported — so you can move the floor yourself. Tighten the CIBIL cutoff and watch approval fall, projected NPA fall, and the flip count climb.
Same eligibility engine and metrics as the prototype's sim.js. The 220-applicant cohort is regenerated synthetically with the same distribution shape and deterministic labels — none of the original rows. Projections model the test cohort, not a live book.
The design detail I'd defend hardest is the propose button's existence condition: it only renders when the change actually reclassifies someone. A rule tweak that moves nobody produces no change request, because a request with no impact attached is noise in an approver's queue. When it does render, the simulated deltas get written into the request's rationale automatically — the simulation is the evidence, filed where the approval will happen.
Scope was cut to fit the month, on purpose: five shared knobs across the four credit policies, not fully custom per-policy variables. The other five policies get an explainer instead of a broken form. The roadmap's phrase for this was build it cleanly, separate from the half-baked old test bed, a small sentence that is most of what I believe about prototypes.
What the prototype changed, and what it took to show it
Three reversals are traceable from document to code, which is the prototype-as-spec workflow paying for itself. A two-edition packaging model was defined, then retired to a single version with connectors parked behind no-op stubs. A standalone people directory was built, then explicitly disowned in the docs and merged into an access-centric view. And compensation answers — the flagship demo of permission-faithful retrieval — were removed entirely, one line: visible to no one. The docs still use salary as the example; the code overruled them. When a document and a running program disagree, the program is the one telling the truth.
One gap runs the other way, and it's the sharpest lesson here. The audit log was a phase-one, top-priority feature (audit is also the compliance sell) and it was never built in the app. Then a later standalone demo made an append-only audit ledger its most prominent surface. The demo prototyped the missing feature the product never got, which says something uncomfortable about demos: they get built for what the audience in the room needs to see, not for what the roadmap says comes first.
And what it cost to put the prototype on this page at all: its seed data was never synthetic. It carried a live org chart, real work addresses, real internal product names, and named prospects, straight through to the deployment docs — so for a long time the honest thing to publish was a description of the prototype rather than the prototype.
What runs behind every screenshot above is the real program with a new cast. An aliasing pass regenerates the whole directory before the app boots: sixty-five identities replaced, employee ids kept intact so the policy owners, approval levels and Jira assignees still resolve to the same people-shaped rows, addresses moved to a reserved domain, product names aliased the way this page's prose already aliases them, and a check that fails the run if a single real name survives into a file the browser loads. The code is untouched. Only the cast is fictional, and this paragraph is here because a reader deserves to know which is which.
What I'd do differently
Seed data deserves the same discipline as code. Using the real directory made every demo instantly legible internally, and it welded the prototype shut for any external use. The fix was exactly what it should have been on day one — a generator producing a fictional org with stable ids — and it cost an afternoon, which is the whole point: an afternoon spent at the start would have bought a publishable artifact for free, and instead it got spent a year later, under a deadline, to make one page's figures possible.
The abstain path deserved a gate, not a prompt. The deny behaviour (say the user is not permitted, never guess) rides on a prompt instruction. The context filtering makes real leaks structurally impossible, but the polite refusal itself is unverified model behaviour. One post-check, that an answer must be a refusal whenever the context said "none in scope", would have converted the last soft wall into a hard one.
Build the audit log when the roadmap says to. It was ranked correctly, deferred anyway, and then needed urgently by the demo that came later. Features that exist to create trust are the worst possible candidates for deferral, because by the time you need them the absence is the story.