Why One AI Model Is Not Enough for Bookkeeping

One AI model is not enough for bookkeeping because it can give a wrong answer with full confidence, and nothing in its output tells you it has done so. Bookkeeping errors are quiet: a miscoded expense or a misread VAT figure looks exactly like a correct one until the reconciliation fails or the accountant finds it at year end. Several independent models, combined with rule-based checks and a human review queue, give you a signal a single model cannot: whether to trust this particular entry.

The core problem: a single model has no reliable "I am not sure"

Large language models are trained and evaluated in ways that reward a plausible answer over an admission of uncertainty. Research by Kalai, Nachum, Vempala and Zhang, published with OpenAI, argues that hallucinations persist because standard training and evaluation procedures reward guessing over acknowledging uncertainty. On a benchmark, a guess sometimes scores and a blank never does, so models learn to guess.

For bookkeeping, this matters more than raw accuracy. A model that is right most of the time but never flags the entries it guessed on leaves you two bad options: review everything by hand, which removes the point of automating, or review nothing and absorb the errors. The ICAEW's generative AI guide for accountants makes the same point in professional terms. It warns that models can create false data and "confidently rely on it", and that they can contradict themselves when challenged.

The ICAEW guide also names the human side of the risk: automation bias, where people become overly reliant on technology output without questioning it. A clean, well-formatted journal entry from an AI tool invites approval. The more polished the output, the less likely a busy reviewer is to catch the one wrong nominal code in fifty.

Why bookkeeping is especially unforgiving

Many AI tasks tolerate some error. A slightly weak email draft gets edited. Bookkeeping is different in three ways.

Errors compound. A wrong posting affects the trial balance, management accounts, VAT or sales tax returns and any decisions based on them. The cost of an error grows the longer it sits.
There is no immediate feedback. Nobody checks a supplier invoice against the ledger the moment it is posted. The error surfaces weeks later, by which time the source document context has been lost.
A confident wrong answer is worse than no answer. An entry marked "needs review" costs a few minutes. An entry silently posted to the wrong account costs an investigation.

That last point is the design principle we work from. In a bookkeeping pipeline we built for Pixelet, it shaped the whole build: no single model gets the final say.

How multiple models solve it

Instead of asking one model to read an invoice or bank line and produce the entry, you ask several models to do it independently, then compare their answers field by field. When they agree, the entry is accepted. When they disagree, the entry goes to a person.

This works because correct answers tend to converge and wrong answers tend to scatter. The same principle appears in published research. The self-consistency method from Google researchers samples several reasoning paths from a model and takes a majority vote, and it produced sizeable accuracy gains on arithmetic and commonsense reasoning benchmarks. Work on combining different models goes further: an Uncertainty-Aware Fusion framework presented at the ACM Web Conference 2025 reported an 8% improvement in factual accuracy over other hallucination mitigation methods by combining multiple LLMs based on their accuracy and self-assessment. The ICAEW guide even suggests a manual version: ask the tool the same question repeatedly, because inconsistency exposes fabrication.

The real value for a finance team is not only higher accuracy. It is that disagreement becomes a usable flag, which gives you a built-in triage system:

Entries where all models agree and rule checks pass are posted, or queued for light sampling review.
Entries where models disagree on a field are routed to a person, with each model's answer shown side by side.
Entries that fail a hard rule are rejected regardless of agreement.

In the Pixelet pipeline, several AI models process each entry independently, and an entry is accepted only when the models agree. That replaced a process where entries were handled by hand, one at a time, without moving to the risk of trusting one model blindly. You can read more in the Pixelet case study.

The catch: models can agree and still be wrong

Agreement is strong evidence, not proof. A 2025 ICML paper, "Correlated Errors in Large Language Models", evaluated more than 350 models and found that when two models were both wrong, they picked the same wrong answer about 60% of the time on one leaderboard dataset, against a random baseline of roughly 33%. Models from the same provider or with the same base architecture were more correlated. More troubling, the authors found that more accurate models had more correlated errors, even after accounting for shared provider and architecture.

In practice, this means three models from the same vendor, or three prompts to the same model, give you less independence than it seems. A multi-model bookkeeping design needs to account for this:

Mix providers and model families. Shared training data and architecture produce shared blind spots.
Vary the inputs where you can. For example, one path reads the document image directly while another works from extracted text. Different routes to the same answer fail in different ways.
Do not let models check each other's work as the only control. The same paper found that when one model judges another, shared errors lead the judge to inflate its rating of less capable models.
Back agreement with rules that do not depend on any model.

The deterministic checks that should sit alongside the models

A sound AI bookkeeping system treats models as readers and classifiers, and leaves arithmetic and policy to ordinary code. Useful checks include:

Arithmetic: line items sum to the subtotal, tax at the stated rate matches the tax amount, and debits equal credits.
Master data: the supplier exists in your records, the bank details match what you hold, and the nominal code is one your chart of accounts allows for that supplier.
Duplicates: same supplier, invoice number and amount already posted.
Dates and periods: the invoice date falls in an open period and is not in the future.
Thresholds: entries above a set value always go to a person, whatever the models say.

These checks catch the cases where models agree on something impossible. They also give your accountant an audit trail that does not depend on explaining a model's reasoning.

Design decisions a buyer should understand

What counts as agreement

Agreement should be defined per field, not per document. Two models might agree on supplier, date and total but differ on the nominal code. The system should accept the agreed fields and flag only the disputed one, so the reviewer sees exactly where to look. Free-text fields such as descriptions need normalisation before comparison, or trivial wording differences will flood the review queue.

How many models

Two models can only tell you they disagree. Three or more let you take a majority, though for bookkeeping a unanimous rule is often safer for high-impact fields such as amount and account. The right number depends on your error tolerance, volume and budget.

Cost and speed

Every extra model multiplies inference calls per entry, so running costs and processing time rise with the number of models and the size of the documents. The factors that drive the total are entry volume, document length and quality, which models you use, and how much of the work you route to cheaper models for simple fields. The trade-off to weigh is that extra per-entry cost against the staff time spent finding and correcting errors that a single model would have posted silently.

The review queue

The disagreement rate is the number to watch. Too high, and staff spend their day reviewing, which suggests prompts, document quality or model choice need work. Close to zero on varied real documents may mean the models are too similar to disagree usefully. Plan for people to own the queue, and log every human correction so the system can be tuned.

Ownership and auditability

Your accountant or auditor may ask why an entry was posted. Keep the source document, each model's output, the rule check results and any human decision together. If the system is built in code you own, you can change models as better ones appear without rebuilding the controls around them.

Questions to ask before you buy or build

Does any single model have the final say on an entry? If so, what catches its mistakes?
Are the models from different providers, and how does the system handle correlated errors?
Is agreement checked per field, and which fields require unanimity?
Which checks are rule-based rather than AI-based?
What happens on disagreement, and who sees it?
What is the expected review rate on your own documents, measured in a pilot rather than claimed?
Can you export a full audit trail per entry?

When one model is enough

Not every finance task needs this machinery. Drafting a payment reminder, summarising a supplier statement for a person to read, or suggesting categories that a bookkeeper confirms before posting can all run on one model, because a human is the final check by design. The multi-model approach earns its cost where the system writes to the ledger without a person reading every entry. That is exactly where a single model's silent errors are most expensive.

If you are weighing AI for invoices, receipts or bank feeds, our AI document processing work covers pipelines of this kind, built in code and handed over so you own them.

Talk to us about your bookkeeping workflow

If you process entries by hand today, or you have tried a single-model tool and do not trust what it posts, tell us about your volumes, document types and accounting system. Send us a brief and we will tell you how a multi-model pipeline would fit your workflow.