Preventing Enterprise AI Agent Failures — the "20% AI, 80% Deterministic" Design I Learned in the Field as an FDE

Introduction — what I learned as an FDE

For the past year or so, I have worked as an FDE (Forward Deployed Engineer), embedded inside several companies, building AI agents for HR and finance operations — from design to production. Instead of gathering requirements in a meeting room, I sit next to the people who do the work and watch how the work actually flows.

Recently I read a piece about bringing AI into enterprise finance (a post by the founder of Varick Agents). Based on my field experience, I agree with most of its core argument. But there is one point where I clearly disagree. In this article, I use two AI agents I actually built and operate to describe what works and what does not, from first-hand experience.

My three conclusions:

1. Inside a working enterprise AI agent, the right mix is about 20% AI and 80% deterministic code. Decisions that must never be wrong are handled deterministically; AI is limited to the parts only AI can do.
2. Rolling out a general-purpose AI assistant company-wide does not automate operations by itself. Mapping each company’s specific workflows into the system still takes real engineering effort.
3. However, I disagree with “everything runs fully in the background.” To earn the cooperation of the people on the ground, you need a UI that shows what the AI did, and a design that keeps room for human judgment — a system built for coexistence.

Enterprise AI fails at a high rate

The statistics quoted in that article match what I see in the field.

7%

of finance leaders report strong results
while 84% have adopted or plan to adopt AI

95%

of enterprise generative AI pilots
show no measurable P&L impact

40%+

of agentic AI projects predicted
to be canceled by the end of 2027

Why is the gap between adoption and results so large? In my view, it is not a technology problem. It is a design-philosophy problem. Let me explain with real examples.

The limits of general-purpose assistants — trying Claude Cowork

The first failure pattern is the expectation that distributing a general-purpose AI assistant — Microsoft Copilot, Claude Cowork, and the like — across the company will automate the work.

I tried Claude Cowork myself. My first impression: the interface for tasks common to any job (file handling, document creation, research) is very well made. But to use it for actual operations, someone has to write down each company’s specific workflows, internal rules, and exception handling as prompts and skill files, and hand them to the assistant. That takes serious effort. It is work that requires someone who understands both the business and the AI — not something a busy staff member can do on the side.

What general-purpose assistants do well	What they cannot do out of the box
Tasks common to any job: file handling, document creation, research	Reflecting each company’s specific workflows, internal rules, and exceptions
Making one person faster	Automating the business process itself
Interactive trial and error	Audit-ready traceability of reasoning and operation history

This is my first-hand impression, but it is consistent with public information. Anthropic itself positions the organizational rollout of Cowork not as a one-time installation but as an ongoing program that includes skill building, pilots, and staged expansion. So my conclusion is this: a general-purpose assistant is excellent as a tool that makes one person faster, but on its own it does not function as a system that automates a business process.

Also, in finance work where audits are mandatory, you must be able to trace the AI’s reasoning and operation history afterward. General-purpose assistants are still maturing on this point.

Example 1: HR — an attendance-check agent

In one company’s HR department, staff manually checked the attendance records of about 300 employees at the beginning of every month. Missing clock-ins, inconsistencies between shift and working hours, approval status — there were close to 30 types of checks, done by eye over several days.

The design of the agent I built there is clear.

• The core of the judgment is 100% deterministic code. All ~30 check rules are implemented as a Python rule engine, so the same input always produces the same result. Every rule is grounded in work regulations or labor law, and the system can explain why each item was flagged, with the reason, for every single case.
• The LLM is limited to support roles. It only handles parts that need language interpretation or summarization: writing the overall summary report, generating advice for staff, and absorbing variations in shift-name notation.

Here is the interesting part: in the early prototype, I had the LLM judge the attendance data directly. The measured results below are why I replaced it with deterministic code.

Judgment method	Time per person	Accuracy	Characteristics
Rule engine (deterministic)	~2 seconds	100% against the spec	Same input, same result, every time. Can show the reason for every flagged item
Direct LLM judgment (early prototype)	~18 seconds	~75%	Could not reliably follow fine-grained conditions like role-based exemptions or holiday handling

The decision not to use AI was itself made by actually running the AI and measuring it.

We estimated the efficiency gain in work-hours before deployment. This checking work involved several people doing double checks, adding up to more than 100 hours per month across the department. The agent automatically screens out the roughly 90% of records that need no correction, so staff focus only on the items that need fixing. The estimate is a reduction of about 50% in working time.

Before

100%

After

50%

~50% reduction in working time (worth millions of yen per year in labor cost)

Example 2: Finance — a payment-voucher matching agent

In one organization’s finance department, before approving payment vouchers registered in the accounting system, staff manually cross-checked them against attached invoices and internal request records.

Here, AI was truly necessary in exactly one place: reading amounts and vendor names from invoices in inconsistent formats — including ones captured as photos. This used to be possible only by human eyes, and vision-capable LLMs made it feasible for the first time.

Everything else is deterministic.

• Arithmetic checks (subtotal + tax = total), withholding-tax calculation, master-data matching, normalization and comparison of dates and bank accounts — all Python code.
• Even the escalation decision — whether a hard-to-read document should be sent to a stronger model — is a deterministic rule, not an AI judgment.
• Only when a mismatch remains after deterministic matching does the LLM get involved, answering questions like “are these two names the same counterparty?” And the LLM can only answer “same (OK)” or “uncertain (send to a human)”. The final NG decision is never made by AI.

When you draw the flow, you can see that AI is involved in exactly two places.

flowchart TD
    A["Payment voucher + attachments"] --> B["AI: vision reading<br/>extract amounts and vendors from free-format invoices"]
    B --> C["Deterministic: Python code<br/>arithmetic checks, withholding tax, master-data matching, normalization"]
    C -->|"everything matches"| D["Organized as OK"]
    C -->|"mismatch remains"| E["AI: identity check<br/>are these two names the same counterparty?"]
    E -->|"judged same"| D
    E -->|"uncertain"| F["Escalate to a human<br/>final review with the evidence laid out"]
    style B fill:#e0e7ff,stroke:#4f46e5
    style E fill:#e0e7ff,stroke:#4f46e5
    style C fill:#d1fae5,stroke:#059669
    style F fill:#fef3c7,stroke:#b45309

We also estimated the scale of the efficiency gain. This voucher check occurs at a scale of several hundred items per month, taking dozens of hours of work across the department. With the agent automatically screening out vouchers that need no correction, the estimate is again a reduction of about 50%.

Before

100%

After

50%

~50% reduction in working time (several hundred vouchers auto-screened every month)

Let me share the measured costs too. The AI usage fee is about 1 yen per invoice. Monthly AI cost is in the range of a few hundred yen, and more than 95% of the total system cost was the fixed server cost. The image that “AI agents burn expensive tokens” simply does not apply if you design them deterministic-first.

~1 yen

AI usage fee per invoice
(less than one U.S. cent)

A few hundred yen

AI cost per month

95%+

share of total system cost taken
by fixed server cost

The ratio of time to code is also interesting. More than 90% of the execution time is spent on AI reading images, while the overwhelming majority of the decision logic is deterministic code. AI dominates the time; deterministic code dominates the decisions. That is the reality of enterprise AI agents.

Breakdown of execution time

AI (image reading) 90%+

Breakdown of decision logic (code volume)

Deterministic (Python) 90%+

Design principle: deterministic → LLM → human, in three stages

Generalizing the principle shared by both projects:

flowchart TD
    A["Business data"] --> B["Stage 1: solve deterministically<br/>calculation, matching, routing, rule-based judgment"]
    B -->|"can be settled"| O1["Judgment finalized automatically<br/>fast, cheap, auditable"]
    B -->|"needs interpretation"| C["Stage 2: pass to the LLM<br/>free-format reading, notation variants, summarization"]
    C -->|"confident"| O2["Results organized and presented"]
    C -->|"uncertain"| D["Stage 3: pass to a human<br/>escalate with the evidence laid out"]
    style B fill:#d1fae5,stroke:#059669
    style C fill:#e0e7ff,stroke:#4f46e5
    style D fill:#fef3c7,stroke:#b45309

1. Solve it deterministically first. Calculation, matching, routing, rule-based judgment — most of the work falls here. It is fast, cheap, auditable, and the same input always gives the same result.
2. Pass only the “interpretation” that deterministic code cannot handle to the LLM. Reading documents with no fixed format, identifying notation variants, summarizing results. Only an LLM can do these.
3. What is still uncertain goes to a human. Do not force the AI to decide. Escalate to a person, with the evidence laid out.

As a rough ratio, it feels like 20% AI, 80% deterministic. And if you look only at the core judgments where mistakes are not allowed, it is 100% deterministic.

Deterministic code 80%

AI 20%

The article I mentioned says a good AI agent is “mostly not AI” — 85% code, 15% model calls. The ratio I have implemented in Japanese workplaces is about the same.

Where I disagree: do not go fully background

Here is where my opinion differs from the article. It describes the ideal as an agent that finishes work in the background without anyone prompting it. Technically, yes, you can build that. If you look only at efficiency, that is probably optimal.

But what I felt keenly as an FDE is this: the people on the ground fear that their jobs will be entirely replaced by AI. The moment you try to build a fully automated, fully background “do-everything system,” you lose their cooperation. And without their cooperation, you cannot capture the real workflow — the exception handling and unwritten practices that never appear in the SOP. The success of an enterprise AI agent depends on capturing exactly that field knowledge.

So in both projects, I deliberately designed the system for coexistence.

Coexistence mechanism	Concrete design
A UI that shows what the AI did	Results are listed for review; for each item you can see which rule fired and why, and what the AI read from which document, side by side
Humans keep the trigger and the final decision	Staff choose the targets and start the check themselves; final confirmation and approval is always done by a person. The agent records “confirmed” status but does not hold the approval button
Human corrections grow the system	What staff confirm and fix feeds back into the rules, so accuracy improves through operation

After this design change, the reaction in the field shifted from “our jobs are being taken” to “the tedious part is already done when we arrive.” Designing this psychological acceptance is harder than the technical implementation — and more important.

What I learned from working as an FDE

Finally, about the way of working itself. The article says documented SOPs barely reflect reality, and that was exactly true in Japanese workplaces too. A process that looks like a straight line in the manual is actually a bundle of exceptions: “in this case, the person in charge emails them directly,” “if the amount is small, it goes through a different route.”

There is only one way to learn this: sit next to the person doing the work and watch the actual operations. Thirty minutes of observation gives you more information than any requirements document. Many of the ~30 attendance-check rules I built were added or revised after release, based on feedback from the field. An enterprise AI agent is not something you build once and finish. It is something you grow in dialogue with the people who use it.

Summary

• Inside an enterprise AI agent, the right mix is 20% AI, 80% deterministic. Decisions that must never be wrong are deterministic; AI is limited to the interpretation only AI can do.
• Distributing a general-purpose assistant does not automate operations. The real work is the engineering that maps each company’s specific workflows into the system.
• Do not aim for fully background automation. A coexistence design — where the AI’s actions are visible and human judgment remains — earns the field’s cooperation and knowledge, and that is what produces accuracy and adoption.

“AI agent” is a glamorous phrase, but the substance is steady engineering built on an accurate understanding of how the work is actually done. And as far as I know, that is the only way to produce results in a field where 95% of projects are said to fail.