AI Workflow Automation With Human Review
Human-in-the-loop AI automation for back-office work: review patterns, where AI fits first, the risks, and why we switched our own AI estimation off.
AI workflow automation with human review means the AI does the first pass (reading a document, sorting a request, drafting a reply or a quote) and a person approves the result before it reaches a customer or changes a record. In back-office work this is the safe default: AI is fast at drafts and unreliable at being right every time. Below are the review patterns that work, where AI fits first, the risks, and how we built and then switched off our own AI estimation step.
Where does AI fit first in back-office work?
Look for work that is high-volume, text-heavy and easy to check. Three kinds of task meet all three conditions in most small companies:
| Task | What AI does | What the person checks | Why it suits AI |
|---|---|---|---|
| Document intake | Reads emails, PDFs, forms and photos; pulls out names, dates, amounts, items | That the extracted fields match the source | Tedious for people; quick to verify side by side |
| Classification | Sorts enquiries by type and urgency, routes them to the right person | Edge cases and anything marked low-confidence | Mistakes are cheap if a person sees the queue |
| Drafting | Writes a first version of a reply, quote, report or summary | Facts, numbers and tone before sending | Editing a draft is faster than starting from a blank page |
Work that does not suit AI on its own includes final pricing, anything that legally binds the company, payments, and decisions that significantly affect a person. AI can prepare these, but a person should make the decision.
What does "human in the loop" look like in practice?
"Human review" covers several different designs. Pick the one that matches the cost of a mistake.
| Pattern | How it works | Good for | Weak spot |
|---|---|---|---|
| Approve before send | Nothing leaves the system until a person presses send | Quotes, customer replies, contracts | Review becomes a bottleneck if volume grows |
| Review by exception | Items the system marks as uncertain go to a person; the rest pass | High-volume classification | Only as good as the rules that mark uncertainty |
| Cross-check | Two models (or two runs) do the same task; disagreement goes to a person | Numbers, extraction | Doubles AI cost; agreement is not proof of correctness |
| Sample audit | A person checks a fixed share of completed items every week | Mature, low-risk flows | Errors can run for days before the sample catches them |
| Suggest only | AI proposes, a person does the work | Early pilots, sensitive data | Smaller time savings |
Whatever the pattern, four things keep it honest:
- Keep the raw output. Store what the model actually returned, including failed runs, so you can see why a draft was wrong.
- Fail into the review queue. A timeout, an unparseable answer or an empty result should land in front of a person, not disappear.
- Constrain the output. Ask for structured output, validate it against a schema, and reject anything outside the allowed values.
- Make the switch obvious. An administrator should be able to turn the automation off without a deployment.
How did we build our own AI estimation step?
We built this into our own back office, and it is the clearest example we have.
When a project request arrives, the system can run an AI estimation job. The job builds one prompt from the request, our active price book and up to five past quotes of the same project type. It then sends that prompt to two model "lanes" in parallel, which can be a local model or a hosted API.
What happens to the answers:
- Each answer is parsed against a fixed schema. A failed parse or a timeout is retried once.
- Line items that do not match an entry in our price book are marked as out-of-book, so a model cannot slip in an invented market price unnoticed.
- The two totals are compared. If they differ by more than 15%, or if only one lane succeeded, the draft is flagged. If both lanes fail, an empty, flagged draft is created and the failure goes to the error log.
- Every outcome, successful or not, moves the case to a review stage. The raw output of both lanes is stored, including on failure.
- A person reviews the draft, edits it and sends the quote. The quote a customer receives is sent by a person.
Then, on 4 October 2026, we shipped a change that made automatic estimation off by default. An admin setting, "run AI estimation on intake", now decides whether the job starts by itself. Until it is switched on, new requests wait for a person, quotes are written by hand, and the AI run is a button an admin can press on the case page.
The lesson is not that the automation failed. It is that a well-guarded AI step is still a choice you should be able to reverse in one click, and that "off unless chosen" is the right default for anything touching prices. If you are planning a back-office system with steps like this, our custom ERP cost guide covers how such systems are scoped and priced, and English intranets for teams in Korea covers approvals for bilingual teams.
Should you use an automation tool or build a custom flow?
Tools such as Zapier, Make and n8n are a good first step. They connect common services quickly, and many teams never need more. A custom flow makes sense when the automation has to live inside your own system: your data model, your price book, your approval stages and your audit trail.
| Automation tool | Custom flow inside your system | |
|---|---|---|
| Time to first result | Hours to days | Weeks |
| Cost | Subscription, per-task pricing | Build cost plus AI API usage |
| Access to your data | Through the connectors on offer | Direct, with your own rules |
| Review steps | Usually an approval step or a manual check | Designed into your stages and permissions |
| Records | In the tool's logs | In your database, next to the case |
In our pricing catalogue (version 1.1, 23 September 2026), one automation flow inside a business system is KRW 0.8M–2.0M, typically KRW 1.2M, excluding VAT. AI API usage is passed through at cost and listed separately in the quote.
What are the risks of AI automation in business processes?
- Confident wrong answers. Models produce plausible numbers and names that are not in the source. Verify against the source document, not against whether the answer "looks right".
- Instructions hidden in inbound documents. An email or PDF can contain text that tries to steer the model. Treat document content as data, and do not let the model take actions on the strength of it.
- Personal data leaving your systems. Sending customer data to an external AI service may count as entrusting or transferring it, and Korea's Personal Information Protection Act sets conditions for transfers abroad (Article 28-8). Decide what may be sent before you build. Our Korean website localisation checklist covers privacy notices.
- Automated decisions about people. Under the same Act, when a fully automated decision significantly affects someone's rights or duties, they can object or ask for an explanation (Article 37-2, in force since March 2024). The Article covers decisions made by a completely automated system, so a person making the final call changes the picture. Design for objections anyway.
- New AI rules. Korea's AI Basic Act has applied since 22 January 2026. It sets extra duties for "high-impact" AI and for generative AI services offered to users. Check whether your use case falls within its scope.
- Silent failure and cost drift. Monitor failures and API spend the way you monitor uptime.
None of this is legal advice. If a flow touches personal data or decisions about people, have it reviewed.
Frequently asked questions
What is the best AI workflow automation for a small business?
The best one is the one that automates a task you can check quickly. Start with document intake, enquiry classification or drafting, use an off-the-shelf tool if it connects to your systems, and move to a custom flow when the automation needs your own data, rules and review stages.
How do I automate a workflow with AI?
Write down the current steps, mark which ones are reading, sorting or drafting, and automate one of those first. Constrain the output to a fixed structure, send uncertain or failed results to a person, keep the raw output, and add a switch that turns the automation off.
Can AI create business process workflows?
AI can draft a process map or suggest steps from a description, but the workflow still needs a person who knows the exceptions: cancellations, changes and approvals. In our experience the exceptions, not the main path, are where most of the design work goes.
Do AI automations run without human review?
They can, but for quotes, customer messages and anything that changes records we design them not to. Our own AI estimation step sends every result, including failures, to a review stage, and automatic runs have been off by default since October 2026.
Is it safe to send customer data to an AI API?
It depends on what you send, where the provider processes it and what your privacy notice says. Under Korea's Personal Information Protection Act, transfers abroad must meet one of the conditions in Article 28-8. Send the minimum, strip identifiers where you can, and check with a privacy professional.
Sources
Checked October 2026.
- Personal Information Protection Act, English translation, Act No. 20897 of 1 April 2025 (Korea Legislation Research Institute): Article 28-8 (cross-border transfer) and Article 37-2 (automated decisions)
- Personal Information Protection Act, Korean original (National Law Information Center)
- Framework Act on the Development of Artificial Intelligence and Establishment of Trust (National Law Information Center)
- NIST AI Risk Management Framework
- EU Artificial Intelligence Act, Regulation (EU) 2024/1689 (EUR-Lex): Article 14 (human oversight)
- NQ Solution's own estimation code and the 4 October 2026 change that made AI auto-estimation off by default; pricing catalogue v1.1 (23 September 2026)
Talk to us
If you have a back-office task that AI could draft and your team could check, describe it in the project request form. Our system and AI automation service and software development in Korea pages show what we build. Working with a Korean team for the first time? Read outsourcing software development to South Korea.