A thirty-second result can be impressive and still make the job worse. I use five questions to find that out before an experiment becomes a project.
The list of possible AI projects is endless. Summarize meetings. Sort customer messages. Draft reports. Check documents. Most of them can produce a convincing example.
But a working example does not tell you who prepares the input, how long checking takes, or what happens to the odd cases. You need to count those parts too.
First, write down how the job works today
How long does it take now? Who is involved? Where does it usually slow down? Which mistakes keep coming back? If you do not know the starting point, almost any new tool can look like progress.
This does not require a six-month study. Follow five or ten normal cases. Note the time, the handoffs, and the corrections people already make. That gives you a useful comparison.
Use one report, one invoice, one support ticket, or one meeting. Start the clock when the material arrives. Stop when the next person can actually use the result.
1. Did it save time?
Count from beginning to end. Collecting and cleaning the input takes time. So do instructions, fact-checking, rewriting, copying the answer into another system, and dealing with failures.
A summary may appear in thirty seconds. If checking names and moving action items takes another twelve minutes, use the twelve minutes in your calculation. The tool may still be faster. Just use the real number.
It also helps to separate waiting time from attention. Ten unattended minutes can be easier to live with than three minutes of watching and correcting every step.
2. Did it reduce the cost?
The subscription is only the obvious cost. Add setup, training, integrations, review, and maintenance. Then ask what a mistake costs.
During a trial, people often keep doing the old process as a backup. That may be sensible, but it means you are paying for both. Include it.
For recurring work, compare a month or a quarter. For a one-off task, count the complete setup. And remember that somebody will have to understand the workflow after the person who built it is gone.
3. Is the process easier?
Draw the old steps and the new ones. Look for extra logins, new queues, manual copying, approval loops, and accounts only one person can access.
One step can become faster while the whole process becomes harder to follow. That is common when the AI output has nowhere obvious to go.
Then break it on purpose. Can the person doing the job see what went wrong and continue without calling the builder? Saving two clicks is not much of a win if every unusual case needs a specialist.
4. Is the work easier on the person doing it?
Automation can remove dull typing and replace it with constant checking. Ask the person doing the job what changed, not only how many minutes disappeared.
Pay attention to interruptions. A stream of low-quality alerts can make a day worse even when each individual action is quick. Reviewing a batch at a set time may work better.
This question is easy to miss because employee effort does not always appear in a dashboard. Talk to the people using the process.
5. Is the result better?
Choose a definition of “better” before the trial. Maybe you need fewer missing fields, faster replies, consistent terms, or a clearer handoff. Pick something you can check.
Good formatting is not proof. Compare a sample with the original material. Look at dates, names, numbers, and the exceptions that could change a decision.
The result may improve in one way and get worse in another. Standard wording can make reports easier to scan but remove useful detail. Write that down instead of forcing the test into a simple pass or fail.
Keep a small scorecard
For each question, record what happened before, what happened with AI, what evidence you have, and which new problem appeared. The notes matter more than a grand total.
Before: 18 minutes per handoff. After: 11 minutes including review. Evidence: ten normal cases. New problem: names from handwritten notes still need manual checking.
The idea does not have to improve all five areas. Better quality may justify a modest increase in cost. A small time saving can matter if it removes a daily bottleneck. Be clear about the benefit you are buying and the downside you are accepting.
Run a pilot that is allowed to fail
Use normal examples, difficult examples, and at least one case you expect to cause trouble. Decide beforehand what would send the workflow back for changes.
Then give it to somebody who did not build it. Builders remember all the instructions they never wrote down. A new user does not.
At the end, make an ordinary decision: use it, change it, restrict it to certain cases, or stop. “Promising” usually means the decision has been postponed.
Next guideBuild the System Around the AI Tool →