Do you trust AI?
Konnichiwa, welcome to the AI Automation Dojo. Today we are asking the question, do you trust AI? No, really, do you trust it enough to let it answer emails from your clients without checking them first? If you said yes, you are brave and possibly a bit crazy. I am your host, Andrzej Kinastowski, one of the founders of Office Samurai, and today we are going to teach you how to automate your ticketing systems without letting the AI burn down the server room.
So, whether you are a service desk worker, tired of resetting passwords for people who cannot read, or a manager terrified that ChatGPT is going to promise your clients a product you do not sell, you are in the right place. Now, grab your favorite katana or that green approval button, and let’s get to it. I want to start today’s show with a topic that is near and dear to my heart and critical to the global economy, lawnmowers.
The story of the imaginary lawnmower
Earlier this year, my lawnmower broke. It was already more than 10 years old, so there was not much point in trying to fix it. And my grass was already tall enough to hide a small family of badgers. Now, buying a lawnmower involves research. It involves reading reviews. It involves effort. And I did not have time for effort, and my dog needed a walk. But I thought, hey, I live in the future. I have the sum of human knowledge in my pocket.
So while walking my dog, I fired up Google Gemini voice mode. I felt very Tony Stark. I gave it the prompt, listen, I need a battery-powered electric mower. Here is the size of my lawn. Here is my budget. And I want you to buy it from this specific shop because they are a client of ours and I, like our AI overlords, believe in the circular economy. And Gemini says in that soothing, confident voice, searching, found some matches, but if you expand your budget slightly, I can find you better ones. I said, OK, computer, upsell me, expand the budget. It comes back with three models from three different brands.
It lists the specifications. It compares the battery life. I pick two that sound perfect. I tell it to go find reviews for those two and summarize them. And it does. It tells me model A is better at mulching and model B charges faster. It was beautiful. It took like 20 minutes, zero friction, apart from my dog being a bit confused about who I am speaking to. I came back from walking the dog. The problem is solved. I came home, sat at my computer to click buy. And that is when I discovered that none of those lawnmowers exist.
They were not real. Their names sounded real. They followed the manufacturer’s naming convention perfectly. The specs were plausible. The reviews were convincing, but they were total fiction, a hallucination. And not only that, the specific shop I told it to check, they do not even carry one of those brands.
The reality of the agentic future
I felt betrayed. I am a tech geek. I love LLMs. I evangelize this stuff. But every once in a while, the AI reminds you that it is not a genius assistant. It is a very confident improvisational actor who did not read the script. Because if you listen to the evangelists and the consultants and the guys on LinkedIn who list their job title as an AI visionary, they will tell you that the agentic future is already here. They will paint this picture of an AI command center where you have a team of AI agents and you have an agentic manager managing the agents and an agentic auditor auditing the manager.
And they are all communicating flawlessly, executing complex processes, and you are just sitting there sipping an espresso watching the productivity line go up. And look, I believe this will happen. In 10 years? Probably. 5 years? Maybe. But today? With your messy processes, with your legacy systems that run on code written when the Spice Girls were topping the charts? No. It works in a lab. It works in a demo. But in the real world, the AI will invent a lawnmower that does not exist and try to buy it from a store that does not sell it.
Introducing human in the loop
I am not saying do not use it. I am saying you have to understand which things those technologies are good at and where they are still lacking. You have to know the difference between the science fiction movie and the documentary. So I want to tell you about something that does work today. In one of our past episodes, I introduced you to Cyber Ola, our digital alter ego for the head of our admin department, a Teams chat assistant that helps your employees.
And today I want to show you how to take that concept and plug it into the things that refuse to die. Ticketing systems and shared mailboxes. You know these places. The IT service desk. The black hole where emails go to die. Users asking the same three questions until the heat death of the universe: how do I reset my password, can I have access to SAP, why is the printer on fire? But we do not just unleash the AI and hope for the best. This is how you get imaginary lawnmowers. We use a concept called human in the loop.
We need to have absolute control over what the tool does. We simply cannot let it roam free, making mistakes or hallucinating answers that sound plausible but are completely wrong. We need a safety net so that we can service requests both from inside and outside of our organization.
How the master agent works
We have connected our framework to Jira, but it works with any ticketing system or a mailbox. Let’s see if the user creates a ticket. It first goes to what we call the master agent. Think of the master agent as the triage nurse. It looks at the incoming ticket and decides is this a tell me how question or is this a do this for me question. If it is a question like this one, do we have any rules as to how passwords should be constructed? When the user creates this ticket, we will use RAG, so Retrieval Augmented Generation.
It finds the answer in your actual company policy, not on Reddit. But, and this is the I tried to buy a fake lawnmower lesson, we do not let it send the answer automatically. We do not trust it yet, especially when we answer questions from external users. It sends the answer to a validation station. A human operator sees the user’s original ticket, the AI’s proposed answer, along with names of documents it took the data from, and the AI’s reasoning. The human operator can edit the answer if they want, and when they click approve, the answer is sent to the requester, and the ticket closes. The human does not do the work. They are a guardrail for the AI, a final judge, if you will.
GenAI magic and translation
This is cool by itself, but let me top that. The human operator doesn’t have to understand the language the requester speaks. We can set the tool up in a way that all tickets incoming into a particular queue are translated to a default language, which in most cases will be English. Now, if the requester sends the ticket in, say, French, both its contents and the tool’s proposed answer will be automatically translated to English and shown in the validation station in English. Once the operator approves the answer, possibly editing it in the process, it is translated back to the original request language before it’s sent to the requester. So, from the requester’s perspective, they’re both speaking French, and from the operator’s, in English. Gen AI magic at its finest.
Automating VPN and SAP requests
Now, what if the user says, can you reset my VPN password? They create a ticket, and now the master agent switches to the action mode. It checks, do I have a sub-agent for VPN password reset? Yes. Do I have the parameters? We only need user ID, and I can get it from the ticketing system. And then it goes back to the validation station. The human operator sees the agent for the resetting VPN password is chosen from the list of available agents. They check whether the parameters are in order and whether what the tool has decided is in line with what the requester asked for, and then they click approve.
When this happens, a set of API calls is being made to reset the password, and I get a new password on my mobile phone. And it is sent to the number set in the Active Directory for my account. Done. Another thing we have implemented, which is a very common ask, is give me access to SAP. When the user creates a ticket like this, the master agent again switches to action mode. It checks, do I have a sub-agent for SAP access? Yes.
It extracts the parameters, user ID, sub-instance, role from the ticket. The human checks everything and clicks approve. Now here is the cool part. The automation follows your process. It goes to your workflow system, and here we are using Microsoft Flows, and asks the user’s boss for approval. It finds the person superior in the Active Directory. When I approve, it fires up an RPA robot, the clicky-clicky kind. In our case, it is UiPath, that logs into the virtual machine, opens SAP, adds the role, and then emails the user, telling them they now have the access. Done. Ticket closed. The human operator did not have to log in. They did not have to navigate SAP menus. They just had to say, yes, that looks like a real request, not a hallucination.
Business use cases in Finance and HR
This thing here is not sci-fi. This is happening now, and you can apply this logic everywhere. In finance, an assistant that knows every booking rule and procedure. It can handle that endless stream of, where is my money, did you get my invoice emails from vendors. This is actually one of the most common use cases that we are currently implementing. An agent that answers emails, hey, did you get my invoice, and when is it going to be paid.
Or think of HR. Think about onboarding. It is a nightmare of checklist items. Create email, add to teams, order the laptop that is not terrible. Instead of your HR specialist clicking buttons for three hours, the agent preps the whole package. The human sees, new hire, John Doe, actions, create account, ship laptop, assign buddy. You click approve, and the new hire actually gets their computer and accesses on day one. A corporate miracle.
The end of meatware
The possibilities are endless. Wherever you have tickets or emails in huge numbers, internal or external, an agent like this can help unload a lot of them. Building and implementing this takes time and effort. But you should do it. Instead of dreaming of electric sheep, we need to adopt the things that Gen AI can already do for us. Because we need to stop doing the stupid things. We humans need to stop being copy-paste robots. We cannot keep on being meatware.
We need to get our time back for the important stuff. Like finally mowing our lawns. And that is a wrap. Ticket closed, resolved, archived. We have learned that the key to happiness is not fully autonomous AI. It is an AI that does all the heavy lifting and lets you take the credit just for clicking approve.
Domo arigato for listening. Big thanks to the team at Office Samurai for building this agent, so I have something to talk about. And also to Anna Cubal, our producer, and the real validation station of this podcast, keeping us from hallucinating our own facts. And to Wodzu Beats Studio, our development environment. If this episode helped you close a few mental tickets, hit subscribe, leave a five-star review, and tell your IT department we come in peace. Until next time, may your agents be smart and your humans vigilant.
