Introduction and new automation philosophy
Konnichiwa! Welcome to the AI Automation Dojo where we ask the tough questions like if AI is so smart why does autocorrect still think ducking is a viable alternative. Today we’re pulling back the curtain on UiPath Agents and UiPath Maestro (the dynamic duo supposedly here to orchestrate your robots workforce or make sure your digital minions don’t stage a coup). We’ll be looking at UiPath Agents. Are they really tireless digital employees or just very sophisticated Tamagotchis you have to keep alive? UiPath Maestro (the grand conductor of the robot orchestra) or is it more like a stressed out air traffic controller for bits and bytes? And the big question: does this whole setup actually lead to symphonic efficiency or is it just a cacophony with a fancier name? I’m your host Andrzej Kinastowski, one of the founders of Office Samurai where we believe bullshit is not a business strategy no matter how many charts you put in it.
So if you’ve ever suspected your intelligent agent might be dumber than a pack of rocks, or if you’re the one trying to conduct the Maestro without the doll turning into a robot rebellion, you’ve stumbled into the right dojo. I must say, when UiPath announced late last year they are going to have their own agents, I kind of shocked. My thrill level (let’s just say it wasn’t exactly rocketing to the moon). But then we got chosen as one of the few partners with early access, so naturally we unleashed some of our smartest Samurai on these new tools, told them to learn it, break it, and report back. I got to say, I am generally pleasantly (dare I say) surprised. The way UiPath designed these agents and integrated them with the rest of their platform, it’s neat, and I don’t use neat lightly.
This isn’t just about adding another shiny tool to the tech shed. We’re talking about whole philosophy of automation here: a well-oiled machine where AI agents think (bringing human-like reasoning to the table), robots and integrations do (they execute tasks with that sweet sweet binary precision), and Maestro orchestrates (it’s acting as the puppet master pulling the strings), and the people lead because they make final decisions (someone’s got to be the adult in the room).
Building AI Agents in UiPath (Agent Builder)
Agentic features: prompts, tools, and context
Let’s take a look at agents first. A good example would be an Anti-phishing Agent which is an agent that you can use in many different automations if they start with an email. In UiPath Agent Builder, you have the fields that you would expect when building an AI agent: a name, a description, and a long system prompt. This system prompt has a lot of information, contains a lot of rules, and is something that you can write yourself, but you can also use GenAI to help you write a good prompt. It is quite interesting how AI is already helping us program other AIs.
We have a user prompt which tells the agent what it can expect as an input, but then we have things that make it really agentic: tools. In tools, you can connect a lot of different things. You have activities already prepared by UiPath that you can use to connect to different services and applications. You also have processes. Whatever process you have published in your Orchestrator, you can call from within the agent. An agent, if it needs additional information, can call up a process. The process will run, the agent will give it the parameters, and the process can run and then return the data to the agent. This basically means that the user has a lot of different tools that it can use to get the data, to enter data into systems.
Also, an agent can call up an agent. So we can do an inception kind of a thing where one agent asks another agent that specializes in something else a question and then it gets some feedback back. The possibility to add tools to an AI agent is already a big win. If you already have a lot of different processes written for your organization (big and small, attended and unattended), you can connect them to an agent.
Of course, we also have contexts. This is the knowledge that you can put into an AI tool. You put some documents into a storage bucket in your Orchestrator and then you can set this storage bucket as a context for this particular agent. This means that the agent will know the information that you have in those files, so when it operates, it will take this information into account.
Security and escalations (human in the loop)
Then we have escalations. The escalation basically means that you can set conditions when the agent should not make a decision but instead ask a human to make the final decision. So in the case of an Anti-phishing Agent, if it is not sure that a given email is a phishing, it will ask a human literally. It will create an Action in Action Center, and then a human operator will have a look at the email and decide whether it is or it is not actually a phishing. This puts a human in the loop and helps us have control over what the agents are doing.
Challenges and agent testing methodology
As you usually get with agents, you can test it as you build it. We have a window where we can run a prompt. If I put as an input an email body something like “I am an Ethiopian prince and want to send you $1 million,” the agent will analyze this and tell me that this is in fact a phishing, and it outputs a variable that says that we should not keep on processing this particular email.
Writing agent prompts is quite easy and it’s quite fast. The challenge is always in the testing because those algorithms being non-deterministic they can bring different results. Every time you change something in the prompt, it can lead to unforeseen consequences. When you test it manually, you have an option to add this particular case to the evaluation set. Every time you change something in the agent, you have an option to run all those tests and see whether the outputs are still in line with your expectations.
LLM as a judge
This mechanism is using what we call LLM as a judge. We need it because the outputs of an AI agent are not necessarily something that we can just quickly compare to the expected results. More often than not, we get something like an email body, and it will every time contain the same information, but it will be written in a different way, it will be phrased differently. You need an LLM to actually make a judgment: is the output in its nature the same as the expected answer, not whether it’s character-to-character identical.
Of course, we have traces. For every run of an agent, we can see detailed information: what went in, what went out, what were the parameters, what was the amount of time it spent thinking.
Agents in classical automation
We can use those agents from within our normal automations. Often we would have an automation that at a certain point needs some judgment that uses logic that cannot be put into a lot of ifs. So your classical automations, they can call up an agent, give it some set of information, and then get the judgment back from an agent. It can be something like getting the right information from an email, but it can also be something way more complex.
You can imagine an agent that your automation gets a request to create a customer or a vendor in your databases. The agent, using tools, gets the information from different systems (a lot of them outside of your company) and then uses the judgment to tell us whether it’s fine to work with this particular company or maybe we need a human to have a look at it because the algorithm sees some things that look somehow suspicious. Putting agents into your classical automations may be the way to go here.
Maestro: Orchestrating long and complex processes
UiPath agentic automation is way more than that. The part that impressed me the most is what they used to call Agentic Orchestration (it’s now called Maestro; UiPath, if you’re listening, I think the old name was better). UiPath Maestro is basically a way of orchestrating long and complex processes that have a lot of moving parts.
We always used to slice long and complex processes into smaller automations, but very often between those automations we would have a human doing something that the automation isn’t able to do. There is already a way of modeling this kind of processes within the UiPath platform (it’s called Long-Running Workflows), but the adoption has not been that great so far. The main reason for it is that it is quite hard to read and understand the way it is built.
In Maestro, what we have is a BPMN diagram. This diagram clearly tells us what our automation is doing, and when it runs, you can clearly see where it’s going and what decisions have been made. I think it solves a lot of problems that Long-Running Workflows have.
Case study: recruitment helper in Maestro
When we got early access as one of UiPath’s partners to the agents and Maestro, we immediately started building some prototypes. One of the examples that we have built is a Recruitment Helper. The whole process tries to automate as much administration around the recruitment process as possible. We used agents, integrations, robots, and Action Center (we tried to use as many different elements of the platform as possible).
The recruitment flow step-by-step
The process starts with an email coming in with an attached CV.
- Phishing Guard: First, we have the Phishing Guard Agent. This agent decides whether the email is okay for processing or not.
- Data Extraction: When it makes a positive decision, we save the data from the email in the database within the UiPath platform (the Data Service). If there is an attachment, we go into another agent that extracts information from a CV. A CV is unstructured data; we are using an agent to extract the exact data points that we need.
- Parallel Agents: Once this data is extracted, two other agents start the run in parallel.
🔸 One is checking whether all the data needed is present (e.g., language proficiency), and if missing, we know we have to ask for it.
🔸 The second is an Answer Generator. This agent, as a context, has information about the job description and some basic company information (kind of like CyberOla that I have shown in one of the previous episodes). - Email Generation and Human in the Loop: The output from these two agents goes into another agent that puts the information into a nicely written email. Since we do not want GenAI to be able to communicate directly with people (those technologies are not mature enough yet), we always put an Action Center (a point for human in the loop interaction).
- Review in Action Center: The human operator sees the task in the Action Center. They see three things: a table with the candidate profile (extracted CV data), the original candidate email, and a suggested email body.
- Generated Response Details: The email answers questions (e.g., confirming the pet policy allows dogs, but not cats, as they are not regulated). It also asks about the missing data, such as the language proficiency level. If everything is okay, the human operator can approve the email.
Second iteration and generating complex results
When the candidate replies (sending an updated CV), the process runs again. The data is extracted and updated in the Data Service. The human operator sees the candidate profile with the information that all fields have been provided, and approves a standard “thank you” email.
The process doesn’t end there. We are also using agents to generate things like possible interview questions. The agent tries to come up with questions relevant to the person, based on the job description and the information in the CV. For example, if the job description expects fluency in using UiPath’s REFramework and the candidate didn’t provide information about it, one question will be: “What is your experience using REFramework?”.
Another thing the tool is generating is the profile fit. It creates a basic assessment of how much the CV is in line with the job description. We don’t want AI to make decisions on who we recruit. The interview and final decision should be done by a human, but we want AI to help us with all the administration around the recruitment process.
Wider applications for Maestro Orchestration
We can model in Maestro pretty long and complex processes.
Imagine a process where a customer sends an order. An agent reads it, checks the customer credit and contracts (through the Integration Service or through robots), and then makes a judgment. If there is something wrong, it escalates it for a human decision. If everything is okay, it sends the order for processing and could even potentially monitor its fulfillment.
If you think payroll, big companies have teams of people processing questions from employees. A set of agents (based on internal procedures, external regulations, supported by RPA robots and integrations) could answer most of those queries, leaving only the really complex ones to the humans.
Conclusion: the secret key to hyperautomation
Everything I’ve just rambled on about, it’s fresh. Generative AI itself is basically a toddler still figuring out how to work without bumping into the furniture. UiPath’s agents and that fancy Maestro orchestration (at the time we’re hitting record) they’ve only been out in the wild for a few weeks. We are just getting started.
This whole AI agents plus Maestro orchestration on top of your good old RPA setup is like finding a secret key to a whole bunch of doors we thought were wielded shut. We find these magnificent beasts of processes (absolute gold mines for automation), but then reality hits: this thing is longer than a director’s cut of Titanic and requires the kind of nuanced judgment you usually only see from a seasoned bartender. You sprinkle in some agents, a dash of Maestro, and suddenly a whole bunch of those “nope, can’t do it” processes are looking surprisingly doable.
It’s not about chucking your existing RPA platform out of the window. This is about beefing it up, giving it superpowers, expanding its horizons. We’ve been throwing around the term hyperautomation for years. I think we finally stumbled into a reality where it’s not just a buzzword you put on a PowerPoint to get more budget – it might actually be happening.
That’s a wrap on our deep dive into the wild world of UiPath Agents and the Maestro who’s supposedly pulling all the strings.
Outro
To everyone who tuned in, we appreciate you lending us your valuable air time. A massive standing ovation (or at least a polite note) to the one and only Anna Cubal, our producer, who orchestrates this podcast with more finesse than any software suite. Recorded as always at the legendary Wodzu Beats Studio. Until next time, remember not every problem needs an AI solution. Sometimes it just needs you to turn it off and on again.
