Introduction and Expert presentation
Andrzej Kinastowski (AK): Konnichiwa. Welcome to the AI Automation Dojo, the show for people who believe that artificial intelligence is still on most days a very generous description. Today we’re once again putting on our hard hats and exploring the digital construction site that is UiPath Agents and UiPath Maestro. Are they the tools promising to build you a perfectly synchronized automated enterprise, or is this the skyscraper of efficiency or more of a digital shanty town? I’m bringing in an expert architect, senior developer Andrzej Ekiert, to help us read the blueprints.
On today’s agenda: UiPath Agents (are they the tireless digital employees we were promised, or are they more like gremlins that you definitely can feed after midnight?). UiPath Maestro (is it really the master conductor, or is it just a glorified traffic cop standing at the intersection of “it should work” and “it doesn’t”?). And the philosophical conundrum: when an unattended robot completes a task in a digital forest and no one is around to see it, did it actually save the company money?
I’m your host Andrzej Kinastowski, one of the founders of Office Samurai, the company that believes thinking outside the box is a good start, but maybe we should check what’s inside the box first. So whether you’ve ever felt like you’re hurting digital cats, you suspect that Maestro is just three hamsters on the wheel, or just simply trying to understand the future without falling asleep, you’re in the right place. Today we are joined by Andrzej Ekiert, one of our senior developers, a guy who has a lot of experience in a lot of different technologies and projects. Andrzej, welcome to the podcast.
Andrzej Ekiert (AE): Hello, thank you for having me.
AK: Tell me, why are you here? Why are you the one that I am talking to about UiPath Agents and UiPath Maestro?
AE: I’m the lucky guy. I was chosen to do one simple flow using the AI Agents and Maestro from UiPath. I was really excited to do this because in my day-to-day work, I’m not using UiPath as much, but I’m highly interested in AI topics. When I saw the opportunity to do something new with UiPath and AI, I was happy to start and just explore the new things from UiPath.
AK: You do have some more extensive knowledge working with AI, right?
AE: Yes. In my RPA career, I think the first thing was Document Understanding, and then I started thinking what else we can use to automate processes. RPA is amazing, we can do a lot of stuff with it, but unfortunately not all. Then we have to help ourselves with Document Understanding or AI. So I started to explore this AI world. I’ve done some courses, I finished post-graduated studies in this field. What I really wanted to do is to involve somehow AI in processes because using day-to-day AI is fun, but pure potential is when we actually see how we can use it in the processes, how we can save some money using it.
AK: It is quite important because a lot of people go deep into the AI field, exploring a lot of different things AI can do, but then have a lot of trouble translating those AI skills into something that is actionable. In our case, we’re working in business processes field, so for us the question is how can we use all of the things that AI can do to actually improve or automate business processes?
AE: The topic of what we’re talking about today, the UiPath Agents and the solution, this is exactly the answer to that. It’s one thing to know how to build something AI, it’s completely another thing to know how to apply it to a business process. Starting from looking for the processes where we can use AI Agents, I would say it’s difficult at the beginning because the hardest is to find the first one. After that, when you notice that there is actually something that we can use these AI Agents, then your mind is open and you think: “Oh my god, we can do that one, that one, and that one, let’s use it, let’s just try it, let’s play with it and maybe we can do more stuff with it”. But this first step is very difficult.
Before the Maestro and AI Agents with UiPath, for me (I’m not like the typical programming) it was difficult to combine UiPath with my LLM scripts from Python and other stuff. Of course, it’s not like super hard, you can connect it, but still it was a bigger effort. And then UiPath came and showed us AI Agents and Maestro, and we saw that now it’s really easy to combine RPA and AI Agents, and this is amazing.
First impressions: ease of use and platform integration
AK: It is one of the upsides of having an integrated platform, right? All of the pieces fit together, and it’s way easier to use them, unlike a strategy that a lot of companies take where they buy a lot of different bits and pieces and then they put all of the effort into actually combining those things and making sure they do work together. When you started working with UiPath’s Agents and UiPath Maestro, you already had experience with AI and with RPA platforms. We got early access, right? So it didn’t look the same way as it does today, even though only a few weeks have passed. What were your first impressions working with those tools?
AE: To be honest, the first thing was: “Oh my god, it’s easy, it’s super easy,” because the interface of creating AI Agents is very simple. It’s like all the stuff which I learned on the courses (how to create some kind of agents using Python), and then UiPath comes and I just copy-paste. It’s so easy. Everyone can do this. My last project with UiPath I’ve done like a year ago, even more, and I didn’t spend any time with UiPath during this time. This project appeared, and I was eager to do it, but I was afraid that lack of my knowledge with the latest UiPath updates might be problematic here. But it wasn’t, and this is amazing, because it was super easy to start.
There were some minor bugs at the beginning, but we were lucky because each bug we could report, they were fixing it, helping us in case that something didn’t work. We had help from UiPath. For me, that was a nice surprise that it’s super easy to start. First what I’ve done, just for me, is I took the agents which I created for myself and just copy-paste prompts to the UiPath. It worked almost exactly the same. I spent a lot of time creating my own agent, but with UiPath (of course I had the prompt and many stuff already prepared) I could just copy-paste to the UiPath AI Agent, and it just worked. It was amazing how easy it could be.
AK: It kind of reminds me of my first impression with RPA many, many years ago, because I used to write scripts like VBAs and in some other languages to automate things in the office. Then RPA comes, and I can do the same things way easier with way less work and in a more scalable way that is easier to control. It kind of feels like it’s the same story again. It doesn’t mean that there’s no point in learning how to do it in Python and how to do it on the low-level, but clearly not everyone has to go that deep, right?
AE: Exactly. Many of my friends in the RPA field do not have any IT programmatic background. They just learn the tool, they just learn UiPath and start doing. Of course, with UiPath you have to know the basics, but still you are not like the developer and you can still use it, and this is amazing. I wasn’t scared. Here was just pure joy, honestly, because I wasn’t expecting that, I thought it’s going to be difficult. I spent a lot of time doing courses to find out the power of AI Agents, and then UiPath came, and you just need to copy-paste few prompts and it’s working. Those were the good things about it, right?
Pain points and best practices for Maestro
AK: But what were the pain points? You mentioned that some of the things didn’t work. I imagine, and I hear from the developers that worked on this project, that not everything is as smooth as perfect as we would like. What were the things that were a pain?
AE: Always there are some things that you would change. Maybe I will start with the Maestro, because over there I noticed more things that annoyed me a little bit. First of all, we couldn’t work together in the same project. For us, it was a big deal, because we don’t have too much time to finish the project. You have other responsibilities, you sometimes just have like a two-hour window to work on it, and your colleague is working there. You can copy-paste the solution or do something else, but then you have to move it back.
AK: For those who haven’t tried it yet, Maestro is like the top level of where you orchestrate the process. In Maestro you connect different things, like agents, API calls, robots, and Action Center (all of those things). You can work on those small things, but the Maestro view, the BPMN view, only one person can be editing exactly at the same time.
AE: With Maestro now we would plan our work differently, to not interrupt each other. But, you know, it’s a new thing. You’ve done a piece of a process, and you’re eager to test it, to just run it to see what’s working and what’s not. Testing just the agents and the small parts of the flow, yes, you can do this, but the joy is when you can click from start to end and see the results. It’s very pleasant to see that it’s working finally.
AK: Especially that with Maestro you can see it very visually where it goes, it turns green and so on. But it is probably also about us finding the best practices of working with the new tool. The same way that when we were years and years ago learning how to build bots, we’ve been learning how to split them into components, how to test each component separately and so on. So I presume in time this will clear up a little bit.
AE: Also, I’m wondering because with the RPA we start with some best practices, they evolve. UiPath provides some best practices, but we as a company change those process. You can see that you have two variables with the same name, but the ID is different. That was tough and wasn’t easy to find what we actually want.
AK: We’re talking on the Maestro level, right? The variables where if you connect something like an agent it gets some inputs, it gets some outputs, you need variables for this, and all of those variables are seen on the main level of Maestro. If the project is big, I imagine.
AE: Yes, there’s a lot, a lot. Another thing as I remember when we did some change in the agents, like we wanted to add some variable or remove it, those also, as I remember, are deleted or just not working in the Maestro. So you have to provide those once again. If you have three input variables and then you change a name or something. As I remember with UiPath those are adjusted immediately in the Studio. Here not. We have to remember that it is just the beginning. The official release was like six or seven weeks ago, so it is quite fresh.
Testing Agents: non-determinism and LLM as a judge
AK: Exactly. It’s like a small trial for now, and we will see how it grows, but I strongly believe that they will do the great work here as usual. When it comes to testing, you mentioned testing. It’s one thing to test the whole Maestro flow, but when it comes to agents, what’s your experience with testing? Testing things based on Generative AI is a problem. You get non-deterministic results, and it’s even hard to guess whether the answer it gave you on a given case is right or not, because usually it’s not just comparing two strings, right? How does this work with UiPath Agents?
AE: First, simple testing is just putting your input after your first prompt, and you get output. It’s a very fast way to test the stuff during automation. But if you finish your agent and you want to do the proper testing, with UiPath we’ve got the evaluation set (I don’t remember the exact name) where you can put all the test data which you got, and with one button just run it, and you will get the results of each test.
Our case was related to reading CVs and getting information from CV about people who want to start work in our company as an RPA Developer. We just fed it with a bunch of CVs and just run it, and we just observed if the output is really what we wanted. For production processes and production agents, this set of the test data should be really big, and maybe it’s better to think about other LLM which will check our LLM, because doing it manually, we’re at that stage. We just used this evaluation feature, and it worked.
AK: There is also in the evaluations a possibility to use from what I saw LLM as a judge, so the system where it doesn’t just check one to one, it doesn’t just compare the strings, but it makes an informed decision. Testing LLM tools is a pain in general, so we’ll see where it goes. The integration, you mentioned that being within the platform makes it easy to integrate with other pieces of the platform. What’s your take on that?
AE: That was another nice surprise for me. Honestly, I’ve never used Data Service from UiPath before. It was my really first time when I opened Data Service. I never needed it for the project, but here we need it. It was so easy. Connection of AI Agents with other tools is super easy. It’s just like a drop-down list where you can choose what you want, and you’ve got it. It’s amazing.
For my agent, I needed to extract some HR data (in case that a person would ask about some stuff of our company, when we’re starting to work). We needed to feed the agent with some data, so I just copy-pasted our HR welcome PDFs or something like that, what we got, and just put there. Then just connect with a drop-down list where I choose Data Service, I choose the folder where it is, and that’s it. Same things with other tools. I didn’t test it yet, but you can start a robot from these agents, as I remember. It’s amazing because it allows us to just write a simple chatbot where you can just type, and everyone in the company could start talking and say: “Okay, I need that document or that report” and the AI Agent starts the flow which creates the needed report. It’s amazing because you will get real data gathered from some databases, and whenever you want, you get it.
Hyperautomation promise and future functionality
AK: With a very easy-to-use interface, right? This is something that we have been struggling with. Over time it’s gotten easier and easier. I remember running the automations on demand. We had a lot of versions of how we do it, even starting from executables that would call up UiPath’s API to run a job. Then we moved to things like connecting Teams, but you had to type a very specific command. Then UiPath Apps, and so on. But now it’s kind of you can interface with it like you would with a human. You can ask it: “Hey, can you run me a report?” And then it’s going to ask you which report do you want, what are the dates and so on. Once it gets all the information, it can run the automation to get it for you.
For years now, we have been talking about hyperautomation. There was this term coined that hyperautomation is when you have a platform where all of the pieces can easily fit together. It feels like UiPath is finally fulfilling that promise, right? The fact that you can just put a box in Maestro and then choose one of the published projects or robots in your Orchestrator, or you can choose Data Service and get data from the Data Service, or run an Integration Service (which for those who haven’t been using it, you can have prepared API calls to basically any system that is reachable). Once you have them prepared, you can just choose it, give it parameters, and it just runs. This is what hyperautomation was promising us (that it will be easy to call up different parts of your ecosystem and work with it), like you could call up a Document Understanding and give it a document, or Communication Mining, or a robot, or whatever else.
AE: That’s true. It really looks like hyperautomation now with Maestro. It’s a very easy way to connect. I don’t want to say a whole company in just one flow, but you could, if you want to.
AK: I don’t know why you would want to, but it’s doable. Are there any functionalities in the agents and in the Maestro that are still missing? Anything that you or our team are still waiting for to be implemented?
AE: For sure those things which I said that annoy me now (it would be nice to change it or adjust somehow), but honestly with the Maestro probably more than with the AI Agents. What I’m afraid is that if we add too much stuff to AI Agents, it’s not going to be so easy. It starts to not be accessible for everyone, like even the RPA Developer now is able to work with that.
They could provide more options for inputs to the agents. Now we have string objects and some few others, but it was missed that I couldn’t provide these agents with whatever I want. Here there is no way to do this.
AK: One thing that I’m personally waiting for is the possibility to connect your own LLM engine. This is something that is in the pipeline, we know it’s coming, but a lot of companies would rather have their own engine that they could call up.
AE: That’s true. Now we can connect to GPT-4 and one more (I think there’s Anthropic Claude in there, or GPT Mini, or something). If you already have a license to any other AI tool, it would be great if you could use it.
Low-hanging fruit: ideal automation processes
AK: When you think about what the AI Agents and Maestro can do, what processes do you think it will allow us to automate? What do you think will be the low-hanging fruits of the whole agents and Maestro thing?
AE: I think customer service, customer service. I would say that customer service would be the first one. Of course, in the bigger companies, HR. For small companies like ours, I think there is no point, but I heard that we are getting a lot of CVs for the last job opening (we got like 500). So it’s not easy to check all of them. We can help ourselves with the agents.
AK: It’s okay for the AI to help you structure the data and then filter it. It’s not okay for it to make a decision who you’re going to hire. We don’t want the AI to do the actual decision for us.
AE: Customer service I would say will be the first one. I believe that in the logistic I could imagine that you have an agent who just helps you plan your trip (what’s worth it, what’s not), and it could also be beneficial for companies. I know it’s a more difficult and harder task to do, but in the future, for sure, I can bet that companies will start applying it. The potential is very big. I remember when Document Understanding was something super new and amazing, and we thought we will not have anything better. Now Document Understanding is almost everywhere. It is business as usual.
I believe that we will have a lot of work in this area. Now, companies don’t have any agents, but they have a budget to just do something with it because they want to have it because other companies have it. I can imagine that it’s going to be almost everywhere, and if the company uses UiPath, sooner or later they will use AI Agents as well.
Barriers to adoption and low-level vs. high-level AI
AK: What do you think is stopping companies from using those technologies? A lot of companies are hesitant to do this. What do you think the reasons may be?
AE: Let’s say the obvious one: money. I would say it’s still expensive, and maybe it’s not profitable for every company. But Document Understanding five years ago had the same problem. I think the biggest one is the same what we have with the RPA: people are just scared of the capabilities. People are scared that they are going to be fired or something like that, and it’s not true.
AK: When you automate processes, you do not fire anyone, you just move them to something that makes more sense for a human.
AE: Exactly. Maybe this is the best possible motivator: jealousy. You see others doing it, getting the benefits, and then you say: “Okay, maybe we’re going to do that too”. I know it’s going to be like a small snowball effect.
I think now for us, during the business analyst phase, it’s now hard to find a very simple RPA project without anything else. I believe it will also be focused for AI Agents and Maestro for flows, because really it’s not hard to do and very helpful.
AK: If you have the budget for AI, the AI Agents and the UiPath Maestro may be the best way to use it at this very moment.
AE: Totally agree. Even if you don’t have a budget, it would be nice to find some and just try it, because it’s simply worth it to know what’s going on and to see the capabilities. Now, if you are not evolving, you are just going back.
AK: You are a part of the AI Devs 3 training, right? You’ve been building AI Assistants and agents low-level programmatically. What would you say are the pros and cons of doing it low-level, the programist way, or high-level with things like UiPath Agents and Maestro?
AE: Of course, with the low-level you are able to adjust everything, and use whatever engine you want. You’ve got your own LLM like Llama (which is free, I think) which allows you to do simple stuff, and when it’s not working, then you use something more expensive. It’s convenient and just cheaper. But it’s not for everyone. If you don’t have any background with any programming language, it may be difficult for you. It’s just much harder to start with the low-level.
With the AI Agents, it’s much easier. You don’t have to have any knowledge of programmatic basics. Of course, you have to know how the LLM works, you have to know how to write a prompt (it’s very important in both of those things). The first thing I would say is that it’s just easier to start with the AI Agents. The second thing, maybe you can do more. You can also use it to do stuff with graphics and even doing transcripts from some conversation and then extracting data from it. So the capabilities are bigger.
AK: You mean when you do it yourself programmatically, there’s a lot more things you can do.
AE: Yes. In the end, probably the best approach would be to be able to do both and then choose the right technology for the project. Especially that if you know how to do it low-level and programmatically, you can also use UiPath Agents or Maestro to call up a piece of code that will do something for you that isn’t available from the graphical interface, right?
AE: Exactly. So from that perspective, it’s best to be able to use the both worlds. I completely agree that the entry is way easier if you have a platform that you can use, because otherwise, if you want to write something from scratch, it is extremely interesting and it is a lot of fun to do, but it requires a lot and a lot of work. The same thing is with Communication Mining. Using UiPath and its interface is just much simpler.
AK: This makes it more scalable, because you can introduce way more projects in the same amount of time and get them to production. But in the end, it is kind of a choice where you either retain full control over your automation (but you pay with your time and the amount of skills you have to have) or you give up some control, but then you are able to execute way faster and in a more scalable way.
AE: Yes.
AK: Thank you so much for joining me on this episode. I’m looking forward to seeing what new things you and other people from our team will develop using those technologies.
AE: I hope so. Thank you very much for having me. It’s a new experience for me, I’m glad.
AK: Thank you, Andrzej. And with that, we closed the curtain on another performance of the digital orchestra. A heartfelt thanks to Andrzej Ekiert for taking the time to separate the operational reality from the marketing fantasy. The episode was produced and directed by the one person who keeps the whole operation from turning into a blue screen of death, Anna Cubal. Recorded at Wodzu Beats Studio where we believe in one single process: coffee in, podcast out. Until next time, remember to schedule some downtime.
