Introduction

AK: Konnichiwa, welcome to the AI Automation Dojo. My guest today is Mateusz Ślemp, a Senior Manager for AI and intelligent process automation at Amer Sports. If the name Amer Sports doesn’t immediately ring a bell, the brands underneath it absolutely will: Salomon, Arc’teryx, Wilson, Atomic, Peak Performance, and Suunto. So, the gear in your ski bag, on your tennis court, and quietly judging you from the back of your closet.

It’s a globally sprawling sports company, and somewhere inside it is a team in Kraków making sure the invoices get paid, the robots behave, and now the AI agents stay on script. So whether you’re an accounts payable employee sick of answering “hey, did you get my invoice” questions, or a security officer who breaks into a cold sweat every time someone says “but what if we just plugged the agents into SAP,” you’re in the right place.

Now grab your favorite katana or that squash racket you haven’t used since the pandemic hit, and let’s get to it.

Today our guest is Matt, who has been with Amer Sports since 2013. He started in accounts payable, moved into continuous improvement, and ended up running an automation team which has produced more than 200 automations so far. His team has also been working with AI agents for some time, and together we have just shipped the first global AI agent into production, a chatbot called My Invoice Assistant that we’ll get into. We’ve worked with Matt and his team for about seven years now, which in our industry is somewhere between a long-term partner and a common-law marriage. Matt, welcome to the dojo.

MS: Hello, welcome, and it’s a pleasure to be here. Thank you for the invitation.

Defining the role of AI and Intelligent Process Automation

AK: Tell me, your official title is AI and Intelligent Process Automation Senior Manager. That’s a very long and fancy title. If you could explain to us, what is it that you actually do?

MS: Indeed, it’s a long one, but actually it explains in a nutshell what I’m responsible for. On one hand, I’m responsible for the AI in finance. I’m managing a small AI specialist team focusing indeed on scaling up AI, delivering AI solutions around finance, and also promoting AI in finance.

On the other hand, I’m responsible also for the team of intelligent process automation which is like the old rebranded robotic process automation team which is at Amer Sports managed by Ewa, who’s quite brilliant in making sure that our bots are working properly and are delivered according to best compliance practices.

AK: So you have those two legs of kind of RPA and AI. Are those technologies collaborators, or do you see them eating at each other?

MS: Actually, that’s a good question. On one hand, it is very much collaborators. When I was starting with AI, I was really sure that it’s a proper way to add robots a bit more possibilities so they will complement each other. Both will be a bit more intelligent beasts which we call now agents, and even more like agentic AI that can use bots on the way.

But on the other hand, AI seems to be a much wider thing because we are really talking here about building a bit standalone things without the robot itself. These are extra possibilities that AI gives us. Looking in the future, when you think about that agents will be eating robots, I see maybe two extra things worth mentioning here.

On one hand, it’s a kind of naming convention that I guess business will be using. It was something similar with the robot some years ago, what is actually the definition of the robot in the business? Now there would be a question: what is actually the definition of the agent in a business? I have a feeling that at some point we will not have robots; it will be robots for us and for the audience that are listening here because those are the pros who understand this in detail. But I guess from the business perspective they would like to have an agent. More and more bots that will be kind of combined with AI here or there will be understood or named as an AI agent or something similar.

MS: On the other hand, when we’re talking about eating, I’m very much looking into the solution from cloud which is called Cloud Flows, which is already basically live. Microsoft is announcing that Copilot will also give this possibility to use Cloud Flows in the end, which means that your personal processes can be, by the end user, automated with this.

On one hand, for the business, it is still a risky area. We will need to really get mature and understand it a bit more. On the other hand, it is a big opportunity and we should closely look into this. That might be the part where AI could be actually eating robots, automating your desktop things. Maybe we will not have to build some of the robots here or there if we get to enterprise in this area at some point.

The rise of citizen development and personal productivity

AK: It will take some time and a lot of consideration around the costs and risks. I have a kind of similar feeling that in kind of high-volume Center of Excellence kind of automations, AI is more an addition to what you currently have, just expanding the possibilities of what you can do.

But when it comes to personal productivity, what we always used to talk about as a citizen development kind of a thing, it has become even too easy. We used to do this with VBA, which required learning and a certain level of technical prowess. Then we had those citizen development robotic tools, though they didn’t really catch on as much as we thought they will. Now we kind of go into this AI territory where you basically don’t need to have any technical skills to actually produce something that you can use. It’s a different thing whether it will be tested enough and secure, but it is possible to do those things very easily.

MS: That’s a super valid point. First of all, it’s a great opportunity and already some of the key users are catching that potential and they are trying to translate those things into their business. To be safe, I’m saying that for the time being, it is a very good tool for prototyping solutions that could be delivered. On the other hand, we are really talking about delivering the software with all of the processes that have to be around it.

The importance of governance in enterprise AI

MS: When I’m thinking about the future of the enterprise, then it will be super important to have a governance over it and it will be super important not to block it. I guess this approach with having the opportunity to have a prototype from business and then having a team that actually will deliver that as fast as possible, because we cannot talk here about delivering the software in half or one year. It doesn’t make sense for the business.

If you think about how good AI becomes into the coding, there should be a team in the businesses and enterprises have to really rethink and have that capability to use AI and AI agents that will be building in a safe manner those ideas from the business. It will be really tough to control those sometimes bottom-up ideas. It’s no problem at the moment to build a macro, an Office Script, or put html files into Excel. It’s opening so many doors for the business and the key users, and indeed governance is the most important part here.

AK: For sure, it has a huge potential to speed a lot of things up. I really like the idea of the business already developing a prototype that they come to you with. Instead of filling in a form, they come to you and say, “Hey, this is what I came up with, can you make it proper and give it to us?” And of course, the central team will also use AI to build it, but they will have the knowledge about the security and all the best practices. When you do, make sure to tell me and come back to the podcast and tell everyone how to do it properly.

Case study: My Invoice Assistant (MIA)

AK: I would like to come back to the AI agent that has just been put into production, the My Invoice Assistant agent. I’m wondering, your team has delivered more than 200 automations over the years, and most of them I presume were kind of classical automations. What made you decide that this particular problem of the invoice status was something to build an AI agent around and not try to go into the classical things that we usually do?

MS: The story is not that super short. We started really with looking on the accounts payable queries that our accounts payable team is getting. Most of us already know and recall from different companies that there are agents that are solving those queries, dispatching them, maybe solving a few of them or trying to prepare a draft and getting some insight.

Our idea was to deep dive into the P2P queries and let’s see what is the potential there at Amer Sports. We did that analysis with the support of Office Samurai. In the end, we actually have designed how an agentic world could support this. Then we started to think how to implement it, but at the same time, we needed to really be careful and think long-term.

In short, we decided that we need to make some clean-up and stabilization first, from system to process, so that stopped us from moving further in this direction for a while. But at the same time, we were thinking one of the biggest and most simple questions that P2P is getting is actually: “Has this invoice been paid or not? Do you have it actually in workflow?” My vendor is actually approaching me and I don’t know what to answer. The end user is always sending a ticket and waiting for some time. Sometimes there is a “ping-pong” email because they only gave a vendor name but didn’t answer anything more that we need, like a reference of the invoice or an amount. This throughput time could be improved.

MS: At the same time, we were considering that we could actually build a PowerBI report with all of those data that will be available for all of the users. It would be faster, it would be easy, and there would be a satisfaction for the end user. But at the same time, we are causing some risk because we are giving it to everyone in Amer Sports, not only finance, we are talking about sellers or whoever is doing business with those external guys. We are causing a risk that someone will extract all of those data or whatever happens from a security perspective.

So we started to think about what options we have and it was quite natural to try a chatbot that could give that answer. We can have a guardrail in there, like don’t share all of the invoices, let’s say maximum 10 invoices per vendor with one response, so we are a bit more safe. At the same time, you’re getting an immediate answer. If there’s a ping-pong where you didn’t provide enough information, you would get an extra question from the chatbot like, “Hey, please provide me a company code because I have no idea from which business unit you are from.” It seems that it’s helping extra for the business.

In the end, it’s something that we will not replace in the future. We still are considering to have the agent over the queries, but this is something that we still would keep: this MIA chatbot which we delivered. In the end, the idea was that we will limit the number of queries. It was really a standalone project with really low risk and basically a quick win that we could deliver.

AK: This is great thinking. When companies try to automate their mailboxes and ticketing queues, they think inside the ticketing system or inside the mailbox. But here you moved from tickets to live chat. This not only automates the whole thing so that people don’t have to just go to SAP to check something and answer a ticket, which is not a very interesting thing to do, but that response time is seconds instead of possibly days. Even when you get to automating the actual ticketing queue, this chat can still be there to offload some of the work from the queues.

MS: Indeed. The biggest benefit of this chat is the time saving for accounts payable, but the more important part is related to the satisfaction of the client, the satisfaction of the business that is asking a question. If the business has a query about what’s happening with the invoice, they are already not happy because they think something has been screwed up. Our main benefits from our perspective was actually to speed up the response for the business, to make this bigger change to have this immediate answer and have this data available for the end user. That’s our big game-changer. This Amer Sports-wide tool is not limited to some financial people. It’s a really scale-up tool.

The end-user experience via Microsoft Teams

AK: I know how it is when people get a vendor screaming at them and then they scream at us and then we do our best but then something is missing. If we can remove this friction, then this can be a process where they can scream at an AI agent all they want and it won’t care. It is also an added bonus. From the end-user perspective throughout the Amer Sports organization, how does it look like when they get a query from a vendor about an invoice? What do they do?

MS: Basically this chatbot is available for everyone via Teams, which is the tool that Amer Sports and most of the companies at the moment are using. It’s super easy access and then you basically ask a query about the invoice. You don’t have to know all of the numbers like a vendor number or company code or specifically the invoice number. AI is trying to understand you. If you say the name of the vendor, it will first of all try to catch what kind of vendor it is.

If someone will explain what’s my business unit or my entity, again, not really a company code, you can explain it’s Poland, it will give you a proposal if it’s not sure. If you don’t know the invoice reference or invoice date, it will try to give you a proposal; maybe it’s one of those few you can check out. Basically it’s a natural language conversation for the end user. As an outcome, you get information that the invoice is in the system or we don’t have this invoice in the system. If it is in the system, you’re getting the status: is it paid, is it not paid, and when most probably it will be paid.

AK: Those things look way more complicated under the hood than the end user will probably imagine. If they tell you that there’s an invoice from Lufthansa, you probably have more than a few Lufthansas in your systems because there are different group companies. To find the right vendor and the right company code the invoice should have been sent to, the tool needs to help them and keep narrowing it down until it is able to find it. The same thing with references; those are really funky strings. Behind all of this there’s a lot of business logic that the agent needs to go through to be able to talk to the person like a human would.

MS: Indeed, we had to go deeply into those scenarios that might actually happen. Take the example of the selection of the vendor and Lufthansa and the different groups that are there. The chatbot is quering the data for potential vendors that could be what actually business or user is asking for. In the end, we are giving the proposal and then the end user is deciding, “Yeah, I know that this is the vendor,” and then we can move on for further selections in the end to give the proper answer.

Architecture: Using PowerBI as a secure data source

AK: When those things happen, the agent isn’t querying the SAP directly, right? What is the data source that the agent is working with?

MS: The data source directly from the agent is actually a data model from PowerBI that we have used. We didn’t go directly to SAP, even though PowerBI is refreshed once in a half hour. It’s a short delay for invoice processing; it doesn’t make really the difference except in some really narrow cases.

We went with PowerBI not directly with SAP for different reasons. First of all, it was a security reason. Within PowerBI we actually have only the data that are required for us; we don’t allow the agent to go into the core system. We don’t want to overload SAP with different queries. We also don’t want to open too much tunnels to that particular reporting database.

We had within our team Bob, who’s actually a great PowerBI expert who could really help us very quickly within the project. Super important for this case, we had the data there. We didn’t have to go through different requests to get some extra data. It was good enough at this stage to get there.

AK: Getting it directly from SAP would pose a lot of different challenges. Through a GUI that would take too long for a chatbot. Not overloading SAP is extremely important if you want your people to be able to work on the core system comfortably. But I think it’s worth noting for the people who are thinking about such a project that this data security aspect is extremely important. In this case, rightfully as you said, in the data that the agent had access to there was only data that it can give to an employee. You don’t have guardrails within the agent and hope that it’s really not going to tell someone something they shouldn’t know; you just trim the data to the data set that is safe to share. I wanted to ask you about the amounts. In the end you decided to remove amounts from the data set and the agent isn’t even able to discuss the amounts. What was behind that decision?

MS: As you can imagine, AI projects are something general new for Amer Sports. Even though if that would not be new, we are very careful with what we are delivering because we have to understand how it works. Within our processes we are consulting whoever we have in a company that could help us, legal team from the data privacy perspective, our auditors to make sure that we are not overlapping somewhere with our proper processing of the data, and cyber security of course.

Having that in mind, thinking carefully what kind of data we can share and which one is a risky one, we decided that really amounts is something that we cannot have. It is a bit tricky because we had much discussion about it. For the business it is one of the key elements to understand which invoice is that because the amount is usually unique to recognize the invoices. They will often have a question like, “I should receive an invoice for €10,000, did we get it?”

But from the security perspective we had a few fields that we decided that even though we have them in a system and we could put them into PowerBI, we decided to take this data out. We’re talking really about amounts from the perspective of giving access to someone from business like data that should be secured, like rates between the vendors that we have. We have also different brands which, though we are one company, it would be a bit risky to show in details what are the prices here or there.

Other data that we excluded were things like who is actually the coder or approver for the process. We didn’t want to touch any personal data in that. And of course the coding, information of a cost center is often confidential data. We sometimes have B2B guys that are working for us and it could be one employee checking what’s the rate for other employees. It would be really tricky, so we had to be super secure there.

AK: This requires a lot of attention. A lot of companies doing these kind of projects don’t think about those small corner cases and combinations where each piece of the data by itself is fine, but when you combine a few pieces you can get information you shouldn’t. When I was at UBS we used to call this “toxic combinations of data.” In this case, if someone’s working B2B and you can check any amount of any vendor, then you can find out how much the company is paying them.

Scoping version one and expansion plans

AK: In terms of things that the agent doesn’t do, like payment confirmations, Ariba stuff, or PO queries, how did you get to decide what was going to be in the scope for version one? And do you plan to expand this?

MS: The story behind the scoping for this particular project was the idea that we should deliver something valuable for the business. We wanted to balance it, maybe not go with a Minimum Viable Product, but something that would be bringing real value for the P2P and for the business. We decided at some point that this scope is enough. Some of the extra things that we have in mind we could deliver later.

At the moment we are really looking into how this bot is actually in use and how much it is helpful for the business. We are planning to send over some questionnaires to some of the users to get their feedback and we are planning the further expansion and iterations of growth for this particular agent because there’s a big potential in there.

One that we clearly see is extra information related to PO numbers because we noticed at the very beginning that some users would rather search using a PO number and vendor versus an invoice reference. Unfortunately we didn’t have at this stage enough data to go further with that. It would just take time and you know how it works with AI: if you struggle too long, AI is changing quite fast and then you have to reset the project because there’s always something better on the market. Those projects cannot last too long, so from this perspective we wanted to be faster.

The sneaky plan for the future of MIA

MS: For further development, we have in mind a few interesting features. We decided to give some time to this agent and for the business to promote usage and see what the feedback will be. But at the same time we have a sneaky plan coming back to our improvement of the queries or making the queries agentic. We hope that actually MIA would be playing a key background role.

We don’t need to build an extra database to get invoice statuses. Within those queries, 60% of them require checking the invoice status first. We already have MIA in the back that could answer not to a human but actually to other agents that could use it. Other than this for MIA, we can add some knowledge like policies from P2P that would be helpful for the business users. We could expand maybe to the travel and expenses team. A lot of ideas, obviously.

AK: It is a tool built with humans in mind, but the logic and data behind it could serve other automations too. It doesn’t have to be through a Teams interface; it can call an API and get a checked response.

MS: And Amer Sports is not unique, we have enough of the tools. We are trying to be a bit more smart with the number of the tools that we are delivering and have to maintain in the end. It would be better to put more things into a fewer number of tools.

AK: When it comes to the response, on one hand we have your P2P teams that potentially could feel anywhere between “the agent is taking my job” and “finally something is going to be answering those burdensome queries.” And then we have the end users who are used to a particular person checking things for them. What was the reaction on the P2P team side and the business side?

Internal and external reactions to MIA

MS: Our P2P team was really heavily involved in that project and they were really supportive. I’m quite happy to have the P2P team as specialists because they have really good ideas and wide knowledge around the processes. Thanks to this, our project actually was successful because we were getting the proper narrow cases and unique scenarios that we had to take into account.

Based on this, I can say that they were generally quite happy to have that tool. People do not like changes, but this one was really a burden. I liked solving cases when you need to do an investigation and check all the mess on the accounts, but just answering the status was boring. That was one of those annoying things that nobody’s missing.

We are still in a promotion phase, looking closely on if the number of tickets is lowering. For the business, it’s a big role of our P2P team to educate them. In response to queries we are still getting, we can add information like, “Hey, here’s a link, please use it next time.” As you can imagine, there will be always someone that says “I know the guy in P2P, I will make a call.” And I guess we will need that, it’s a kind of customer satisfaction that will remain for some users.

The human element in customer service

MS: Privately, when you think about being a customer of a bank in Poland, it’s really hard to get the phone number to call them. Everything is digitized and nobody wants to take your calls anymore. I’m super annoyed by that. But these are our internal employees and we care about them as a GBS. From our perspective, this is the service that we provide and for some exceptional cases there will be still that.

AK: There needs to be a kind of second line for when the tool isn’t able to do something. That model that Facebook or YouTube are using where it’s virtually impossible to speak to a human just doesn’t work because there’s always a corner case that a chatbot won’t solve. But the bank example is good; I used to have a particular person that knew everything about my case when I had my mortgage. I would rather do things through a chatbot if it worked, but very often it wouldn’t. We’re probably not going to miss those human interactions in service as much as we think we will if the tools are good enough.

MS: It’s a big job of functions like myself within organizations to promote the usage of AI. At the beginning, the business will still keep calling because they’re used to it. When everyone discovers that chatbots, Copilot, ChatGPT, or whatever, are actually giving good answers, that will change. I never liked the chatbots of the old times when there was just a scenario and not really AI behind it. Now I’m really curious how it works and I use them often. Together with the growth of the mindset that AI can be used, those tools will be more and more in use.

AK: Did you catch anyone trying to jailbreak MIA?

Testing and preventing hallucinations

MS: They were trying! We thought before that we need to really limit the data. The risk of hallucination is something we had to take into consideration. You know very well from developing Office Samurai that in the AI era, the time spent actually on the solution is smaller, but the test phase is much more crucial. We did spend a lot of time on testing and tried to narrow the cases involving a quite wide team.

In the end, thanks to this, we’ve been quite successful. Besides this internal P2P test phase, we also had a test phase with the business. One person from the business caught a narrow case that we didn’t think about. It was a good idea to have that extra phase because P2P are not the end users. We managed to catch it still before going live.

AK: I agree completely. There is a lot and a lot of testing to do it properly because there are a lot of things that can go wrong. If you give people something half-baked, they’re going to take a look at it once and never use it again.

MS: That’s the worst-case scenario because you’re burnt. I’ve been doing quite a lot of testing on how Copilot in Excel could work. At some point, it really didn’t work well and I wanted to test the potential. I compared it and now Copilot is a different story within Excel, it’s doing great. But you needed an impulse to come back to it again and check it because everything is moving so fast. With Microsoft, it’s even a bit more tricky because it’s not always clearly announced.

AK: When we were discussing the architecture, it was decided that every logic we can, we put into the classical layer of the tool. We only use AI where classical programming doesn’t cut it. The AI is needed to understand the user’s intent, but getting and filtering the data is all classical logic. This way we also get rid of possibilities to hallucinate an additional invoice.

MS: Correct. Looking from a finance perspective, we have to be as accurate as possible. We have a lot of controls and we have a big discussion with audit about what it means to have an agent in a regulated environment. These are not standard controls. SOX company requirements are another round of discussion that we are tackling at the moment. If there’s a possibility to have rule-based things, let’s go with that because it’s easier to control and more accurate.

Vision for the next three years

AK: In the perfect world in three years, what is MIA doing?

MS: Hard to predict honestly. On one hand, it’s rather clear to me that we should give a bit more data there that it will be safe to be available. From the end user perspective, they would really love to have the amounts. We will need to tackle that topic and analyze what it means, perhaps for some group of key users.

At the same time, I’m seeing MIA as a key agent for accounts payable. A one-stop shop for the end user where whatever I have, I can check very quickly via the chatbot. It could create a ticket or forward the case to a human if needed. Maybe it could trigger actions, like requesting a copy of an invoice. As I told you, it could be a back engine for other types of automations. My Invoice Assistant might not be only for invoices; it might have an identity change. When you think about the future of AI in three years, I will not risk saying where we could be because it’s so much constantly changing.

AK: I share the sentiment. It’s really hard to make a prediction for six months, let alone three years. For the people who are about to kick off their AI agent project, do you have some piece of advice?

Advice for starting AI agent projects

MS: Try to do something with AI. The idea is important, have a good idea but not too big a scope because that might kill the project. Something with a bit more limited scope with some nice impact. Before that, try to do something smaller, like a chatbot with a knowledge base to understand how it works and promote that AI is helpful.

Also, trying out in the area of possible. If you want to be faster, it is good to have someone with experience. We had you guys and you knew what was possible and what to take into account. It’s about not inventing the wheel from the beginning and not making all the mistakes that other people have already made.

AK: And with your experience, what’s the dumbest thing that people think AI can do but it cannot?

MS: AI still gets stuck with thinking that it’s right. It is so self-confident that it’s correct even when it’s making a mistake. It’s hard to convince AI that it’s actually making a mistake. For me, it’s a bit frustrating. On the other hand, it’s also a matter of experience in how to properly prompt.

AK: I had this experience recently. I needed to write some simple Python code for an API and it worked, but when I asked for a change, it stopped working. I was working with Gemini and it gave me a solution, and then another solution, and then it switched back to the first solution in a loop. I stopped it only by moving the conversation to Claude and saying “help me fix it,” and it was fine. Sometimes you just need a different approach.

MS: My advice is to reset, go to another model, and start a new conversation. Don’t get frustrated.

AK: Matt, thank you so much for sharing your experience with My Invoice Assistant. I am looking forward to seeing where MIA goes. And once you figure out the citizen developer part, I would love to hear how it should be properly done.

MS: Thank you very much, and thanks for having me again.