The rise of AI has made technology an extremely hot topic, and everyone wants to engage with it. However, trying to run AI projects using established methodologies, such as the classical Waterfall approach often used for predictable RPA projects, is proving difficult.
AI projects are inherently different. They are less predictable and require a new approach because AI is not a direct product you simply buy and deploy; it is a long maintenance process.
If you aim to launch an AI program that delivers real business impact rather than just becoming another “showoff” technology, you need a robust framework. Here are the 10 golden rules Office Samurai uses for managing AI projects.
The hybrid methodology: merging control and experimentation
I projects necessitate a blended approach, mixing the best parts of both Agile and Waterfall methodologies.
Agile for experiments:
Agileis the best way forward. This is because AI projects must focus on experiments and continuous learning. We must work in small pieces or cycles, allowing us to discover potential issues and check results quickly.
Waterfall for sponsors:
While the delivery team uses Agile methods, you cannot abandon Waterfall entirely. Waterfall elements are necessary for communicating with sponsors and decision-makers. Sponsors have budgets, targets, and milestones that they need to report to their supervisors. The Waterfall component allows for checkpoint reviews, ensuring that the budget is used properly, and providing self-diagnosis on speed and direction.
The crucial role of retrospectives (retros):
Retros are absolutely essential in AI projects. They should be conducted not only at the end but after every small cycle. A retrospective is a safe space for the delivery team (developers, analysts, architects) to talk honestly about what went wrong, what went well, and what they learned. This continuous learning is crucial because AI projects are newer and far less predictable.
The 10 Golden Rules for Managing AI Projects
Based on our experience, these are the key principles that ensure success in AI delivery:
- Know your team and talents
Identify the potential within your existing team and invest in skill development. The tools for utilizing complex AI technologies are becoming less complicated. Often, you need a very good analyst more than a “big brain developer”. People with analytical thinking and strong natural language communication skills (crucial for prompt writing in GenAI) are perfectly suited to start working with AI, representing a huge opportunity for business analysts. - Build internal AI Champions
Empower people to become “apostles of AI” who will promote the AI culture and support adoption across the organization. This is a critical part of change management, similar to the process seen during the implementation of Citizen Development in RPA. - Manage sponsor expectations clearly
Communicate transparently that success is not guaranteed, and AI projects carry inherent risk. These projects are similar to R&D (Research & Development). Sponsors must be aware that the way to the goal may change. Expectations are often extremely high due to market hype; you must keep them grounded. - Keep learning and upskill your team
Technology evolves so rapidly that continuous effort is required to keep up. Organizations must run workshops and learning programs. However, it is essential to make an informed decision (informed choice) and not upgrade just because a new model (e.g., GenAI) is released, especially if the current one is stable or “hallucinates less”. - Ensure data quality and availability data is the fuel of AI
Without clean and available data, projects cannot succeed. If there is a “mess” in your unstructured data, it is much more difficult to train the model properly. Even though GenAI is better at handling unclean data than traditional machine learning, striving for order is crucial. - Plan for model maintenance (MLOps)
The AI project does not finish upon deployment. It is a long maintenance process that requires an ongoing budget. Testing is difficult because the results are nondeterministic. Lack of maintenance means the solution gets worse over time. - Transparency, ethics, and risk management
Minimize bias and ensure regulatory compliance. Be transparent with employees about what the tool will bring and how to work with it, ensuring they feel safe. This helps mitigate fears that AI will take jobs. The world will change, and new professions will emerge (like bot shepherds in RPA or machine learning engineers). - Measure business impact
Define KPIs and ROI in business terms, not only technical metrics like accuracy. You must show that the project affects the business positively to secure funding for the next phase. Measuring impact can be hard, especially when a tool helps many people only a little bit. - Communication and expectation management
This rule emphasizes the need for transparency and grounding expectations. Expectations can be extremely high due to consultants “selling bullshit” at conferences or on LinkedIn. You must avoid building the AI strategy on unrealistic promises, as this could prematurely end the entire AI program. - Embrace continuous change
Models age, and technology evolves. You must be open to change. If a better, commercially available solution is discovered mid-project, you must be ready to stop or change the project’s direction. If an organization requires 100% certainty that something will work for a very long time, it might not be the right time to play with AI yet.
The four faces of AI projects
Organizations typically engage with AI through four distinct types of projects, each requiring different managerial focus:
Pilot projects (PoC) or experimental projects:
These are short (two to three weeks maximum) and low-cost projects used to “taste” the technology. The goal is to gain a feeling for how AI works with your internal tools, applications, and teams.
Deployment projects:
These are the larger, long-term, and difficult experimental projects. They are tough to manage but deliver the biggest value to the organization.
Buying and implementing AI tools (e.g., Copilot):
This involves purchasing commercial tools like Microsoft Copilot and implementing them. The challenge here is less about building and more about change management and training. You need to buy not just licenses, but also the knowledge to show people how to use the tool properly, often relying on internal “ambassadors” or “champions”.
Building an AI strategy:
This is a long-term process that determines what to implement, with what resources, and what policies (e.g., ethical guidelines) will guide the organization. It is vital that strategy building runs in parallel with early PoC projects, allowing the results of those first experiments to inform and shape the strategy.
AI is R&D: Why you need a maintenance mindset
Understanding that AI is fundamentally a Research and Development activity is crucial for budgeting and planning.
The R&D Imperative Since AI technologies are constantly evolving and are so fresh, we often lack the experience to know exactly what will work. We must take a risk. If an organization always wants to stick to safe solutions and requires 100% certainty, they may not be ready for AI.
The maintenance challenge (MLOps) AI solutions require a long maintenance process. Models need continuous support from the business side for training. The budget must include a buffer for future maintenance.
Furthermore, maintenance involves rigorous testing, which is especially difficult because AI systems yield nondeterministic results. It is necessary to have a way of testing thoroughly (often using another LLM engine to compare results) to ensure the solution doesn’t deteriorate over time. Without continuous maintenance, the solution will eventually get worse, forcing the company to shut it down.
If we embrace these rules – prioritizing continuous learning, hybrid methodology, and acceptance of risk and change – we can navigate the current hype and harness AI to build a good future.
