Video: AI in the Flow of Work: Turning Project Data into Executive Decisions | Duration: 3704s | Summary: AI in the Flow of Work: Turning Project Data into Executive Decisions | Chapters: Welcome and Introduction (4s), Speaker Introduction (87.22s), Market Pressures (315.895s), AI Evaluation Framework (459.215s), AI Implementation Framework (839.805s), AI Maturity Arc (1139.455s), Crestline Case Study (2431.765s), Agent Orchestration (2807.83s), Leadership and Next Steps (3149.205s), Q&A Data Management (3410.86s), AI Risks & Wrap-Up (3573.585s)
Transcript for "AI in the Flow of Work: Turning Project Data into Executive Decisions": Good morning, good evening, good afternoon from wherever you're joining us today. Thank you so much for spending some time with us today, as we talk about AI in the flow of work. Before I introduce myself and jump into the content, just a couple of housekeeping items. First of all, for the best webinar experience, please use Google Chrome or Firefox. Audio is being streamed through your computer. You'll be able to download the slides, from the download center or the document center within the Goldcast, UI. You can adjust your screen to get things sized just the way you want to get the best viewing experience. We will be emailing, an on demand recording of the session after the webinar ends. So if you wanna share this with others within your organization or their peers, please go ahead and do so. We will have some time at the end of the session for q and a, so please enter your questions in the q and a box and we will get to as many of those, as we can. So with that, again, welcome, everyone. Thanks for spending about the next hour with me. We've got a lot to cover. So I'll get right into it. I'm gonna spend about forty five minutes on, you know, what I think is one of the most important conversations happening in consulting firms right now, and that is how AI actually changes the way project based firms, consulting firms run, and what that means over the next twelve to twenty four months. First, a little bit on me. My name is Brett Tussaus, VP of product and technology at Deltek. Prior to coming to Deltek, I spent fifteen years, inside a project based firm running technology and operations actually as a Deltek customer from the inside. And then I've spent the last sixteen years here at Deltek leading product teams focused on consulting, professional services, engineering, and architecture solutions. And my focus then and now is how technology actually solves the operational pain points firms like all of yours feel, every single day. That's the friction between project work, financial control, partner level visibility, all of the the various frictions that that many of us experience on a day to day basis. So while I'll be talking about AI today, what I'm really talking about is operational discipline in project based businesses, consulting businesses, like yours. So just sort of keep that lens in mind over the next forty five minutes or so. And to start that conversation, here's the prediction I'd put real money on. Most AI initiatives in consulting firms are going to underperform expectations over the next eighteen months. And it's not gonna be because the AI itself is bad or the the models are remarkable these days, and they're getting better and better every quarter. It's going to be because firms keep confusing AI features with AI in the flow of work. And the pattern's becoming familiar. A firm buys a copilot, runs a proof of concept. The demos look magical, of course. Six months later, utilization on the tool is anemic, maybe 4%, and nobody can point to a single project or engagement that was delivered differently because of that AI capability. In fact, Gartner recently predicted that over 40% of a agentic AI project will be canceled by the end of twenty twenty seven, citing escalating costs, unclear business value, and the phenomenon that I think many of our many of us are experiencing seeing these days, agent washing. And that is vendors or anyone relabeling conventional automation as agentic AI. And these are not really AI problem. It's it's a deployment problem, I would posit. The AI maybe was adjacent to the work and not inside the work, and that's part of the root cause here. And that's why I wanna be clear upfront about what this session is really about. It's not a vantage point pitch. It's not a Deltek road map walk through or anything like that. The goal over the next forty five minutes is to give you all a clear way to think about your firm's AI journey, what to look for, what the real maturity levels actually are, and how to navigate them with intention and discipline. I'll use vantage point as, an illustrative vehicle because it's what I know best, and it lets me show you concrete examples you can truly relate to them in your business. But everything I share applies to any AI capability that takes this seriously. The goal isn't, adoption of a tool. The goal is clarity on your maturation. The firms, in my opinion, that get real value from AI over the next twenty four months or so will be the ones who, you know, stop chasing features and start asking a different question. Did this AI close a loop in our delivery life cycle or did it just answer a question? I think that's the line between AI that compounds value, and AI that becomes another tab within one of your software solutions that nobody ever opens. If you're a partner at a consulting firm right now, you're probably being shown roughly one new AI tool every week or coming across one new AI tool every week. Most of them will not change how your firm operates. Some of them will. The framework I'm gonna talk you through is meant to help you tell the difference. So before I dig into AI specifically, I I wanna talk a little bit about the operating context that I'm sort of thinking in as we go through today's presentations and why this conversation, I think, is relatively urgent for consulting right now. I think there's four forces that are converging on the same set of operating metrics, if you will. Utilization, realization, and effective rate per consultant all at the same time. The first force is rate compression, Procurement led RFPs, fixed fee deal structures, and rate cap rate card pushback are all converging. Even firms growing top line are seeing realized rates flatten or decline. The lever is no longer charge more. The lever is deliver more leverage per consultant. Then utilization is harder to defend. Hybrid work, fragmented bench time, and engagements cope that changes mid flight, have made utilization, noisier, if you will. The 80% target is real, but harder to hit consistently. And as we all know, every percentage point of utilization slippage, is real margin that you're losing. And then a third dynamic is AI native challenger. Those organizations that are comp competitors to all of you that are AI native. You know, there are now consulting firms launching that are AI native from day one. They have lower headcount per engagement, lower cost structure, aggressive pricing. They don't have necessarily your relationships or your bench depth, but they have a cost structure your firm doesn't have. And worth noting, they're not waiting for AI to mature. They're starting at higher levels of the maturity arc, and I I'll walk you through that maturity arc in a minute here. And finally, client expectations are shifting. Your clients are using AI internally. They expect their consultants to bring more leverage to every meeting, every deliverable, every recommendation. The the common adage or the adage that's becoming common is I could have gotten that out of a GPT is becoming real feedback in client debriefs. And what ties, these four things together is the consulting operating model. Bill smart people by the hour to think hard for clients is under real structural pressure for the first time in a generation. AI is either the response to that pressure or it's the accelerator of it. It depends really on how you deploy it and how you look at it. And I think that's why this conversation matters so much right now and and why I think firms that move thoughtfully over the next twelve to twenty four months will pull away from the firms who move too slowly or move blindly. So let's get into AI specifically next, and this is the part most AI conversation conversations these days skip. A general copilot trained on the open Internet doesn't know what your realization rate. It doesn't know what a billing rate exception looks like. It doesn't know, that a two hour gap on one of your people's time sheets is a utilization problem, not necessarily a calendar problem. Consulting and project based organizations are structurally different from most industries in the five ways that you see on this slide. First, revenue is project based, so every margin is fragile. One bad pursuit decision can swallow two good engagements. Utilization is the p and l lever, and an hour mistracked is an hour gone. There's no inventory to write down, unlike other sort of industries. It's really just hours that don't get billed. And knowledge doesn't necessarily compound across engagements unless someone within your organization deliberately forces it to, which is why, many engagements kickoffs feels like often feel like starting from scratch. Contracts and approvals carry weight, SOWs, MSAs, conflict checks, billing terms. You know, there's no undo on a scope creep email that lands in one of your most important strategic clients inboxes. And the decisions that actually matter cut across business development, they cut across delivery and finance, which means if you're using, AI in a siloed way, AI that only seems maybe one of those domains, it's really dead on arrival. The firms that that win here won't have the most AI. They'll have the AI that actually understands these five realities, which really brings us to three shifts I think every consulting leader should be tracking right now regardless of which capability, which vendor, or which type of AI, you're currently looking at. And those three shifts, really can anchor every one of those AI evaluations you'll do. And when I say AI evaluations, it's internal pilots, it's evaluating an external tool, it's maybe building something yourself. The same real the three shifts really apply. The first shift, is from AI that answers questions, the chatbot, that we all know and love, to AI that closes loops. The return on investment in consulting isn't I asked a question, got a fast answer. It's AI drafted the status email I was gonna write anyway, flag the pursuits that were stalling, or routed the approval that was about to miss some sort of SLA or something like that. Loop closure, is really where adoption lives and what separates AI that advances decision from AI that just produces a recommendation. So, again, answers from answers to closed loop is shift number one. Shift number two is how we frame the value. Productivity is is really the wrong frame. Decision velocity is the right one. A lot of the AI that all of us come across these days, we see it pitched as a time saver. Save your team ten hours a week, draft emails faster, summarize meeting minutes. That's fine, but it's really not the prize that I think we should have our eye on when it comes to fully realizing what AI is capable of. The prize for a project based firm like all of yours is shorter time from questions decision to action. A faster task doesn't change your operating rhythm. A faster decision, I think, has more opportunity to change that operating rhythm. And faster decisions are only valuable if they're decisions that you can actually defend, which is why, you know, governance certainly plays a factor here as much as the value frame level itself, in terms of, again, moving from productivity to decision velocity. And then the third shift is one that you're gonna hear me repeat a few times in the context of today's session is, you know, from point tools to project context platforms. When it comes to AI, context is one of the most important things right now. It really matters more than the model and the underlying technology. An AI tool without your engagement data, without your organizational data is a bit of a novelty. With it, you get leverage, and we'll come back again to this a minute because I think there's a key point that I wanna make sure I hit home, a few times today. So answers to loops, productivity to decision velocity, point tools to project context platform. You know, really hold these three things in your head as we get through this because they're the lens for pretty much everything else that I'm gonna show here today. But before we get there, I wanna give you a tool, maybe a a little bit of a checklist to apply to any AI capability you're evaluating over the next year. And, again, when I say AI capability, it could be something you're building internally, it could be a vendor you're evaluating, it could be a number of different things. This is meant to apply, you know, fairly broadly and agnostically across different AI evaluations that you're doing. So if you do nothing else with this session, I encourage you to, you know, sort of take this checklist with you. Use it on every one of those proof of concepts, demos, every internal experiment, every conversation about AI happening at your firm. I'd rather you walk out of the session with the checklist than maybe some of the specific examples that I'm about to show you. Notice what's not on this list. There are some things on here, like, does it use GPT 5.5 or does it use SONNET or Opus or Claude or OpenAI or Gemini or any of those things? Is it built on the latest model isn't in this list either. Those questions don't predict whether AI will work in your firm, but I think these six do. And if you sort of use this as a look in the mirror to a certain extent, they can help you, you know, be more successful. So let me walk through each. First of all, embedded. Does the AI live where work actually happens in your organization? Inside timesheets, the engagement record, the approval workflow, or does it sit in a separate tab that your people, your consultants have to leave their current workflow to use? Context switching kills adoption. We see that all the time inside, you know, Deltek solutions and whatnot as we talk to organizations. So adjacent is not good. Embedded definitely scales with with you and your consultants in your organization. Second, is it role secured? Does it respect permissions you already enforced in your systems of record? Surfacing engagement financials to the wrong consultant creates a compliance incident with some challenges. Surfacing client information to someone with a conflict creates an actual liability. So ask and and make sure you evaluate any AI tool you put in front of your firm to know that the security is enforced at the architectural level, because, again, that's a critical piece of being successful and and reducing risk when it comes to these AI tools. The third question is, is it traceable? Can you see what the AI used? Is there human in the loop? Is it observable? Is there human on the loop? Those types of things. You need to be able to see the sources, the audit trail. There needs to be transparency around actions and decisions that you're allowing AI to take in in in any level of autonomy. You know, the moment you take an AI answer to a client, an auditor, or a partner, or a committee, you need to be able to defend it. Obviously, the the AI said so isn't defensible, especially in some regulated industries where your clients sit. You need the receipts. You need to be able to show your work. We've all heard the the horror stories over the past couple years of, you know, somebody using AI to generate something, not reviewing it and, you know, either reading it aloud or putting it in front of someone and then realizing it was, all a big hallucination. So that traceable piece is key as well. Your data, this one is absolutely huge. You know, data is the new oil these days. Do your AI tools run on your data, your engagements, your clients, your contracts? You know, the test is pretty simple. Can it tell you, for example, the job to date margin on a specific engagement of yours with the right role based security applied? You know, if it's if yes, then it's grounded in your data. If it's not, you know, you might wanna question whether or not it's just a demo, versus something that can really be integrated into your your operations. And then the fifth one, human in the loop. I sort of talked about this under the role secured piece as well, but, you know, for any high stakes actions, an invoice going to a client, a proposal being submitted, a journal entry hitting the GL, is there a checkpoint? The the the scary thing about AI sometimes is it can be confidently wrong, and the cost of confident wrong in a project based firm could be real money. So a good capability automates the loop but pauses for human review, you know, at the moments that matter the most. And then lastly, is it measurable? Can you measure the impact in cycle time, risk, margin, cash flow, win rate? If you're hearing from your users, your consultants that they love it, you know, that's a feeling. That's not a measurement. The pilots that get killed, you know, after nine months are the ones with no metrics. So really pick some measurable pieces as you implement AI so you can understand their level of success and, you know, either, change course if they're not being successful or build on that success in other parts of your business. And the line I'd really like you to sort of walk out with, and this is the one to remember, even if you forget everything else from today, is if it's adjacent, it's wrong. If it if the AI lives in a separate tool, a separate tab, a separate workflow from where the work actually happens, from where you live day to day, from where your consultants live day to day, you know, it's really not in the flow of your work. It's just another login. It's just another tool, which is exactly why, again, that embedded piece is important. And it's also why, there's a bit of a platform argument here that I think is also important and should be considered when you're thinking, about AI capabilities, AI tools, and that sort of thing. Any serious consulting AI capability built by a vendor or built by, someone internally, you know, I think has to follow these three principles when it comes to platform, capability and sort of platform foundation. The first is project centric. This sort of builds on what I said before in terms of knowing your project data. It doesn't the tool doesn't have to be told what a realization rate, what a CPI is, you know, what a billing rate exception looks like, or what a write off is. It sort of knows that vernacular, of the industry in which you operate, and that is critically important to its success. Without that grounding, you're asking a brilliant assistant who has never seen your business to make recommendations about your business. So, that's a sort of a key sort of lens to look through again, that project centricity piece. Embedded, we talked about this. The AI shows up in the workflows where work already happens inside those key areas of, of your day to day and your consultant's day to day processes. That is is key, not a separate chat window. Again, if it's JSON, it's not gonna be as successful. And then the last one here, is secure. Enterprise governance is the foundation here, not an afterthought. You can't talk about AI these days without talking about these pieces in terms of, like I said, enterprise grade governance, data privacy, all of those types of things need to be baked into. And that's another reason, why the platform is so powerful. And, these sort of three reasons overall are are structural reasons platforms matter beyond, even just these three principles. The AI model landscape will look very different in twenty four months. Heck, it'll look very different in probably the next twenty four days at the staggering rate some of this technology is moving. So a feature is is is a bet on one model, one moment, one workflow, whereas a platform, you know, it absorbs and flexes to change, whether it be multi cloud, multi model, multi tech stack, whatever the case may be. The right AI handles the right task without locking your firm into a very specific technology. So whatever direction your firm takes, you're going to make architectural choices over the next, let's say, twelve to eighteen months that lock you into an AI direction for the next five years. The instinct will be to pick on features. The right move is to make sure that those features are supported by a very mature platform. So something else to keep in mind because as things evolve, as this technology continues to grow, and I think we all know it will, having that platform is foundational for your organization will set you up, very, very well for for for down the road. So for the rest of this session, I'll use Deltek VantagePoint as my illustrative vehicle. But like I said before, what I'm walking you through, the maturity arc that I'm gonna talk about here in a second applies to any platform. So I'm using VantagePoint just as an example because it allows me to put sort of some real concrete examples on the screen. But, again, this applies to any AI tool that that we're talking about. So like I said, what what I wanna walk you through is the maturity arc. And what I mean by maturity arc is really three stages that describe how AI can show up in a consulting or project based firm like yours, and really sort of be in the flow of work. So three things to know here. First, these are time based stages. All three exist in the market today in various forms. Different platforms operate at different maturity levels. Where your firm sits on this arc is a function of which capabilities you've adopted and how you've deployed them, not which year it is because, again, much of this avail is available today. And it's worth saying out loud that those AI native challengers, those competitors that I talked about earlier that are starting off as AI native firms are starting at stage two or stage three in this arc today. So maturity is as much a choice of firms makes, as it is, you know, based on sort of the calendar and and the technology momentum behind some of these some of these tech some of these capabilities. So second, I'll use vantage point again as my illustration because it gives me a chance to, you know, really show you some, concrete examples of what exists within this maturity arc. And then third, you know, mentally run every example I show you against the six questions we covered earlier. Is it adjacent or is it in the flow? Is it role secured? Is it traceable? Is it using your data? Does it have human in the loop? Is it measurable? You know, those questions predict whether the capability will actually work in your firm at any maternity level. Alright. So with that, let's start with stage one, which I'm calling the ask stage. And the ask stage is certainly foundational, level of the arc or the starting point for the arc. And in plain English, what stage one is all about is action and plain English in and actions and answers out. That's really what stage one is all about. The version that matters in consulting, isn't a generic Copilot that floats on top of your business. It's it's truly, like I said before, in a in a system that's grounded in your engagement data, respecting the way you, consume data and the users within your organization consume data. The executive and and all the people on this call stops hunting for miss for information. They ask a question in plain English and get a grounded answer back. When a partner asks what's the job to date margin on the Concord engagement, The AI isn't guessing. It's pulling from that user's actual project data, respecting their role security, returning a number that they can stand behind. The same pattern shows up in, say, curated briefings, if you will. Open an engagement record and get a one screen summary of financial health, pursuit status, recent activity, all in plain English. What used to take, you know, twenty minutes of clicking through reports or asking somebody else in the organization to generate a report for you, can now take five seconds as a result of some of these tools. And that's the test for stage one, not whether the AI can chat, but whether the AI knows your business. You know, run it against the sick quest six questions again that I talked about earlier. Many of the generic copilots out there fail on half of the tasks or half of the asks and questions that are out there. The right stage one capability, you know, passes all six of those questions. This is where AI and the flow of the work really starts to feel real. But I would say answering questions isn't the most, underrated capability at this maturity level at stage one. I would say where this really starts to get interesting, is where action comes into play. The AI can do things like draft a status email from engagement data, create a contact record, log in activity, update a record, and that's the difference between AI, again, that just answers questions, and AI that starts to close the loops. As I mentioned before, closing those loops is is really a key piece that you wanna be thinking about, when it comes to AI, in the flow of work, even at this sort of foundational maturity level stage one of the arc. Here's what that looks like in a little bit more detail. For example, send the prompt to your system, draft an engagement status email to the engagement partner of the hospital project, include relevant financial metrics, flag areas of concern, and the AI pulls that data directly from your system. It drafts the email in a professional tone. It surfaces what's, you know, might be an anomaly in that data, a budget overrun or a missed milestone or a pending invoice. The partner reads it, edits it, if needed, sends it. And that's not just faster typing. That's faster decisions and faster communications, at the same time. So, again, it's not just about, productivity. It's really about that decision velocity as well. And even at stage one, the value of AI in the flow of work isn't faster answers. It's, again, that loop class closure. If a stage one capability can answer questions but can't take action, you've got an assistant, if you will, not a teammate. And teammates are the ones that really will help you start closing loop. And there's one more stage one capability that gets less attention than it deserves, and it's where consulting firms have a real opportunity to to compound, across engagements. And this may be, for lack of a better way of saying it, the unsexy stage one capability, but I still think it matters, significantly for consulting firms specifically, and that is engagement readiness. This is, maybe another way to term it is the kickoff problem. Every consulting engagement regardless of firm or industry starts often starts with the same friction. The kickoff team needs to know what do we do for this client before? What do we do for similar clients or similar engagements? Who in our bench has done this kind of work? What frameworks did we use in the past? What templates apply? What's the client's history with us? And what political dynamics with that client or as a part of that relationship should we know about. In many firms today with many engagements, that information is often scattered. SharePoint folders, yellow folders, Salesforce records, you know, you name it. It's scattered across a lot of different, repositories in a lot of different mediums. You know, the personal drive of the partner who left two years ago, a consultant who's been at the firm long enough to remember what happened the last time you worked for that client. You know, by the time the engagement team has gathered all of that information, oftentimes, the kickoff meeting, has already happened. And as a result, you as an organization were not able to put your your best foot forward. So stage one AI changes that. Open the engagement record. The AI surfaces every prior engagement with this client and what the takeaways were, every similar engagement at adjacent clients, similar projects and whatnot that you've delivered, the right, SMEs, on your bench ranked by relevant experience, the deliverable templates that you've worked on before in similar engagements, the conflict check status if relevant for that particular engagement, all in a matter of seconds and, again, grounded in your firm's actual history, your firm's history about similar engagements, your firm's history about working perhaps for that same client. This is what the knowledge, doesn't compound problem that I talked about earlier. This is what that solves. And it's not as a a knowledge management initiative, if you will. Those have failed that many organizations have had problems with knowledge management initiatives for the past several decades. But instead, as a byproduct of AI that's in the flow of work, this, again, can help that knowledge doesn't compound problem that so many organizations see. And this is also a great example of what embedded means. Again, the AI isn't a separate search tool. Instead, it surfaces the right context inside the engagement record when the team is actually doing the kickoff. That's the difference between, again, adjacent and embedded, and it's why stage one done well, is more than than just a chatbot. Alright. So that's stage one. Let's talk about stage two, and I'm calling this one insight. And this is where, AI stops just answering and starts actually watching. And what I mean by watching is surfacing what needs attention before you ask or before it becomes a bigger problem down the road. An AI in the flow of work in this case stops being, again, a tool you reach for and starts being a teammate that watches and helps. In the underlying pattern, AI moves from waiting to be asked to surfacing what needs attention before you ask. You know, in stage one, AI compress the gap between questions and answers. In stage two, it compresses the gap between events and awareness. And that's that's really fundamentally a different maturity level. I'll show you four examples across the next four slides. One in business development, one in delivery, one in relationship management, and one So this this particular example, this is the BD partner moving from gut feel to evidence, but the evidence comes to them. They don't have to go find it, and that's really sort of a shift from stage one to stage two in this maturity arc. So the next example, is what I like to call engagement insights. Insights, that word, that concept is is really very prevalent when it comes to, AI capabilities. And the the idea here is every night, AI analyzes every engagement in your organization, and does that sort of by partner or engagement leader, and and rolls it up to that that sort of organizational unit within your business. And it looks for, anomalies, something like maybe an earned value anomaly, earned revenue ahead of invoicing, spending ahead of earned revenue, earned value behind plan, resource burn rate spikes, you know, all those different things that, can be problematic as you deliver that engagement and service your client. And then what it does is it surfaces only, you know, maybe the three to four that are most critical items to that partner's day, not 47 different alerts. Three. And I think that's the difference between dashboard fatigue and decision support. You know, there are dashboards everywhere today, and I think dashboard fatigue is a real thing for people that manage engagements like your businesses do. So, again, being able to to, distill it down to the three most critical alerts, three most critical insights, is a key part of AI in the flow of work as well. The partner still owns every decision, but, hopefully, they own it from a position of being ahead of the work flagged by the AI before the metric tips into crisis territory or problem that you can't can't, recover from. And your engagement leaders aren't drowning in too little information. There's no doubt about that. Just like I said a couple minutes ago, the the dashboard fatigue. Again, they're not drowning in too little information. What they're drowning in is too too much information, but none of it prioritized. It's sort of the the data rich concept, but signal poor. Dashboards and engagement status report, billing report, utilization report, a project performance report. You name it. There's all kinds of reports and and dashboards. But those reports are not necessarily telling the the partner, the engagement leader, the project manager to what to do today as a result of those critical signals or critical insights. So stage two AI does that prioritization for them, and the answer is almost always three things. Again, not a list, three things, and that's teammate behavior, not tool behavior. Tools give you everything. Teammates give you what matter most, for that day, for that engagement, for what's what what needs critical attention, at that moment in time. The third example is about relationships, and, best way to describe this one or to title this one is customer intelligence. This is what consulting partners intuitively want, I think, but general purpose AI tools really aren't delivering well on today. And the idea here is this this tool monitors external sources, news feeds, press releases, social regulatory filings, maybe for, m and a activity, leadership changes, projects that have been awarded, things like that, on on your client base. And when a major client lands a Fortune 500 deal or files a regulatory action or appoints a new c o c I CEO, the the AI surface it. And this is sort of that loop closure piece again. It drafts the congratulatory note, the outreach email, or the LinkedIn post in your firm's voice, and the partner can be the first to call or send the first note before maybe your competitors even notice. And this is really the difference between AI, again, that helps you do things faster, and AI that surfaces things you didn't know you needed to do. And that second category, AI that initiates value is is, again, what stage two is fundamentally about. For partners managing relationships across 30 to 40 clients, this is genuinely transformative. Today, they pick up on these signals if they happen to read the right newsletter that morning or, you know, the right blog post or whatever the case may be. Tomorrow, this AI brings the signals to them, drafts the response suggested response or action, and gives them back the hours they would have spent, on commercial intelligence if they would have spent anything on it at all and gotten some value from that customer intelligence. So, again, this is a lot what stage two is fundamentally all about. And then the first example is is more operational in nature. And this is oftentimes where stage two earns it earns its keep, if you will. And it's worth being precise about how. This is sort of anomaly detection, but it's really not because anomaly detection tools find statistical outliers. This expense report is unusual compared to the average. That's useful, but it's not what most firms actually need. What firms need is something that catches the noncompliant, not just the unusual, if you will. So instead of raw anomaly detection, this example is something that I think is more powerful and and what I've titled here is intelligent validation, and and here's how it works. You upload your expense policy in its native form, maybe the same PDF or Word document that you hand to a new hire on your expense policy. It's plain English. It's no code. There's no rules engine to go in and build and configure. What the AI then does is it parses the pieces from that policy per diem limits, attendee rules, receipt requirements, approval thresholds into, structured validation logic that runs against every submitted line from within an expense report. So rather than telling an approver this submission is unusual, it tells them this $185 client dinner exceeds the $120 entertainment per diem in policy section four dot two. So it's not only flagging it in detail in flagging in plain English, but it's also giving you a citation back to where it it it, found that particular policy element from that policy document that that you shared earlier with the system. And that's a that's a fundamentally fundamentally different conversation. Statistical outliers are interesting. There's definitely value there. I'm not saying there isn't, but policy violations with citations are auditable. Every approval is evaluated against documented rules with a pointer back to the source paragraph to source wherever that policy came from. And this is the capability that scales institutional knowledge. Every approver effectively has the full policy memorized and applies it consistently. And the firm that gets a defensible audit trail through this, and that's something raw anomaly detection really can't produce. So what ties these four stages and or these four stage two examples together is in stage one, AI compress the gap between question and answer. In stage two, it compresses the gap between event and awareness. Same arc, of course, and more maturity. And stage three is where things really get interesting with the concept of of orchestration. But a quick note before I leave stage two because there's a flavor of all of this that that if it goes wrong, you know, it's it's, potentially very risky, and it's worth a beat on what, you know, governance led really means as I mentioned earlier. You know, there's a version of this that looks a little bit like surveillance, cameras on consultants, a o AI watching every keystroke, anomaly detection that turns into a witch hunt or something like that. And that's not what governance led mean or it's not what works quite frankly. Governance led means three things. The first piece is role based security enforced at the model layer, which is what I said before as well. The AI literally cannot see data the user can't see. Architectural enforcement, not a permissions overlay. Second, every insight is explainable. Again, observable, traceable, auditable, human on the loop. The AI can show its work, which which data points triggered the flag, what the threshold was, why this engagement score differently than others. And then third, escalation paths respect human judgment. Anomaly detection doesn't auto block. It surfaces to a reviewer with context, and the reviewer makes that final, that final decision. That human judgment piece is key there. And in in regulated industries, especially things like financial services, health care, defense, government, you know, this is an optional. I think we all know that. Your client will ask you to attest to it, and the firm that can answer that question with a clear audit trail will win work that firms that can't will lose. Governance isn't doesn't have to be friction. Governance is the license to deploy AI, in an environment where, you know, upside actually matters. So, like I said, we're just taking a beat on this governance led, not surveillance led concept because, the risk involved in some of this, with the risk involved in some of this, you need to have this in the back of your head as as you evaluate, as you build, as you deploy, as you implement, and as you train, the people within your organization. And before I jump to, stage three in the maturity arc, I wanna make this a little bit more concrete. I wanna walk you through a a composite hypothetical firm that I'm calling Crestline Advisors. And the reason I like to take this approach is it it makes it a little bit more real while it is a hypothetical firm. This is sort of a composite of, observations that we at Deltek have after working with customers, working with organizations. And it it like I said, it turns this into a little bit of a use case while hypothetical, but it it adds some realness to, the types of things that that you're gonna see come to life and the patterns that you're gonna see, if you really think about AI and the flow of work as I've talked about it in the first two stages of the maturity arc. The setup here is, let's call this, about a 400 consultant firm, mix of advisory and implementation work, a few offices, the kind I think that many many of the firms on this call or this webinar are or maybe you compete with. And here's what, Crestline's advisers, operating picture looked like twelve months ago. First of all, pursuit go no decisions took nine days on average from RFP receipt to commit or pass. By the time they decided to pursue, you know, half the time the client had already shortlisted. The write off rate on engagements ran around 6% of build hours. Month end close took eleven business days with, the controller's team in a fire drill the last week of every month. I'm sure some people can relate to that. Proposal turnaround on RFPs averaged about three weeks. Twelve months later, after they started realizing different aspects of this maturity arc starting with stage one, the ask capabilities, and then layering in stage two insights, then beginning to think about how they would deploy stage three, you know, here's where they landed. Pursuit and go no go is now two to three days. Pursuit score gives the partner committee a a daily rank of you. Conversation moves from should we pursue to we've already decided. Should we change the resourcing? You know, as a result, write off rate is down to 3.2%. Month end close is six days. Controllers team out of fire drill mode and proposal turnaround dropped to nine days. So, again, while this is a hypothetical company, these are the types of things we're starting to see from real firms, real organizations, as a result of of truly applying a a I in the flow of work. So three metrics here, all improved by 40 to 50, and you're seeing, you know, the potential top line growth of 14%, as a result on flat headcount. And I wanna underline something here. The point of Crestline Advisors isn't necessarily numbers. It's where the leverage came from. They didn't buy more AI. They put AI again in the flow of work in the flow of the work that was already happening, and they were deliberate about which level of maturity are to operate at and when to move up. The AI really became the lever in this case, and the discipline was the firms itself in terms of how that lever was applied, in the work they were already doing. Again, hypothetical in in illustration, but certainly drawn from patterns we've seen at Deltek, in real firms over the past, several quarters. Now that our our hypothetical Crestline advisors, you know, what did they get right in this process? Again, this is, gleaned from our conversations with real customers that are going through this maturity arc right now, And the patterns that you're seeing here, I think, are are real as we've talked to for talk talk to firms as well. First, they clean the data before they scaled AI. This is absolutely critical. As I said before, data is the new oil in the AI era that we're in. So in this case, Crestline Advisors took the first three months almost entirely dedicated to data discipline, cleaning the client master, standardizing engagement codes, enforcing time sheet categories. AI didn't go live until the underlying signal was clean because noisy data creates noisy signals. And, again, this is a common mistake that we see in in real engagements. Firms deploy AI on top of messy data, and then they blame the AI, because the the outputs are coming back again confidently wrong. But that confidently wrong is the result and the product of of just bad data, not bad AI necessarily. Second, Crestline Advisors picked two workflows, not 10. You know, the temptation is to, you know, boil the ocean, if you will, when it comes to deploying these these AI capabilities and deploy them everywhere, every team, every workflow, every department. Crestline advisors resisted that. They picked pursuit decisions and month end close. Two workflows, clear metrics, measurable in a reasonable time frame. And once those landed, then they expanded beyond that. They built on the the early wins, to take it further into the organization. And third, they instrumented the the pilot from day one. Three metrics, cycle time, write off rate, and partner satisfaction. So they use those to measure the success of what they were doing so that they could determine, hey. Is this something we need to consider or continue moving forward, or do we need to pivot, to something else? Having that baseline in place and having those measures that or metrics that they were measuring to see if things were gonna be successful, was that was a key to that. Now they did fall into one trap, and that is rolling out some of the stage three stuff that I'm gonna talk about here in a second, the orchestrators before the partners, before the people in the organization have built trust in the stage two insights underneath them. The result was that the orchestrator recommendations that were coming to life got ignored, because the underlying signal wasn't trusted yet. So they had to sort of walk back stage three, and really get stage two in a place of trust and then, of course, move forward in stage three. And the lesson here is maturity is sequential. Stage two trust, getting people in your organization comfortable with and trusting what the AI is producing will enable stage three adoption. If you skip ahead, and you get a higher level capability nobody uses, you know, that's gonna be a problem down the road and and really impact further adoption, within your organization. So, again, the the stages are somewhat sequential, and it's important to think about it that way. And then finally, stage three where the work the agent workforce, idea stops being a buzzword and and and really comes to life. And this is, of course, the most advanced level of maturity in the arc. And, again, the agent workforce, the digital works worker concept really stops being a buzzword, and start sort of being part of the org chart. The pattern here is specialized AI AI agents take on entire functional domains. Sub agent handles specialized tasks within each domain. Humans remain accountable, for any action that ships, decides, or commits the firm. At this maturity level, the org chart of an AI equipped consulting firm has both humans and agents on it. Start with, in this example, an engagement orchestrator. And what this is, is today, an engagement leader spends a meaningful chunk of every week chasing probably four things, schedule, scope, profit, and capacity. With an engagement orchestrator, AI watches all four continuously, runs earned value analysis, flags scope creep when time she comments don't match contracted scope, detect spread variance, suggest resource swaps when someone rolls off or a client requests changes in scope. The engagement leader, again, just like I said before, still owns every decision, but they own, again, from head of work, not behind it. That's the deepest expression, of AI in the flow of work. Doesn't replace the leader. It shifts where the leader operates from from reactive again to proactive. The accounting orchestrator is similar. In this example, it's sort of the same coordination pattern, but it applies, in this example to month end close. A revenue agent runs rev gen. An AP agent watches for, invoice to contract mismatches. A GL anomaly agent looks at twelve month trends for unusual movement. A reconciliation agent matches subledger to GL. A month end agent schedules and tracks the entire closed life cycle from beginning to end. And the CFO and the controller, see one curated, control center, if you will, not 39 reconciliation tabs. The agent really sort of handles the routine. The humans, again, handle the judgment where humans are are very, very good over AI. And this is the work that used to happen. If you think back to my example a few slides ago in in maybe ten day fire drills, maybe two week fire drills at month end. With stage three AI orchestration, it happens continuously in the background with humans pulled in only for what they actually need to decide. And, again, our hypothetical Crestline advisors going from eleven days, six days on close, that is really the lever that you're seeing, in this this stage three, the orchestrate phase. And then another, orchestrator to think about is the billing orchestrator. You know, this is sort of the cash flow example and coordinates the entire billing life cycle, pre draft, draft approval, final invoice, fee billing, time and material billing terms, retainers, performance bonuses. Sub agents check that billing fees align with contract terms. Flags anomalies again before they reach the client, and catch missing retainers, find billing rates that don't match the rate table. All of those things that took, a human a lot of time and a lot of work and sometimes were just missed because of that, can now be handled again by these AI, in this case, a billing orchestrator. And this is where cash flow lives in a consulting firm. It also where the most sometimes embarrassing mistakes happen, a wrong line item for a major client, a missed retainer balance, a a billing rate gap that takes three months to detect and ends in write off because it's too late to sort of recover. Stage three AI catches these before they cause damage or, you know, are are are irreversible down the road. The milling manager goes from chasing errors to approving a system that's already caught many of those errors. And then the last orchestrator is the proposal orchestrator. And this is probably possibly the most leverageable agents, in the consulting business development function. The idea here is an RFP comes in, the AI shreds it into requirements and questions, matches content from your past proposals or past engagement engagements, including identifying the best fit case studies and team members based on what the RFP requires. It drafts new sections in your firm's boy voice because, again, it knows your firm and it knows your data. It checks compliance against the RFP specs, and it exports a designer ready file, if you will. Designer still under the design. It's not intended to sort of replace the design capability, and the proposal manager still owns strategy and message, but the manual hours of cut and paste, of searching for things, of cross referencing, of version chasing, all of that stuff is gone. And, again, thinking back to our hypothetical Crestline advisor story, this is where proposal turnaround moved from three weeks to nine days. That's the opportunity that you have, in in the stage through the orchestration phase for a proposal orchestrator. And if you're running a consulting firm right now, this is probably the single, Most leverage agent on the maturity arc right now. We see this proposal orchestration as a very, very common thing across our customers as well. You know, win rate goes up because response quality goes up, cost of pursuit goes down because you stop burning consulting time on cut and paste, if you will. The partners who used to spend a week reviewing draft proposals can spend time, with the client instead, perhaps. Every metric in the BD function moves moves the right direction, at the same time. And and, again, that's not a small thing when it comes to the time do all of your organization spend on pursuits. So I walked through four examples, what stage three looks like, engagement accounting, billing, and proposal orchestrators, four specialized agents, each one closing loops at a higher level than stage two could on its own. But if we step back from the specifics for a moment, because I think the most interesting question isn't which agent exists, it's which firm which your firm does when those agents are sitting on your org chart, which brings me to the leadership question I want you to take home. When your firm operates with an agent workforce across delivery, across accounting, across billing, across business development, what does your operational leverage look like? Where do you redeploy the capacity you free up? More pursuits, higher touch client work, new service lines, bench utilization for development? Which agents do you trust first? Which decisions still need a human in the loop? And which ones don't? Where is that autonomy come into play? The firms that have already started answering these questions, I think, will move faster than the firms still debating whether AI is real. The firms that win won't have the most agents. They'll have the cleanest data because clean data is, again, what makes agents trustworthy and what makes those signals that we talked about, so so powerful, which brings me to the punch line. If you take one phrase out of the session, take this one. The real story isn't AI. I said this before. The real story is, what happens to your firm when AI is finally in the right place, in the right workflow, with the right context? You stop firefighting. You start managing proactively. And that's really the prize that we're after here. AI is just how you get there, and it's why those six questions that I that I talked about earlier matter more than any model release, any tech stack, any vendor announcement, any conference keynote. If it's adjacent to your work, it's wrong. If it's in the flow of your work, again, the title of this webinar, and it's grounded in your project data, secured by roles, focused on closing loop, it's really the foundation for your organization for the next decade. And before I close, I wanna put those six questions back up because in a webinar, you know, obviously, you can't grab me in the hallway after this, and and I think these are the questions you need to think about come Monday morning. So if you think about these three questions after everything we just talked about, is it embedded? Is it role secured? Is it traceable? Is it your date? Is it human in the loop? Is it measurable? Again, all of those things are things you should be thinking about as you start to head down that maturity arc, as you evaluate AI tools, as you stew pilots and proof of concept for AI capabilities in your business. Keep these six questions in mind because they will point you in the right direction, when it comes to implementing these in the flow of your work. So what do you do come tomorrow morning, Monday morning? Three moves, not 10, not a transformational program. Three moves I think you can start right away. First, pick two workflows. Don't boil the ocean. Again, pick two workflows where you have, what you think is, inefficient cycle time or there's a high risk for your firm. Find those and determine, you know, what happens if that was 30% faster or 30% less risky? What would change, within your organization or your operating rhythm as a result of that? And improve the signals before you scale. Clean up your data. Garbage in, confident garbage out, that's not what that you what you want. You want clean data in so that you get the best signals out, and then run a pilot loop or a proof of concept. Pick those metrics that I talk about, pick a specific, duration of time, run a pilot, learn from it, and build on it from there. Those are the the the three moves that that you should be thinking about come Monday morning as you look to mature AI in the flow of your work. And then I started with. a bet. I do. leave with a bet. you, Brett. You know, here's the bet I think the audience a quick were sitting where you're sitting, submit that win the next twenty four months. us. Again, as I said before, while we do, they won't have the most AI. everybody They'll have the cleanest project in the business of clearest project success close. for have made the the boring decisions about data discipline. role security enable humans stay in the loop, to maximize made sort of the exciting decisions about which models to deploy. through integrated that's true whether you're PSA Delta. customer, somebody else's to visit, our website for more information. your own. And the firms that bring evaluation like to everything us reach about to, you, the six question checklist, the three stages of the maturity the, poll, will pull, ahead of firms that don't. we will in the flow of work isn't an IT. decision. It's a partner level operating decision, and the partners who lead it from the front will define what their firms look like, Great. in the AI era. Alright. Alright. I know we only have a a couple minutes gonna pop, over into the q and a. here, Brett, and we'll kick it off with, some questions just have a couple, questions since we're limited submitted. on time here. Jen, How do have get questions you wanna throw my way? is genuinely a mess? K. That's a great question. And I think and it's a question that I hear with some frequency because, you know, data evolves over time and piles up. And if you're not as disciplined with it, it can become a mess. I think there's probably a few things that I would recommend. First of all, I think any organization needs to inventory their data. As I talk to leaders and and whatnot in organizations, I find that a lot of, people don't even know where data is throughout their organization, because, it could be, like I said before, it could be in documents, it could be in SharePoint, it could be in, you know, a multitude of places. So I think step. one is taking, Brett. an inventory of that data. jump into step one other question that we have, and. then we will wrap up for the day. the reason I say that is because are some of the new risks, from AI? value in, curating data that's old, that's out of date, that's inaccurate, that type of thing. So eliminate. the stuff, the data that just isn't relevant anymore to how your firm operates and and where your firm is going. And then the third thing I'd recommend is you gotta have day data stewards steward or stewards, in your organization. Assign someone a a a role very deliberately with clear responsibilities after you've done that inventory and say, okay, John Doe, you're responsible for this area of the of the company in terms of just being disciplined with this data, getting it cleaned up and getting into a good state. And Jane Doe, you're responsible for this area, and let them go to work. I think, you know, those three pieces are a great place to start because you have to be super deliberate about it. This is never gonna happen organically. It it it's something that has to happen, you know, in a very deliberate way. Oh, boy. We could probably spend a whole webinar on the risks, but I think I touched upon a few of them. I'll I'll just I'll just reiterate and highlight some. You know, I think, sort of the the the garbage in, confident garbage out is probably one of the biggest risk, and this has been characterized in many different ways, hallucinations, noise, however you wanna describe it. I do think it's one of the the biggest risks because, like I said, we've heard horror stories. We've heard about, consulting organizations use, you know, verbatim. out of, AI, thank you so much, Brett. information questions we didn't called to, on it. we will be sure to, answer offline. you know, I'll share those directly with you, AI. and being called on it by a Before Those wrap up, stories are out there. So I think, extend a special one of the biggest risk, thank you think you, is, one of the reason incredibly data just discipline and whatnot, today. We really is so the expertise And then I think just sort of the the governance shared as we explored how firms are applying things are critical as well, without that's where. a big risk is. You gotta know where your data is going. forget gotta know how it's being used. oh, thank gotta. know what AI tools are being deployed you'll receive an on demand recording of today's webinar via email within twenty four hours. tools have so that please check out the resources client comes to you and says, hey. in you know how my data is being the? doc center can answer that with. conviction and with, confidence, we'd appreciate it if you could share your feedback and complete the short survey the market that you've considered webinar. that risk, And you have a, plan in place to thank you all for joining us today. risk. So I think that compliance piece and then, of course, for more said, the hallucination events. garbage in, confident garbage out are probably. the two big risks biggest risks, I would mention here in in a in a short answer.