Transcript for "Research Questions AI Can't Answer": Hello, everybody. Thank you for joining us this afternoon, for the next hour. We are so pumped to be unpacking research requests that AI can't answer. But before we get started, I do wanna just share a few quick housekeeping notes, which are on the slide here so you can follow along with me. Please join today's webinar through Google Chrome or Firefox for the best experience. Audio is gonna come through your speakers so you should be able to hear me now that way. And if not, there is no no need to dial in or any phone number to call into. We do have the slides as well as additional resources from the docs tab to the right of the slides here so you can get today's slides and additional resources, related to today's content as well as upcoming content that you'll want to make sure is on your radar as we get into the end of the fiscal year and get into top up season. If if you're familiar with Deltek and GovWind IQ, we have a big campaign coming up in October to unpack those. During today's session, we have the opportunity to have q and a, submitted at any time. We are likely gonna be following up, just by nature of how much content we have over the next hour, follow-up with after with q and a. But, if we do have time to take a few questions live, we'll see those there. And then lastly, if you have to drop or join early, you'll get a recording of today's webinar within twenty four hours. So with that, I'm going to move us along and remind everyone what webinar you signed up for probably a few weeks ago. At this point, we're gonna be talking about research in the public sector and specifically how public data with AI is not always as trustworthy as it sounds. We're gonna really unpack what questions AI can't answer, and we have a dynamic panel we're getting into, for the second half of today's presentation. Quick agenda just to let you know what the arc of the next hour is gonna look like. We're gonna unpack market conditions. Why does this topic matter most now. We're gonna then talk about AI on public data and what are some of the limitations. We'll also talk to a few things it does really well, and then you're gonna hear from experts, at GovWin. We have five different experts joining us, each with stories from the field, from their own experience, which will help give some color to to this topic. And without further ado, let's get into it. So I think it's really important to talk about the why now because the context this year matters more than it usually does. You know, the conditions government contractors are competing in heading into this year and next look fundamentally different across every market segment. On the Fed side, we're seeing Doge driven contract terminations which eliminated billions in contract value. GSA consolidation is concentrating spending through fewer and larger vehicles. Procurement reform like the FAR overhaul, OTA expansion, commercial first mandates, etc. Are all moving fast and in multiple directions simultaneously. In SLED, agencies are adapting to reduce federal funding with infrastructure and public safety still remaining some of the most stable pockets of demand and the SLED contracting data actually shows a decline in bid and RFP volume but when analyzed alongside spending trends, we're actually seeing they aren't buying less but rather they're buying differently. Much like Fed, they are shifting towards fewer, larger and more consolidated procurements with total purchasing dollars continuing to grow year over year. And then in Canada, their defense and infrastructure build out is reshaping how BD teams think about growth against shifting U. S. Trade relationships. Their strategy is changing more to a build partner buy framework which prioritizes domestic capabilities first, allied partnerships second, and foreign procurement last and this means for U. S. Based contractors, the path to relevance runs primarily through that partner lane which changes their playbook and what we're seeing is in each of these markets, the same challenge is showing up. BD teams are competing harder for fewer opportunities with processes built for a market that no longer exists. And so I think about this in terms of, like, a traditional BD playbook where it was written for a more predictable environment. Stable agency structures, known procurement vehicles, and historical spending patterns that you could model against or forecast against, and that playbook is being rewritten right now. AI adoption is a rational response to teams doing more with less and I can speak personally to that. I know we're encouraged to use AI here at ZelleTech, all the time to, you know, keep up with the market and the demands that happen in a very complex marketplace. But here's the part that matters for how you should think about these tools. In a market this volatile, AI running on historical public data that hasn't been vetted, contextualized, or enriched can generate confident sounding answers about agency priorities, your competitive landscape, and procurement history that simply doesn't actually reflect the current reality and the future may look nothing like that historical record that the AI is modeling against. So this is the crux of today's conversation. We're gonna be unpacking this a lot more. But before we do, I wanna give AI its due. I don't want you to think I'm anti AI. I use it every single day. But coming away from this webinar, I want everyone to know what some of the limits are when working with public data and I put that in air quotes because public data is so, one, there's so many sources. There's especially when you get into SLED, you know, it gets extraordinarily high very quickly and there's not a single source of truth that makes these growth decisions easy to navigate. And so, you know, AI is fast at what it's good at, reading volume, summarizing routine documents, and drafting that first pass. And we learned through our annual industry study called CLARITY, we've been doing this for seventeen years. So the data I'm going to reference all came out of that. If you're interested type CLARITY into the Q and A box and we can make sure you get a copy of that. But from this year's report, we learned that 90% of contractors are using AI this year. That is actually up from only 45% last year. So adoption is through the roof. People are trying to figure out how to bring this into their workflows and are investing in tools like ChatGPT or Claude or licenses for their whole business because there's this pressure to keep up. And we've learned what excel what AI excels at rather. Surfacing that public data fast, it's pretty good at retrieval as long as it can find it. It's great at reading dense documents and serving up citable answers from the text it's pulling from so long as it knows to give you those citations. First pass drafting, we also saw that the number one way BD teams are using AI is actually for some of the most time incented time intensive parts of the BD workflow, which is around proposal drafting and shredding RFP documents. But we know, right, if you've used AI, it's a great ideator. It's great at brainstorming with you or if you give it a really rough chunk of text, it does a great job of massaging that into a certain tone or, against a certain library of historical data. And so if you have content as a baseline, it can often help you articulate what you wanna get across with less time spent on edits. And then pattern spotting at volume. So recognition and retrieval is where this technology really shines. It can cut ramp time for someone less familiar with the data or history and if we think about turnover in this space, right, you know, we know people bounce around from organization to organization and you have to learn, you know, what your firm does well so that you can market that and bring that to the government customer. This is a great way to kind of cut ramp time and make sure people know what your award history is or where your sweet spots are, and can help people get caught up that way. Now where this gets tricky is when AI is guiding business decisions that sound confident. It doesn't maybe flag a limitation or the gap in the data and potentially hallucinates, right? We know this happens. We've all been probably in meetings where you ask someone about something on a slide that Claude generated and they're like, oh, I didn't catch that. So that means it's making these things up sometimes despite the instructions that it's been given. And a quick example, this actually came from our VP of research. She had her team using AI to go through a dense document. And when they went back to verify, some of the questions they were interrogating the document against, there were a lot of errors. And they asked for sources and found that the model didn't actually read the 100 plus page PDF that was uploaded to the chat, but instead found articles online, you know, essentially cliff notes of the article and use that for the analysis it was prompted to do, which is terrifying if you think you're, you know, uploading a PDF, of course it's going to read every line of that. This proved that that was not necessarily the case. And so also from clarity, we learned only 25% of firms have established or advanced governance over AI in their business, and this is where a competitive edge is making a gap in the market. We know AI gives us speed, but what it doesn't give you on its own is clarity and control. That comes from using the tools with safeguards in place, AKA governance, which often include keeping humans in the loops, check ins during the workflow to validate before going on, escalation expectations if issues arise, what is it allowed to act on and which things should it flag to you, to know the bounds of the data you're working with and basing your future on. Okay. So we already talked about what AI is good at. Unfortunately, the research that helps BD teams win isn't already written down in a single location, clearly connected to relevant content that can provide more nuance or easy to understand if you aren't familiar with what you're looking at. And it's the details like who's already talked to the agency, what's really driving a program's timeline, whether an a listed opportunity is even real, and public data in general AI tools, I will use this term throughout today's webinar, but I mean, ChatGPTs or Claude's or Gemini's when I say general or generic AI can only answer questions from data they can find and then it makes assumptions without showing that vulnerability to the end user. So if we think about pre solicitation intelligence and you ask a generic AI tool, you know, who's likely to win this contract before the RFP posts, it has nothing really to work from. There's no public record yet, but the agency conversations that shape requirements are happening during this period before it hits the street. And that's the window where positioning happens. So if you're only asking AI to summarize the solicitation once it's public, you're already asking questions too late. And that's not an AI problem. That's a fundamental workflow change. Research that goes beyond public listings. So what's this agency planning to procure next year? Often can't be easily answered from scraped or indexed data, and that's because the answer lives again back in, like, a conversation more so than a document. AI can summarize a database or analyze a budget, but it can't call the program office and ask what their pains are or how their missions are changing and what they'll need to look at this year. Then we get into validated intelligence and so this is a quieter problem because AI will answer very confidently. It wants to be your best friend. It wants to be a people pleaser. It wants to know that you're, you know, build you up even if you have a silly question. And when the underlying opportunity is stale, canceled, or missing, a real point of contact, it doesn't know what it doesn't know. It ask it whether recompete is still live and it often just restate the last thing it found indexed, not necessarily what's true today. And then the last piece here is AI is really only as good as the data behind it, and this is kind of the foundation that we're trying to hit on. Right? Is AI doesn't fail by refusing to answer. It fails actually by answering anyway. Confidence in the response has nothing to do with confidence in the underlying data, and that's the gap this whole conversation is about, really knowing which questions your AI tool is equipped to answer and which ones it's just guessing at. Alright. So, you know, it's important that we have this foundation because AI will process whatever it can find and hand you that fluent, confident sounding answer and without, you know, clear guardrails to flag which parts of that answer are solid or which parts are stale or wrong or, you know, maybe have some some boundaries. You you are making these decisions with sources that quickly get outdated, or have a limited reach to, you know, how much is actually publicly out there. A quick example is two years ago, scan for expiring contracts was actually a pretty reasonable proxy to build a pipeline from. Today, with how much work is getting bundled or shifted to a different vehicle or quietly canceled faster than public data can track so that the expiration date alone no longer is something that you can bank on a future real opportunity existing because of the nature of, you know, how how the government is changing what they're buying. And so when a tool actually surfaces a phantom lead, that leaves your team to disprove it and the tool's job's done. But your team is gonna spend hours, you know, really working that lead, understanding if this is something they need to put resources behind. And as we know, right, BD is a relationship driven role. So any time taken away from the work that wins actually cuts into your competitive in a market, competitiveness in a market that's never had a more competitive pool of, you know, potential business. Every at bat matters more just due to the nature of opportunities there are to win that business. Alright. So let's talk about investments in this space, because paying more really only matters if the extra spend is buying something the free version can't get you. We know, right, paid tools that scrape stand.gov or state and local procurement sites quicker and display it more cleanly is not actually giving you an earlier signal. It's the same late stage signal in a better interface. And so by the time something posts publicly, the agency has usually been scoping those requirements and talking to vendors for months. And that head start is what a subscription should be buying rather than that nicer dashboard. And I think it's also important to understand, you know, when do you actually need to make that investment. If you're a smaller firm doing occasional narrow market work, you may not need to pay for depth at this point. You likely can still find what you need from free sites and and using the tools available not at an enterprise or, investment level. And so this isn't, again, to argue necessarily totally against free tools, but knowing which ones are gonna actually help you meet your growth goals depending on what you're trying to accomplish. So before we get into the panel, which is my next slide, and we'll spend the majority of today's presentation, I wanted to bring up a few questions that are worth asking. As you, you know, are thinking about intelligence as a service, if you're look if you're a GovWind customer, these are things that you can rest assured we do. But for you, I think the panel will be another great opportunity to hear how you can leverage our research support and talk to our experts. But before we get there, a few questions just to, you know, bring into if you're, you know, considering other platforms or looking in this space. Each one really exposes a different way, confident answer confident answer and correct answer can diverge. And so understanding where the data comes from, scraped public data and analyst verified data can produce similar looking answer with very different reliability behind it. The question isn't whether a platform has data, it's whether anyone's checked it before it reached you so that you can act with confidence going forward. Do analysts or staff engage directly with agencies? This is the difference between summarizing a website and having an actual conversation with the program office. Right? We know scraping tells us what was posted, but engagement tells you what's actually happening which is often not the same thing. How are your recompetes identified and qualified? An expiration date on a contract is not a prediction. It's a guess dressed up as one, a platform that flags, the reef competes off dates alone will surface a lot of noise versus someone something that has someone come come come for confirming, excuse me, the opportunity and point of contract wouldn't. Who validates the AI flagged leads? This one circles back to today's question. If the answer is no one, the out AI's output output is the final word. That's the exact failure model we've been talking about, you know, confidence sounding output for verified information. And if a human checks it before it reaches your team, that's the difference between maybe a lead and more of something that needs to be further verified. None of these questions, are about a single platform. They're just a way to bring this into your research as you're looking for, again, what's going to help you get to where you're trying to go. Alright. Without further ado, let's meet our experts. You're about to hear from five people whose job is exactly what we're trying to talk about, that gap between what's public and what's true. We're gonna have our analysts represented from across the entire IQ business who spend their weeks on data hygiene, our data, our analyst tracked opportunities. So these are the leads in GovWin that our analysts monitor really cradle to grave an app. And what that means is they call directly into the government for details to share with our users. And then lastly, research support requests that come from our customers. So this is where users ask real research questions, and we work each request for up to two hours for free and deliver what we find back to that user tailored to their use case or business depending on the nature of that request. So these are the people that validate, contextualize, and enrich the data in GovWinIQ and help us bring industry tuned AI on top of that data to every stage of the BD life cycle. So let's introduce today's panel. All right. We're going to kind of round robin and we'll be the structure of today, but we're gonna ask everyone to share how long they've been in the GovCon world, what their a day in their life, looks like just to kind of paint that picture and then a couple of stories that they've prepared to share. So to get us started, we're actually gonna start with our data analysts, Jen Sikole and Christa Bali. So, Jen, I'm gonna ask you a few questions first and and then we'll get to Christa. So, Jen, if you don't mind, you know, give us a quick introduction. I know you're actually represent representing someone else from your team that had a conflict for today's webinar that we wanted to make sure her stories made it to today. So I'll pass the mic over to you. Great. Thanks so much, Logan, and hi, everyone. Thank you for hopping on the call today to spend some time with us and and learn what we do. My name is Jen Secole, and I'm a director on Deltek's federal research team for GovPointIQ. I've been in the industry for a very, very, very long time, since before the turn of the century. So I'll leave it at that. What I do on a day to day basis is I lead analysts who manage our federal government products. So things like contract awards, spending analytics, task orders, agency profiles, and org charts, plus products that span federal, SLED, and Canadian markets, like company profiles and smart tags. And then rounding out our team is federal advanced research, where our lead analyst, Jackie, handles complex data requests for our federal clients. So as Logan mentioned, Jackie isn't available for today's sessions, but these stories are hers, and I'm very happy to share them with you on her behalf. So many of our clients rely on Jackie for recurring and nuanced research support. And while some of their questions can sound, you know, relatively straightforward at first, anyone who works in the government contracting market knows that data often becomes much more complicated once you start digging into it. So as Logan mentioned, AI can be incredibly useful in accelerating research, in helping us get into information faster or finding information faster. Where experienced analyst support becomes especially valuable is an understanding what that information actually represents. So recognizing anomalies, knowing where data may be incomplete or inconsistent, bringing together multiple sources, into a response and ultimately interpreting the resultant data in the context of the question that the client is trying to answer. So, one of the examples, for today involves GSA's multiple award schedule. A client came to us because they were trying to identify GSA MAS obligations at the special item number or SIM level, and we're struggling to understand, you know, what was reliably recorded. So on the surface, it sounds like a clear cut data request, identify the SIN and report the associated obligations. But the challenge is that GMS MAS reporting doesn't work that way. So, SIN level obligation information is not available on federal contract awards data. And GSA doesn't provide a way to isolate task orders by specific SIN. And Jack Jackie recognized that immediately. So rather than forcing the data into the answer that the client initially expected, she first explained the reporting limitations. She then developed a report using the data that could be reliably supported. She provided guidance on how the client could interpret those results, and she even included examples of some of the common ways the information could be misunderstood. It really was her explanation that provided value to the client. You know, it wasn't just like, here's the data you requested. It was more, here is what the data can tell you, here's what it cannot tell you, and here's how you can use it. And that mattered because the client was using the information for business development planning, and Jackie's explanation helped them avoid making market assumptions that the source data couldn't support. So after receiving the analysis, the client did send us a note, that said, you know, tell the analyst, thank you so much. You saved us countless hours of scratching our heads. And I, personally, I love that feedback because Jackie didn't just save them time pulling data. She helped them know whether they could trust it as well as how to use it. So a second example is kind of along the same theme. It involves OASIS and OASIS Plus vehicles. And we regularly have clients that ask us to combine historic OASIS data with OASIS Plus obligations and break that information down by attributes such as pools, domains, and cleanse. And again, you know, that that request sounds somewhat simple until you really get into the data. So those attributes are not, you know, consistently or formally reported in the federal contract awards. But because Jackie has researched these vehicles extensively, she knows where the data is likely to be incomplete and what data needs additional verification. She knows to draw on GSA reporting as well as her prior research to fill in those gaps and really build a complete picture. You know, specifically, an example would be, you know, if you were to look for contracts that were just labeled as either OASIS or OASIS plus, you know, that doesn't necessarily give you, the complete historical picture because there are additional vehicles like HCaTS and FSSI BMO that are transitioning into OASIS Plus. So while that's a relatively small detail in terms of the mechanics of a resultant report, it can have significant impact on the business conclusion. You know, our clients are often seeking this type of information to size markets, understand their competitors, and make, business pursuit decisions. So if part of that history is missing, the resulting market assessment can be wrong. And I think both of these examples illustrate where analyst expertise and AI really complement one another. Like I said, AI can help us find the information, organize it really, really well and accelerate parts of the research process. But government contract dating is complex and sometimes the most important insight is recognizing that the apparent answer isn't actually the right answer. And again, that's where our analysts add value. They understand the underlying sources. They recognize what's missing or inconsistent. They bring historical context that may not be obvious from the current data alone. And most importantly, they're really working to help the client translate that information into something that they can confidently use to make business decisions. Back to you, Logan. Thanks, Jen. I think those are two excellent, you know, use cases and and examples to share because they are so relevant. I as you were, you know, one talking about your experience, you know, I don't know if everyone's familiar, but we do have a team of a 150 analysts here with I think we've done the math where, altogether, it's, like, over seven hundred and fifty years experience in the industry, which if you think about, you know, pairing that from, like, an expert perspective with, you know, everyone is using AI. We all kind of think about it as a really good intern. That's a pretty powerful duo, but, you know, it definitely helps to have someone that's familiar with the data, which I also think is super relevant in this field knowing, you know, there's people out there from a tech literacy perspective that are building their own, you know, scraping tools and MCPs to bring public data to chat GPT. We also have people that are, like, just learning what hallucin hallucinations are. And so that bright wide, you know, breadth of experience as well as just getting your arms around the different datasets in the market can be pretty intimidating, at the speed at which you might need to to know, you know, as you go forward in today's market. With that, I'm gonna pass the mic over to Christa. And I know you've got a couple other stories that will add to what Jen mentioned. Hi. Yes. Thank you, Logan. Good afternoon, everyone. I'm Krista Buie, a senior research analyst at Deltek where I have spent the last nine years supporting GovWinds Federal Org Chart product. I am one of three analysts on the team where I personally cover DHS, Army, Air Force, Defense, Justice, State, Executive Office of the President, and a few other independent agencies. Our Federal Orchard product helps our customers understand the complex reporting structures within the federal landscape and identify key decision makers within their agencies of interest. So each day, I monitor media coverage and government headlines for developments relevant to my agencies all while managing my portfolio through a structured for group review cycle. This review cycle ensures consistent leadership and organizational verification and, And of course, this supplemented by ongoing updates whenever leadership changes occur, which is daily, and reorgs are discovered or if a client submits a request. Our verification process, it uses a combination of primary and secondary research where primary research is direct outreach to government offices to confirm specific contact profile information and secondary research comprises comprises sources such as official agency websites, directories, government documentation, trusted news and professional platforms like LinkedIn. And all of our government, our government clients that have the federal org chart subscription have access to our team's full support with the ability to submit requests for us to prioritize research and update and that's another core part of me and my team's role. Together, our team maintains a dataset of over 4,000 federal org charts and 14,000 federal org chart contacts. So, today, I have two stories that reinforce the importance of me and where AI falls short. So about five months ago I had a client that needed a comprehensive time sensitive list of every federal agency chief information officer and chief AI officer complete with email addresses and phone numbers. So of course the client tried to build out this list using chat but found the output slow to validate and of course it was missing some significant contact information. So I began by extracting a filtered list from our GovWind Federal Contacts database using targeted parameters and this produced a foundation of about 200 plus federal IT contacts. I then took this list and cross referenced it against multiple government and industry sources confirming the accuracy of more than 80 leaders but identifying 20 profiles that required updates. So working collaboratively with the team, we corrected the identified profiles and delivered the requested list of leaders within three business days. In total, I think we validated about over four, over a 100, chief information officers and chief AI officers for current role accuracy and usable contact information addressing the gaps the client found with their preliminary AI leadership list. So, with this example, we see that AI tools can surface names from public sources and organize them in pretty little lists but they can't always confirm contact information or whether someone is still in that role. It can update profiles in real time or apply domain expertise to judge which sources are reliable in a fast changing environment like federal IT leadership. So, what I delivered, it just wasn't a list, it was a vetted ready to use outreach tool that saved the client hours of follow-up research and gave them the confidence in the data behind the business development efforts. And so another quick hit example which I realize is not specifically a research request from a client but it's something that occurs in my daily update routine. It's with, a key leadership change within US Transportation Command. And so as a part of my my tracking efforts each morning, I identified a SES promotion of the Transcom Deputy Director of Strategic Plans, Policy, Logistics and Warfighting Development. It's quite the lengthy title but in short, it's the J-four, J-five and J-seven contact. So I pulled my R chart to ensure I had the correct individual and update his promotion accordingly. However, while reviewing the chart, I noticed the contact was incorrect and there was a title discrepancy from existing leadership. The existing title that I had on the org chart only including the j only included the four the j four and the j five responsibility where the new source documents the extension of the j seven. So which was accurate? Well, I turned to AI for help in gathering evidence of this change but the source it returned contained outdated j four, j five title and the previous deputy's name. So only by digging deeper into other dated sources beyond what AI initially surfaced, was I able to confirm the correct new leader and update the title to reflect the expanded J7 assignment. So, this is an example where the AI tool fell short and I actually come across these specific discrepancies pretty often in my updates. AI tends to scan headlines really skillfully and I've had a lot of success on how AI compiles and clearly organizes well documented reorgs. However, it can miss details that are newly implemented like this example or can miss details that are obscure in source documents. I've observed this pattern repeatedly. I've also found that AI often misses employment start and end dates on LinkedIn on LinkedIn profiles. It doesn't read it. It doesn't see it. So it might tell me if somebody's in that role when really they've left that role three months ago. I've also noticed that it's less reliable at tracking civilian or lower level agency leadership changes compared to those high profile military leaders. And several times, I followed the link an AI tool offered as the source for its output only to find it didn't actually back up a claim. So, the gap between the AI generated result and the information in its own source is exactly why validation cannot be overlooked. So, in closing, AI can find and organize and summarize the data, but it takes a researcher to know what's missing, what's wrong, and what's worth trusting. That's the difference I personally bring to every chart, every contact, and every update and probably why our clients keep coming back to a person and not just a search result. Thank you for your time today and I appreciate being on the panel and learning from my my coworkers as well. Logan, back to you. Thanks, Krista. And I love that you brought up LinkedIn because I think, like, as we think about contact information, like, that's something I would assume AI would be pretty good at scanning or, you know, looking at titles. And, you know, I I had no idea that it maybe doesn't catch every promotion update, so that's that's good to know. Alright. We are going to change, research types now. We're gonna invite Sherry, Kalvendra to talk about her, experience as an analyst tracked ops, researcher and share a couple more stories to, show how, you know, the opportunities that we have people really following throughout the process, you know, how how having a human in the loop there, you know, just to come up with a comparison really does make a difference than scraped data. So, Sherry, I'll pass the mic over to you if you wanna give a quick introduction and get started. Yeah. Thanks, Logan. So my name is Sherry Calvendra, and I manage a team of research analysts that are responsible for finding and tracking SLED government contracting opportunities. So, you know, these are the folks that are researching and vetting and updating our gov wind tracked opportunity records through the entire contracting evolution, from a government having a need to a formal procurement and then on to a contract award. And we've been doing this for years now. Myself, personally, I've been working in the SLED contracting space for over twenty years. Government folks know who we are. They know Deltek. They know what we do, and that's because we engage with them every single day. Every day, we're digging for information. We're asking questions, trying to get validation on what's happening in government contracting. I mean, in some cases, we've had government employees proactively reach out to us to make sure that we have information. And I just think that that's just a great reflection of the reputation of our analysts and and what we've built with government. Right? So the two examples that I wanted to share today from my team, I think, really accentuate that value in in highlighting the significance of our analysts that are out there investigating and probing for information. And, you know, on the flip side, may show where the AI tools can fall a little short in that area. So for my first example, we were looking at a project that we were tracking in Govind IQ and comparing what we knew about it to what, like, an AI assistant, an AI tool provided about the project. So we found that a school district was planning for an ERP solution, about ten months before the solicitation had come out. We had added it to the GovWinIQ database as a potential upcoming procurement, and we were continuing to monitor it. And as we were doing so, we were capturing all the materials, to update that opportunity record. So, you know, we got the solicitation, all the amendments, the, who who was who was being selected for the demonstration, all the demonstration materials. They had updated their budget for the project. We captured meeting agendas and notes, all of those materials to give us updates on the project and also so we could, you know, ask our government contact intelligent questions about the procurement so that we were in the know and we could speak, you know, to them about it. And we compared that to the AI output about the project. And in this, particular situation, it wasn't simply like, oh, hey. We found out about this project before AI did. It's it's we did. But the AI tool didn't actually have access to the same info that we had throughout, you know, our manual searching, throughout our outreach. Yes, the AI tool was able to point out that there was indeed a project, an active project, and there were board documents that referenced the project and, where those documents were housed online, but in this particular instance, it couldn't actually access the information within those documents. And there could be, you know, various reasons for that. Maybe the site had restricted it or maybe it was the way that the information was stored in the documents. Whatever the reason is, my point is that it didn't have the info that our analysts were able to obtain. And that, you know, it limits the vendor's ability to see the full picture when they don't have all of that information. Alright? So my second example is we were asked by a client about a state rebid. They had found some information about, a potential new reporting system. So we searched to make sure that it there wasn't already a current active solicitation, in our database or there wasn't an RFI out there and we didn't we didn't find anything. So we started scouring through their planning documents, their program documents, all kinds of documents, the road map. We had three different analysts looking into it, and there was no mention of any plans for a new system, an upgrade, enhancements, or that there were any issues that might, you know, preclude that. There was nothing indicating any intention for a new system because we always look for something tangible that we can reference, something that we can have a real conversation around, not just something like, you know, we heard this from a little birdie. Because in this particular instance, what the client had provided us looked a little suspect to us, because we had found out that the department had already implemented a new system a couple years ago. And we know that budgets aren't really allowing states to replace their systems every other year. So we gathered up what we did find, and what we were able to confirm and we provided that back to the client and come to find out on their end the info that they had originally sent to us was AI produced. And it was very confident sounding. It had timelines and milestones and everything, but it was speculative. It wasn't authenticated. So again, I and it was appeal. I mean, so again, I just wanted to point out that as part of our sourcing standards, our analysts are looking for the clues. Right? They're looking at what is driving a need, you know, what indicators do we have that something is going to happen, and then we we think about it. We use our judgment. Does this make sense? Is this sound information? Is it current? Is it real? Is it actionable? And the analysts are using their critical thinking skills, their research skills, their market expertise, and ultimately the government outreach to qualify the information to make sure that it's credible, and that we aren't just relying on something that AI surface because we've seen that AI can misinterpret those like one time projects as rebids. It can assume that, like an expiring contract is going to be recompeted even when that project was won and done and they don't need to keep rebuilding the same system again and again every two years. So hopefully this is saving that client potentially chasing after something that was fabricated. So with that, I'll wrap it up and I'll pass it back to you, Logan. Thanks, Sherry. Yeah. And I think that's super relevant. You know, we talked about the compounding risk of, you know, make you know, following these false leads, really. being on the onus of your own team rather than on the tool that you're using. And, you know, it's not the tools problem. You're gonna figure it out one way or the other. And it might you know, I can only imagine if, you know, if say a BD team were in your shoes instead and three BD folk were, you know, busy trying to find this. Like, that's that's a lot of work, that takes away from field stuff and things that only people can do. So, really, really great examples there. Let's switch over to our last, two researchers. We're gonna unpack actual, research support requests we've gotten from customers. So, Ali, I'm gonna ask you to take the floor first, and I'll pass the mic over to you to introduce yourself. Sounds good. Thanks, Logan. My name is Alexandra Ellison, and I'm a senior manager for our FOIA and ATIP services and our SLED research support teams. I personally have been in the industry for over fifteen years now, and I have a tenured team of analysts that work on my team as well. My team handles client questions, regarding bids, contracts, lead alerts, and much more. Even not, things that are not tied directly to content in our database. We also manage the FOIA tool in The US and the ATIP tool on the Canadian side of things. So some of the previous panelists have mentioned, AI when used appropriately is a great tool in our toolbox. It really drives process efficiencies, cuts down on manual lift so that our team can spend more time on the analysis that actually matters to you. But the value of what our team brings goes far beyond what AI can touch. Through FOIA research FOIA research requests, public records requests, and even just direct conversations with government points of contact, we capture information that's never publicly posted. AI and scraping bots can only work with the data that already exists online, but we take it a step further to bring you the complete picture. I have two examples that I can share. I was on a call with the CSM and a client who was interested in a specific solicitation. We had already pulled the publicly posted document in govlin, and she had read through it, but the agency was just very vague about what exactly they were wanting. They kept it really high level, and she just couldn't tell what they were trying to accomplish or whether it was worth her time to bid on it. So the CSM kind of stepped in and asked her, if you were sitting in the room with the decision maker, what would you wanna ask them? So she thought about it and came up with about five questions. And our research team was able to track down that decision maker and ask those questions on her behalf completely anonymously. So she wasn't bothering them. She wasn't asking them, like, during the procurement process, hindering them asking these questions. She passed it on to us. We were able to completely anonymously ask those questions and take the answers back to her. So when we gave those answers back to her, she walked away with real understanding of the project scope, and then she was ready to bid with confidence. And so that's the difference between a real analyst making a real connection and AI just guessing at what the agency might want. The second example is a client came to us and they were wanting to use our FOIA services. She was after some documents that were not publicly available, but she'd just seen referenced online. She'd seen that this contract existed, and she had seen that these services were being covered, but that was all the information she had. So she searched everywhere online, came up empty. And so she decided to turn to the Deltek FOIA tool, which is great. However, she was still missing key identifying details like the contract number, the exact title of the project, even the vendor name. And so without any of those, we weren't able to submit the request. But we were able to connect her with our research support team who are able to track down that information. And then she was able to move forward with her FOIA request and get those documents that she was after. So that kind of cross team collaboration isn't rare for us. It's how we work on a daily basis. And for FOIA specifically, we help clients request all kinds of document suites completely anonymously, and then we house them in our GovWind document library for all of our clients to access. So we have decades of FOIA documents that were never publicly posted but are available for you to download right now in the GovWind document library. As always, our analysts are here to help you identify information and documents that truly matter and connect them to the projects that you're working on. Back to you, Logan. Thanks, Ali. And I so glad you picked, one of our most popular requests, which is that you mentioned, well, reach out to the government anonymously on on folks' behalf because every time we go to Elevate, I have customers come up to me and say they use that all the time, just to, you know, stay in favor with the CEO or the procurement officer, and not seem needy after you realize, oh my gosh, I forgot to add this question into my list initially, you know, us going on their behalf and getting that answer for them is a huge value. Next and certainly not least, Niko, we are gonna turn it over to you to wrap us up and then I'll close out today's conversation. Sure. Thanks, Logan. Hey, everyone. I'm Nico Fisher. I've been in the industry since 2007, and I'm a manager in federal research. I have responsibility for federal contract awards, task order awards, spending analytics, and federal technology install base, as well as some of the internal systems, that are behind our data, like our federal gov entity system. So today, I wanna share with you some of the proactive steps we take to stay on top of the market as it evolves. So at Gov one IQ, much of our content is attributed to organizations in the federal government. The buying and contracting organizations you see on contracts and task orders, the buying organization listed, on opportunity reports, and the organizations that our contacts are associated with are all examples of this system or attribution. So underneath, underneath all these different products, we have over 7,000 government entities in our federal structure. There are other systems that we then have that are connected to our federal govern government entity structure, like the systems, we developed that attribute reported spending dollars to those federal entities or organizations. When an agency goes through major organizational changes, which has happened, a lot in the last few years, We pursue evidence of the updated structure, and then we map out the changes necessary to ensure content ends up in the right place. So a recent example of this is the changes the army announced late last year to merge the Army Futures Command with the Army Training and Doctrine Command or TRADOC into the new Transformation and Training Command or T2COM. Through research, we determined how to build out the new structure of Army T2COM and how to merge our existing structures together to form it. This involved around 40 changes we had to make, merging, moving, editing, and adding, different government entities. My team makes a map of the changes we plan to make and communicates them with analysts across the other products to serve as a final point of review. So it's important for us to be proactive about these, gov entity changes as it takes years for these changes to be reflected in the public spending data. And proactive, being proactive means we can reflect the updated structure while keeping all of our existing content where it belongs. The second real example I wanna talk about today is a state department anomaly we found. So in addition to ensuring that we are always obtaining all the new spending data, on my team, we run two weekly audits of the FPDS or now, sam.gov contract awards, essentially, the federal spending data. The first is looking at the data in aggregate across the most recent four fiscal years. This will alert us if there is unusually high or unusually low spending changes week to week. It also serves as a time we can compare the data we have to the data on external government sites. The second audit we run has more detail in it. In the second audit, we're looking at spending by buying organizations, which is tied again to our federal government entity hierarchy, to see the week's new transactions. This is worthwhile because we can spot when a particular organization has major changes week to week. In doing these audits, we discovered a major anomaly in the reporting done by the state department, right at the beginning of this year. So on one state department opportunity, there were eight contracts awarded last year, and there was a data entry mistake made that was revealed in our weekly audit. As the eight contracts, on that state department op had over showed over $20,000,000,000 in obligations in a single week, and that's very, very high for state department. So this was a signal to us to track down where this came from. It turns out that it was a data entry error from the state department where they entered the full ceiling value of each of these contracts as an obligated amount in some of their initial reports. Then they later deobligated or issued sort of negative transactions, deobligating billions in spending. Eventually, this issue was corrected by the state department. They went back and modified all the original transactions and got rid of those those deobligations and initial overobligations. But we communicated this across the Federal Research Organization. And as just one example, an update was made to the opportunity report, that these contracts were on, helping keep our clients informed. So, with just a minute left, I'll pass it back to you, Logan. Thanks, Nico. And, you know, I I think that's a great thing to end on because even self reported public data from the government itself can always be trusted. Right? Like that's where some of the expertise that I've been staring at this kind of data of my whole career. I can tell when, you know, something's got a few extra zeros. So one, I want to thank everyone here today for joining us. I'm gonna quickly wrap us up, and just gonna leave you all with this slide and, you know, really let the numbers speak for themselves. Whether you're already a customer or someone exploring if GovWind IQ is right for your business, This slide captures how seriously we take delivering confident, actionable answers to our users. And in the age of AI, we still believe intelligence requires a human, and that's why we build with that partnership at the center of all we do. If you are interested, we do have a poll, where if you'd like to explore gov win for your business, go ahead and push yes. If you push post no, no worries. We won't bother you. If you are already a customer and want your CSM to reach out about some ways to take advantage of research support requests or if you have questions about any of the thing we went over today, please feel free to, put CSM in the q and a chat and we will get you in contact with your customer success manager, after today's presentation. So just another quick plug. I know a couple people did take me up on the earlier plug I had in today's webinar around getting the clarity report. That's still op an option. We do an industry study every year. We it goes over the full, you know, business unit from BD, finance, human capital management, IT, everything, and, you know, looks at trends and year over year data. So happy to get that to you as well, but that's where today's stats came from. And with that, I will, wrap up today's webinar. Thank you for spending the last hour with us. We hope you learned something, and please let us know if we can answer more questions, either through through the app or find us on LinkedIn. Thank you, and have a great day.