Video: Understanding AI in Architecture: Opportunities and Risks for Practice | Duration: 3608s | Summary: Understanding AI in Architecture: Opportunities and Risks for Practice | Chapters: Welcome and Introduction (6.465s), Webinar Housekeeping (20.93s), Webinar Overview (91.245s), AI in Design (216.24s), AI Research Overview (268.355s), AI Experimentation Trends (364.23s), AI Adoption Patterns (466.285s), Training and Adoption (809.16s), AI Automation Opportunities (1135.275s), AI in Specifications (1390.095s), AI Tools Overview (2005.44s), Common Data Environment (2195.39s), AI Data Sources (2345.965s), Case Study Verification (2632.955s), Key Takeaways & Resources (2819.78s)
Transcript for "Understanding AI in Architecture: Opportunities and Risks for Practice":
Welcome, everyone. Today's webinar topic is understanding AI in architecture, opportunities and risks for practice. Thank you for joining us. The webinar will begin shortly. Here are a few webinar housekeeping notes. For the best webinar experience, please use Google Chrome or Firefox. Audio will be streamed through your computer. There is no dial in. Make sure your volume is up. You can download the presentation slides and additional resources in the resources widget. All widgets can be resized to fit your computer screen. An on demand recording will be emailed within twenty four hours after the webinar ends. If you have any questions during the presentation, please type them into the q and a box. All unanswered questions will be addressed individually offline. On our session today, we're gonna explore how AI is reshaping design and construction, and specifically, not by replacing expertise, but by enhancing how we work and how we innovate. A quick housekeeping note about credits. This webinar will provide you with one learning unit. Here are the learning objectives for today. Recognize strategies and evolution in AI adoption, identify opportunities and inefficiencies in design workflows where AI can provide the most value, identify common AI tools used by firms and how they support design and project workflows, explore educational strategies and resources for responsible AI use in our profession. So here's what we're gonna cover today. The first portion of our session is gonna explore insights from the AIAs, the architect's journey to specification report, which highlights where firms see opportunities, the challenges they face, and their road to adoption. And then in the second half of our session, we're gonna transition to exploring actual tools and practical use cases in how AI and specifications can coexist today. One data point worth anchoring on. While modern AI can simulate creativity and empathy, its real advantage is the scale, unprecedented data ingestion, and computational reach. Today's systems process trillions of words each month, converting raw information into patterns at a speed no human organization can match. Our responsibility isn't simply to recognize this power, but to deliberately harness it, directing where it can be applied, how it's governed, and the outcome it can enable. My name is Lalita Bomanaho. I'm a product leader for specification solutions here at Deltek. I'm also a former practicing architect, now part of a team that's leading the evolution of specifications within our industry. I am delighted to present on this topic and appreciate your participation today. We're all aware of how AI is evolving at an incredible pace and is reshaping several industries. And even within our industry, we're gonna talk about an inspiring challenge. That challenge being, how do we harness AI's potential while also preserving the essential human element of design? As we go through and explore what that means for our industry, by the end of the session, we'll have a clear perspective on AI's role in specifications specifically, ways you can start or advance your AI journey. We'll talk through some of the research and the data source by our industry practitioners. This architect's journey to specification report is published by the American Institute of Architects in collaboration with Deltek and Deltek's sister company, ConstructConnect, both serving as underwriting sponsors. The series was conceived and launched back in 2016, and these reports explore the evolving role of specifications in architecture, design, and project delivery. It highlights strengths, challenges, and opportunities architects face in integrating specifications into their daily workflows. The most recent issue that we're gonna cover today is focused on quantifying the current adoption and the use of AI. I'm gonna talk through some of the key points we want you to take away. So I wanna start off with the key findings. In general, it's low penetration in architecture profession today. Here are the early adopter AI experimentation focus areas. Only 6% of the architects regularly use AI, and they're primarily using it for three use cases, chatbots, image generation, and grammar text analytics. But there's tons of interest. A little over half have experimented with it in some shape or form, but have not started using it regularly. So there's tons of curiosity. They have assessed various tools, explored new applications, and examined different operational possibilities. Experimentation is prevalent in our field reflecting both curiosity and systemic testing. This process is essential for fostering creativity and effective problem solving. So that's not new to our industry. But who is it really common with? Those that are younger and currently less on the firm management side and more on the project architect side are really the ones where the experimentation is coming from. The usage of AI is significantly more by architects age 50 or younger. Those between ages of thirty five and fifty tend to use chatbots more, while architects under 35 tend to use image generators. Let's take a look at where firms are in their AI journey. The adopters here are categorized in blue, and I mentioned this around the firm, not the individuals, that eight percent of the firms have fully adopted it in some form in their practice, and that would be the light blue. So if we combine the light blue with the dark blue, we get the 28% that's little over a quarter are doing something, and they've implemented some type of solution whether it's fully integrated or not. Almost equal percentages, that's the 28%, are also in the non adapter category, and about a third are in the considering bucket. Now switching to the firm side, large firms are driving the earlier adoption of firm wide AI initiatives. So these are the 8% of firms that have fully implemented AI into their operations. It can take any number of forms, and I'll tell you a little bit as we go a little deeper into the presentation. So these are the firms that have 50 plus employees, and they are the ones leading the early adoption in our space. Also, about a fifth are progressing and are starting to adopt different pieces and putting them in place. They are looking at different areas where it would make sense. Large firms also have resources and staffing. Sometimes even have dedicated technology staff like chief information officers, chief technology officers, and have positions like digital practice leads. So, of course, they have resources and the means to try new things and experiment with, different tools and technologies. What's great about these early adopters is that they can show the rest of the industry and determine what best practices are and help them move further. So on the small firm side, especially those focused on single family residential work, they have they often have close relationships with a fewer clients and less need for advanced technology in general. In contrast, firms handling large or complex commercial projects or multi residential projects typically require greater technology adoption. One thing to note here for the lower AI implementation rate for smaller firms, it speaks more to the opportunity as opposed to the opposition of it. What I mean by this is that they haven't really seen, quote, unquote, the need to use AI on smaller projects. So that's where the 7%, which is the highest of the three categories, is coming from. Let's look at the current state of AI. Our next area is about what's untapped. There is a significant opportunity for AI to improve complex processes across firms. According to architects, firms are least efficient at updating product lists, estimating cost and time, and writing complex specifications. This also involves conducting efficient product research, which kind of goes in hand goes hand in hand with specification writing and updating product lists. There is a significant opportunity for technology solution here, including improving inefficient processes within those firms. AI has mostly been trialed and used in low impact areas. So AI adoption and experimentation in firms has mostly been for processes that are already fairly efficient, such as client communications, such as note taking or text generation, code compliance, etcetera. This suggests that the early from adoption was focused more on quick wins rather than solving more complex process issues and inefficiencies. So the most common firm wide applications of AI are in the area of content production, where generative AI is used for image generation or video generation, the use of grammar or tech support, and even for marketing. So AI solutions in areas like updating lists and product research and other complex custom specifications would solve a significant challenge for most firms. And that's where the untapped potential is with these massive complex processes. So let's take a look at, firms seeking education and training. Four in five architects, that's about 78%, are interested in further exploring the potential of these areas. I will elaborate on the key topics they wish to pursue for an additional education and training. They want more than education, more than just information. They seek deep training in these tools and a clear understanding of risks and security considerations. So these are the notable topics that are of interest. Content about the most useful AI tools for architects, continuing education about AI and architecture in general, content about risks and security considerations while using AI, and overall training about the use of AI in architecture. So those are the four notable areas of interest. Firm leaders are especially interested in these areas and should be a key target group alongside technological decision makers within these firms. There's also significant portion of architects that are interested in a charter of responsibility. 82% of them have expressed interest, and they would like to advocate for the establishment of clear standards within our profession, particularly regarding best practices and effective implementation strategies. Early adopter firms and individuals are well positioned to contribute to the development and adherence of these guidelines. The survey also took a look at what type of ethical use of architecture workshops, the firms and individuals might be interested in. And about 50% would expressed interest, to have AI sorry. AIA facilitate these, and about 50% were interested in AI's partner or any other third party firm that specializes in training around the use of architecture. So there's a lot on this slide in terms of what they use it for. I'm gonna walk you through some of the details here. Let's start with the first cohort. So the four in five, that's 79% who regularly use AI or have experimented with it, use chatbots, while half of them use image generators. It's not surprising that chatbots are top of the list in terms of solutions in today's market as we have come a long way from ChatGPT. There are several chatbots that are being used today, and I will, explore some of those in different categories further along in my presentation. But the second cohort here would be to use it for meeting assistance and transcription. About a quarter of them are doing that. Some of them are using it for video generation, design, and other planning tools, etcetera. Even lower is the three d modeling and the true content generators, product management tools, and voice generators. So those are the areas of really low adoption. So in summary, these chatbots, general limit generators, are being used widely, but haven't really explored the deep things specific to our protect profession, which would be the complex, processes. What are they really curious about? And generally, where do they stand on this? This I think sums it up pretty well. Equal percentages of them say, I wanna learn more about it, but I kinda also have concerns. This reflects how individuals view their careers. Utilization of these tools is essential and gaining proficiency in them is becoming increasingly necessary. There's a genuine need for training and understanding as these competencies align with current professional perspectives will also help them advance in their careers. Currently, architects view integration with other tools and existing workflows as well as the belief that AI will enhance built environment, primarily from the perspective of a work efficiency. These aspects are considered fairly low on the priority scale. Next, let's explore opportunities. Where do we see these opportunities? This is strong desire to automate manual tasks as performing these activities by hand is time consuming. Manual work can also adversely affect billing hours since it diverts a dent diverts a attention from tasks that truly contribute to an architect's value. So there is considerable optimism regarding AI's ability to streamline these workflows, increase efficiency, and support product research, ultimately saving both time and costs. A significant portion, approximately 65 to 70%, believe that it can provide inspiration and assist in extracting insights or generating suggestions. This process contributes to improving drawing coordination, transforming data, and supporting data driven decisions by organizing product information efficiently and reducing the likelihood of errors and omissions. That's a lot of positivity. There's a lot of positivity about that, and that's a step in the right direction. A little bit of mixed reviews here for the last piece. Architects see unintentional bias and discrimination as a risk for AI rather than using AI for bias reduction and potentially benefiting from AI adoption to reduce biases. So here are some stats from the AI report. It was noted that firms, 83% of the firms, either reuse specifications from other projects or copy paste them from previous specifications. I know we're all guilty of doing this at some point. It was astounding to see 83%. I was expecting the number to be a little lower than that. Two thirds of the firms are dissatisfied with manufacturers in meeting expectations for providing product information and easy to navigate websites. And that, there arises a need for all of that information to be collated and presented in one, single source. I'll cover a little bit about that, further along. Only about a third of the professionals involve manufacturers during the design stage of a typical project. So this number certainly is lower, and we've not seen a whole lot of growth in this area. And on the data management side, 59% of firms are saying that managing information efficiently is a significant challenge in their projects. On the same note, here are some stats on the engineering firms from ACEC, the American Council of Engineering Companies, as it's stated in the Delta Clarity report. So nearly two thirds, which is about the same number as it is for the architectural firms, see that AI strategy in place is has been working for them. So then that is up by 11 points from the previous year. And about three fourths of firms, 78%, believe that AI will have a positive impact on their firm and better are optimistic about it, which is a 15 increase from the previous year. And nearly two sorry. Three in five firms, 59% say that managing information efficiently is, again, a significant challenge in their projects. Switching gears here. AI specifications today. Where is the need and where can it thrive? The need for AI today comes from these four these six focus areas. Specifications is time consuming. Researching projects, complying with standards, coordinating between drawings, etcetera. You also have inconsistencies in the tool sets that the projects project teams use. And if they're coordinating with consultants using different tools, often require manual processing and conversion, which creates data silos. Just in general, approaches and workflows and lack of standardization with these workflows limits efficiency and consistency across various teams. Collaboration, of course, having lack of tools or inefficient workflows will lead to communication breakdowns and rework. Product research, this is a big one. Keeping up with the latest product information, removing obsolete products from Office Masters, sustainability criteria, evaluating products with different data points. All that becomes cumbersome. And little to no data to begin with. Sometimes even the technology we have in hand, is being underutilized, which can lead to not being able to extract meaningful data at the right time. So where can AI specifically thrive in specifications? As I mentioned, one big area is product research, and there's a huge gap today. The main goal of using AI would be to quickly find products to match the criteria like product, features, performances, or even design intent. To make your decision process easier, but not really for AI to decide for you. Definitely a good starting point. Number two would be content management, employing AI tools to ensure that the specification remains up to date. This may be including and updating content or updates from master spec or spec text or any other database you're using, integrating these changes in all the way down into your masters and subsequently applying relevant updates to projects during the appropriate phases. The third would be stakeholder collaboration. Analyzing AI can assist in formulating more informed decisions for technical experts. For example, ahead of a meeting focused on critical aspects of specifications, assemblies, or systems, AI can help establish a solid foundation for meaningful inquiries. So if the team is meeting with a specialty consultant, the questions that they need to ask them or need to understand what systems are being specified, AI can help with that type of stakeholder collaboration. Conversely, those responsible for administrative duties may also employ AI to ensure timely provision of information by the project teams and to the project teams all the way back to the decision making process, and reduce some of the manual workload. The next would be risk analysis, utilizing AI to proactively identify potential issues and specifications, could be clash anywhere from clash detection to drawing coordination and to address concerns early on and prevent problems from arising later in the process. And finally, sustainability analysis. Validating products involving evaluation, involving materials and methods and, particular environmental impact, specific requirements or design intent that that using AI can make that analysis easier. References often, to standards such as ASTM codes or ASTM standards or building codes may be inferred rather than explicitly stated. So references to that cannot be easily found through a simple keyword search. So using advanced tools such as AI can assist in analyzing the content in context and improving search efficiency for relevant information. Here are some key data, data points and specifications that AI can leverage, references, such as cross references to sections, assemblies, systems, sections all the way down to product types, codes and standards, ASTM and CUL, etcetera. Product types based on material properties, performance criteria, or approved products for particular type of project or even a client. On the same note, administrative requirements, generating submittals, quality assurance, warranty, etcetera, installation requirements, methods and tolerances for product types and accessories, and creating complex assemblies or systems and generating schedules would be another key area. Now let's take a look at some AI use cases. Here are some key terms. I'm not gonna read through all of these, but I thought it would be good to have a slide as a reference if you wanna go back to it. So we're gonna take a look at a couple of practical approaches for prompting principles here. The initial principle is to begin with a specific task. Therefore, the opening of your prompt should feature a keyword such as create, summarize, compare, polish, which serves as an action work while formulating your prompt or instruction for the chatbot. This approach guides the chatbot to identify and interpret the primary objective efficiently. Second would be to strengthen using implied context. So it's advisable to utilize implied context where possible. This approach involves incorporating a relevant secondary context into your prompt. For instance, when conducting a product comparison and focusing on specific criteria for evaluation, explicitly outlining those priorities enables the AI to factor them into your output more efficiently. The third would be to strengthen with examples when applicable. So a lot of us, if you've experimented with AI or any sort of chatbot, would have noticed, you know, sometimes it provides too long of an answer or too short of an answer. Sometimes it's not the structure that you would expect or in the template that you would want. So strengthening the prompt with examples after you've sort of given it your command and given it some reference would really help, direct it to get a better result. So if you wanna create, for example, an outline, maybe it's, early early on within a project and you're keyed in on creating a preliminary project description or a PPD for an exterior wall assembly. Give it a basic template without obviously sacrificing too much of your company data, and give AI your actual template on how you would like the output to be, what sort of properties you would like to see within that template, and how you would like it arranged in a particular format. And lastly, break down the complexities using chatbot prompting. So in other words, break down the problem. A lot of times, we kind of have prompts that build upon each other, and that's okay to have them in a single prompt. And it's also okay to go back and forth, have a chat conversation with the chatbot. But you can instead help AI understand the context of your initial prompt by providing some of the basic project information. For example, maybe you're an industry expert. You've been in the in the industry for twenty, thirty years, and you don't want AI to give you a basic answer. Or on the other hand, you're maybe new to specifications or new to the industry, and you want AI to know that and provide you an answer that really curates your needs. So help AI understand the context of your initial prompt, with those basic project information and have a couple of resources help it to make create an output to make an informed decision. So let's take a look at some of the common tools used in architecture today, and these are by the three categories that we saw in the survey. It's used for chatbots, image generation, and grammar text analytics. So for chatbots, we have ChatGPT five, Cloud, Gemini, Copilot, Perplexity, and then other minor chatbots. For image generation, what we've seen most of the architecture firms use would be MidJourney, DALL E, Verisk, Stable Diffusion, etcetera. And those would help in concept visualization, photorealistic rendering, and exploring complex design ideas that cannot yet be, visualized. The third category would be grammar and text analytics. Grammarly, Notion AI, ChatGPT five, and Copilot are some of the the tools that are being used for document editing, reporting, writing specifications, proofreading specifications, and even text refinement. So here are some AI tools by Projectface. So early design, schematic design concept, and even some of the BIM. Design stages, there's parametric design concepts using Grasshopper. And early design exploration. Arc design is a popular tool. And we have Merit, Veras, and Lumion. These are all for three d massing, rendering, etcetera. Then in the DD and CD stage, Delta spec point is our, spec writing AI assisted spec writing and product data. We also have make it, which would be design and documentation support, and swap would be more for drawing coordination and detailing. And the preconstruction and CA stage, we have, Procore construction IQ, OpenSpace, MidJourney, for various different tasks and applications. Now for those of you who might be new to SpecPoint, it's our next gen specification platform that essentially combines the rich quality of AI master spec content with advanced technology, which also includes agentic AI capabilities. And for those of you who know what Specpoint is, maybe you've used it or you're working on transitioning over to Specpoint, we'll be excited to see how we are leveraging AI and the trends that we are seeing in the industry, that will make your workflow even more efficient. So at its core, Specpoint is designed to solve some of the challenges we all face in developing specifications today, which we saw a whole lot of those in the architecture unit specification report. What are the fundamental strengths of a modern specification platform? What benefits does it provide, and how do those benefits translate to value to specifiers or even end users? Some of the advantages may be explicit, and others may be quite implicit. So first and foremost is the ability to provide the industry with what we call a common data environment. So in other words, a shared foundation that ensures all stakeholders are working from the same source of truth. So it's a centralized digital platform used to collect, manage, and share project information throughout the life cycle of a project. So AI technology is only as strong as the data it's provided. And if the data lives in emails, chats, Word documents outside of the platform or in SharePoint somewhere, it's gonna skew the output of the AI and, therefore, the decisions. So regardless of whether you're using Specpoint or not, the recommendation is to establish a common data environment. This approach typically involves transitioning all members of the project team, including external experts such as building product manufacturers or consultants to operate within a unified platform. These participants interact with specific projects while adhering to appropriate permission controls. Having everyone work on a single platform facilitates the implementation of AI and improves data reporting and the creation of quality deliverables. So prior to providing an example of how the four prompting principles I covered earlier And, before I can discuss the relevant case study, I would also like to cover risks and some facts on data security and privacy. Let's take a look at some of those. What are the current sources for these AI tools? And specifically, any AI tool, it's critical to understand very early on what data sources are being powered by your a powering your AI tools? Is the data coming from the web, such as ChatGPT, for example, or is it coming from a specific repository within your firm? Can everyone should everyone have access to that data? Or is the data coming from a third party? Or is it a blend? And typically chatbot chatbots these days use a blend. Or maybe they use the web version and a specific repository within your firm. So considering those sources is absolutely required. Now for Della within Specpoint, specifically draws from four authoritative sources. It includes AI and MasterSpec library, spec text content library, both of these in its entirety, little under 1,000 sections. And the second source would be the supporting documents. So it includes evaluation and coordination checklists within these supporting documents, which interestingly enough was our initial inspiration for creating this AI powered tool. A lot of times those who subscribe to AI MasterSpec or SpecPoint don't even know that they have access to these supporting documents. This contains a lot of research. The third is Specpoint digital product library. This references product cards from building product manufacturers tied to specific content areas within Specpoint. This information can be leveraged to assist us in our product research, product comparison, product additions, etcetera. And finally, within Dela is help documentation. While adopting a new specification tool, some individuals become fairly proficient with minimal or no training, while others might require additional support. Regardless of one's experience level, questions regarding features or functionality are likely to arise. So recognizing recognizing these that users might not always have the time or desire to consult lengthy resources, we have incorporated health documentation to facilitate effective use of this tool. It's also important to note what's not covered in Della, what's not included in Della. Intentionally, we wanted to maintain strict privacy standards, and that's important for any AI tool you're considering. We don't use individual project data. It has been a request, of course, but that sometimes takes time, and we wanna we wanna make sure we understand and regulate that. So at the moment, we do not use individual user data. We also do not include community data or web data user information. We don't wanna provide the user information to you or provide information from the website that we haven't fully vetted. So the key is to ensure that your data remains protected and the output provided is of a higher quality standard. So think about it. If you're dumping all the data into an AI tool, you know, over time, it's gonna really lower the quality of the output because it's looking at so many different types of sources, and those sources could even conflict with each other. So it's absolutely necessary to use precise methods and select focused data sources to get the best quality output. So whether it's spending hours researching and updating products or masters or coordinating product requirements and so on, Specpoint's AI capabilities as a tool can help you navigate your complex projects. Now in terms of case studies, let's take a look at Dela in action, and I'll summarize this prompt. It's kind of a long prompt. But, essentially, what I wanted to do is to generate a specific table of manufacturers, a product suitable for a given criteria. So in this case, it was an acoustical panel assembly. So I did specifically call that out for the AI. I also gave it criteria I wanted it to stick to. Say, for example, NRC of at least 0.9. A couple other things related the key criteria I was looking for. I included warranties and a few other information. So I asked it additionally as more of a chain of thought prompt to provide a table at the end of the day to include the manufacturer's name, the product name, and I wanted to I wanted it to build me the information in a table format. So here's the output that I received, and it's pretty good to start with. It gave me a pretty nice table, listed all the manufacturers and the products for me. It also put the NRC rating and all the other things that I asked for in a table format. This is a good place to start with AI. And when you're prompting it, ask it a question. Give it a problem that you may already know an answer to. That way you have an opportunity to gauge how good the answer is or how strong the answer is. I also wanted to see where it's pulling the information from. So I asked what the sources were. I did want to verify a couple of things. So the nice thing is it provided me a couple of sources with web links. And it is crucial to consider the responsibility associated with using AI tools, particularly when we assume lie when we assume liability and our credibility and our professional licenses are on the line. While AI may provide what appears to be conclusive answers, it remains necessary for us to exercise due diligence before making a final determination. This example demonstrates how AI can facilitate rapid analysis and efficient processing of large volumes of data. However, it is absolutely imperative to verify all the outcomes thoroughly. And specifying a prod product, I'm especially mindful that it fulfills all required criteria before proceeding. So I'll start to summarize what we talked about today. So here are our key points. It's been low adoption, but high curiosity and exploration. In terms of engagement, it's medium engagement currently through tools like chatbots and image generators. While we have high aptitude for training and cautious optimism about the view of the future of AI in our profession. Here are some links to the resources that I used in the presentation. The first one is a link to the AI's architect's journey to specification report. I also have a link here for Deltek clarity report to which outlines the ACEC stats. And the last one is a link to AEC hub. It's a repository slash portal, and it's it was developed by two emerging professionals here in the Metro Atlanta area. One of them is a good friend of mine. So it's he consolidated all the tools available in the market today, tied it to different use cases, phases of the project, and it's a pretty cool website, if you'd like to check out. It has a lot of information and on programs and AI tools that are used by firms today. So before we move into the q and a section, we have a quick polling question to fill out. And while you're filling out the polling question, I'm gonna go over other questions that have been submitted. I will give you a minute here to fill out the poll and we will dive into the q and a. Thank you for taking the polling question. I see several great questions that came in. We'll have time to address a few of those today. The first one is, what tools would you use for AI based cost estimation task? Today, many firms are using AI enabled capabilities within platforms such as Procore, Autodesk, Construction Cloud, Toggle AI, OpenSpace, and other pre construction and estimating solutions. In conjunction with that, general purpose AI tools like CHET GPT, Copilot Cloud, etc, can help analyze project requirements, can help summarize assumptions and compare alternatives, etc. However, the actual cost estimating tools itself, I would recommend grounding the analysis and authoritative project data, whether it's coming from historical project costs within your firm or, engaging, like, third party softwares that can do quantity takeoffs. I know Construct Connect has some tools, using cost database, tools like, RS means or similar databases, and even firm specific information if you have a repository within the firm using one of the enterprise level general purpose AI tools to pull information from that would be a good way to go. That said, it's valuable for early conceptual estimating, in my opinion, to pull early analysis and identify cost impacts, early stages of design. But as the project becomes more defined, I do feel from cost estimating task point of view, traditional estimating tools, historical cost database information and professional estimators remain essential, just to get reliable budget information as the project is further along. I hope that helps. Let's see. The next question is, who retains professional responsibility when an architect uses AI generated research or content in a project deliverable? That's a good question. The architect is solely responsible for any accountability for the final work. So AI might help process information and generate preliminary response, but it doesn't really replace professional judgment, due diligence, or validation. So going back to a point I made earlier, our licenses and credibility remains connected to these decisions and the documents we approve. So regardless of whether the content generated came from AI or the project deliverable use used a certain percentage of, AI research. I think it all needs to be vetted, before approving. The next question would be, what are the best practices for preventing AI generated specifications from containing inaccurate, unsupported, or, hallucinated content? This is a great question, and I I think all of us have it. That that is the risk that we go into every time we use any sort of AI interface. So one of the most important considerations while using generative AI in specifications, for the best practice would be to treat AI as a research assistant, not as a final authority. So the content must go through a subject matter expert review and source verification prior to approval. Just like how courts now require lawyers to personally validate every citation, I think it also becomes important for us to validate where exactly that is being pulled from. I would also say using AI tools, that are grounded in authoritative sources rather than solely relying on information coming from the web. So that ties back into where is that being drawn from? Where are those conclusions being drawn from? I think would be very important in making a determination of what part of or what percentage of your, AI answer would you take into consideration? And the quality, again, highly, highly will be dependent on what standards it's pulling from, what library, what databases are those internal, are they coming from external sources, etc. The other factor to consider would be transparency. If you cannot verify the information or further through chain of thought prompting, if you're unable to get a clear answer from AI, that should definitely be a red flag. Critical information, especially from standards organizations or codes, I would say, we would need further verification before just accepting it at face value. So I I feel it kinda ties into the accountability question. Professional judgment exercising professional judgment is absolutely required, and everything needs to be taken with a grain of salt. And the more your models are trained on the type of queries that you're putting in, the more better it will get over time. Believe we have time for a few more questions. How can smaller firms adopt AI, when they do not have dedicated technology staff like the digital practice leaders. I think I covered a little bit of, little bit about that. I think smaller firms can start with a focused pilot rather than a firm wide transformation. So a good use case that we recommended to one of the small firms that was just looking for guidance, overall AI implementation guidance, not specifically tied to specifications was to take one of their existing workflows and have someone oversee the evaluation portion of it, what type of AI tools that they are, considering, what type of endpoints they have, and defining what information they're comfortable entering into that tool and comparing it against a current process. So in establishing an alternate workflow through an AI, but in a controlled environment instead of trying to do it on a larger scale, That would be the best way to go. Let's see. Think the next question would be, what skills should architects and specifiers develop to prepare for AI? That's a good question. I think it varies, depending on the comfort level, I would say, more than the level of proficiency. But I think three skills that will become increasingly important for us would be to understand how to ask better questions, the prompting piece. And the second one would be evaluating reliability of the information and applying judgment when and as needed. I feel the most successful professions won't necessarily won't necessarily be 100% AI reliant, but using AI responsibility effectively and efficiently would be the way to go. So those would be the ones that I would highlight. I think we are right at time. So that brings us to the end of today's session. If we didn't get to your question, we will be happy to follow-up offline. Please feel free to connect with me using the information on the screen. I will share my LinkedIn details and my contact information. AI is transforming our industry, and technology alone isn't the answer. The real opportunity lies in combining human expertise with intelligent tools to work smarter, make better decisions, and deliver better projects. I'm grateful to be part of the Deltek team helping advance the intersection of technology and specifications, and I'm excited for what the future holds. Thank you for joining us today and for your thoughtful engagement throughout the session. Have a wonderful day.