Datagrid: AI Agents Using Knowledge, Tools, and Actions to Augment Humans

THIAGO DA COSTA: Welcome to our talk for Datagrid. We'll talk about AI agents. Agent automation, and how companies are starting to leverage AI across the construction industry and also in finance and other fields. We're super excited to see what people are building with AI. I think AI is here to stay. Everybody has been using it. We've been talking about it. We've been leveraging it. So this talk is really about agentic automations. Agentic AI, how agents perform, and a bunch of use cases that are very, very exciting and that we're seeing customers do every day.

But first, an introduction. Datagrid is a company that I started. Datagrid is actually a combination of Toric which was an ETL platform and then became this AI automation platform. We have two products, a data movement platform that's used widely in the industry, and today I'll be presenting to you an introduction of the company, but also the use cases, the customers, and how a lot of this AI capabilities are coming and helping everyday customers.

So really, I'm going to cover a genetic automation, a genetic AI, and then talk about how these products are enabling people. My background is in data and AI. I've been working in data for the past 20 years. And then building manufacturing, construction, software tools. I worked at Autodesk in the past. I also had a company that did mechanical engineering software on the cloud, and then built this company that builds data and AI solutions that are enabling construction companies.

The Datagrid is connected to the historic platform, which is a data movement platform. This is widely used in the industry. Many E&R companies use it every day to move their data from their construction projects and job sites to their back office, to their data warehouses and data lakes to then use that data for analytics and leverage on performance optimization, operations, finance and so many other things. We have now powered more than 50,000 construction projects this year in the platform. We ingested over 50 million records of data, which is an insane amount of construction data. And we are super excited to see how AI is now this layer on top of all this data that customers are enabling and then leveraging it in so many ways.

So Datagrid is this agentic automation platform that we built, and it's composed of knowledge, tools, and actions. And I think we'll walk through that through this presentation and explain what are agents and how do they leverage knowledge, tools and actions. And hopefully, by the end, you're going to be inspired to start leveraging AI agents on your day to day on your projects.

Quick introduction Agentic AI, I think large language models came in into the picture very recently in the computing picture and in everyday life, people are using AI agents in so many ways, and AI models in so many ways with ChatGPT and other tools. So there are models, foundational models that are able to generate information.

And when we say generate, we mean really be able to predict. So predict a sentence, predict the next word on a blog post, or predict what the words would look like if they were arranged in a different way, such as rewriting or translating something into different languages. So AI is capable of doing all these things through the advent of predictions, and many people now believe that a lot of human intelligence was encoded into text, and then AIs were able to learn through what we wrote on the web and in YouTube videos and whatever else. And AI is now able to learn from all these unstructured data and basically create all the concepts and ideas that we as humans have expressed for so many years.

And we have now created this trove of data that AI can use to learn and pretty much behave and act in a way that's similar to the way that humans do, and mimic the way we reason, mimicking the way we think. So we have models are powerful tools that we can use to generate text, generate images and generate any kind of information. They're trained on private and public data. So many companies spend hundreds of millions of dollars training and building data pipelines to train models that are effectively learning from data that was chosen.

So you might choose to learn from books, from social media, from blog posts, from scientific articles. So these models are learning a variety of things, and they are learning through composition of data sets. So data sets are composed from science data and literature, and then code. And in this way, the AI models and the foundational models, what they are doing is actually they are learning tasks that we do every day.

So whenever we wrote something like to say hello in Portuguese, you say hola. Then I think whenever the agent sees that request, you can actually predict the word, and it can keep on predicting from there. Because it has seen that through its training. So that's the way that AI models learn, and is a very versatile thing.

But there's one main problem is that learning is frozen in time. So models don't continue to learn from that data. That data is not continuously updated because it's very expensive to continue to train a model. So we're left with this situation where models are incredibly smart, can do all kinds of things. But they're always using information from one to two years ago. And because that's the time it takes to train a model and then we're left with out of date information, facts.

Except that when facts are general enough. But that's really the conundrum here. So AI models are super powerful, but they take a long time to learn. And you can't really update those learnings very quickly.

So Agentic AI is really how do you take those models and how you give agency to a model like a human would have agency, or have the ability to decide, do I go left? Do I go right? Do-- if I'm asked to do a task to do some research before I do the task is something I would do because I have agency to do the research. I know that if I do the research, I'll do better. So AI agents, it's really this idea and this capability that agents, that AI models are paired up with in ways that they are able to then take that agency and decide to use tools, decide to go to the internet and learn a little bit more before they do something.

So the difference will be if you ask a question to a model, the model is just going to respond with its own learnings. If you ask a question to an agent, the agent would respond with potentially the research that it has done on that topic before it responds. So its a think before you speak kind of problem. And AI agents are goal oriented. So they can have goals. They can have objectives, they can have decision making criterias within them.

Do I provide this information now or do I hold on to this and learn a little bit more to provide information? So a self-reflection mechanism. AI agents can also do multi-step workflows, is the ability to decide to use multiple tools, like I might check multiple websites, multiple things before I respond, multiple data sets. And I call these tools and learn about this information and then combine with some other information before I do something. And after I do all of that, I might send an email or am I make a phone call. So these are multi-step workflows, which are very powerful and kind of unleash the power of agents and AI.

And then learning and adaptation, how do AI agents actually learn and adapt through those learnings to do better. So if they do a task every day, like you hope that a human learns to be better as they are learning to repeat that task. And it's the same thing with the AI agents. We want them to learn and adapt as they are going through their own learnings.

And at the core of all of this, there's a key problem that we'll talk about in a second. But essentially the idea is really agentic AI is really the key enabler that we'll see in the next three to five years, I think, in AI and will be a critical innovation on top of AI models.

So there's an optimist. There's a negative view for AI that I don't share, which is the AI is going to take everybody's jobs and so on and so forth. I do believe the AI will optimize a lot of the time, but I do believe we can all do more with AI. We can all be augmented and do better. It doesn't mean we don't need people to do work, that people might do more interesting work than repetitive work that may be an agent might be able to do.

So the optimistic view is that I think we are all overloaded with an intense amount of information. There's lots of very complex tasks that are multi-step that are very difficult, that require a ton of research, and that we need to all be adapting all the time to change. And as people, we probably take a little bit longer than an AI to adapt. Maybe not an AI model, but an agent can adapt very quickly to its learnings.

And in terms of executing tasks, you really want to be able to delegate and just say, hey, go off and do this and come back tomorrow, right? With the results. Don't need this information right now. So I think that's how we work as humans. We work in distributed environments now, and so on. And we think agents can really be able to augment us with in a lot of ways that I think will be enabling in the future.

Separately from that, I think that's just it's a very big change in us right now. And what kind of work is important for us to do. So that's something for us to reflect on as we start working with agents.

So what agents need to be successful. So if agents are so cool and can do so much, well, why aren't the ones that everybody has right now being kind of enabled with AI agents? Well, there's one key problem. And the problem is still the same problem as always, it's data. You have an insane amount of information inside of the companies where every interaction that is done in a job site, in a shop floor, in the office is recorded. Every interaction is effectively recorded in a digital media. And that interaction is really not accessible to most AI.

And most tools, you wouldn't probably trust throwing all this data inside of an OpenAI model. By OpenAI, I mean an open model on the web, or a company that's training on your data. So you probably want to keep that data still private. Like I don't want my personal transactions and financial information, my company to be used for training. I want that to be used, but not for training.

So there's this really interesting edge where you want to enable agents with your data, but you want privacy and you want them to get better, but you don't want them to learn from your data as much. At the same time. If I was to hire somebody today and they were joining my company, I would want them to come in with a lot of learnings. A lot of things they have learned in their previous companies. I don't need them to tell me what they learned, but I expect them to have experienced. We hire people based on experience.

And it's the same thing with AI agents, I think, and even with the AI models to a degree, you want them to already have experience in a thing, and if you're doing law documents, you want them to have learned something about law before they begin the job. And not only you want them. To learn how you do law in your firm, you want to learn how you do contracts, how you do project reviews, how do you analyze risk.

These things are unique to companies and they don't all do it in the same way. So it's really important that we enable agents to go learn from that data when they join the company. So if an agent joins my company, I want the agent to be able to go and learn from all that data and use those learnings.

There's one critical problem is AI agents don't have access to data, and it's really hard to think about how to do that safely when systems are so open and made for everyone. So our focus and what we believe is that there is a space for enterprise level and business enabled AI agented automations.

The other problem is multi-step reasoning. Like how do agents reason. We're just seeing models that are starting to reason. But reasoning is a very difficult thing. You want to reason with context. You don't want to reason just on training. It goes back to the problem of data.

And then contacts on your company or organization, like these are not just data context. These are like semantic contexts, like who's connected to who. Why are we connected? What is the past and what does the future look like? Those things are things about reflection that agents need to have, and then access to things like APIs and tools. How do I access an API? How do I call a platform that I'm being using to update a record? How do we call it to get information?

So these are things that are very difficult for AI agents to do until recently. And then the last thing is, you don't want to just be doing this every day. You want to automate it. So if I want to do something every week, I want that to be remembered and done on time by an agent workflow.

Navigating adoption is really tricky for sure because there's just a range of adoption, people are saving so much time with AI. So from writing emails to real estate listings to RFPs to the next thing. And it's really important that it's fast adoption continues in companies, and that people are able to leverage the best of the technology safely. But it's still a challenge. Many companies have been kind of blocking AI or trying to prevent the usage of AI. I think there's a path where you can do this safely and adopt it in a way that is enabling, enabling the company without adding any risk.

And I think the last piece is explainability. How do you explain, how does AI explain itself? How does it explain what it has done? And it has to be engineered to be able to do that. If it's not built in that way, then it doesn't just become good at explaining. So it's really important that AI models learn to explain themselves and the agents learn to explain what they have done and explain themselves. So this adoption curve is I think it's very, very early days still in enterprise SaaS.

Building safe AI agents and other aspect. How do I trust the data? So if I have an engineer working on a blueprint, I definitely trust that blueprint because the engineer went to school for years to learn about that, and they're not going to just hallucinate and start making banana cake recipes. But AI agents are provably able to switching between something to something completely different. Not normally, but it can happen if they are misconfigured.

And truly AI agents are determinist. You can have fully deterministic predictions with the AI agents using low top p values, low top K values. There's a lot of science behind this stuff, but AI models that are publicly available, they are made to be very random because there's something fun about not generating the same answer again, and also attempting to generate something that is a little bit unpredictable. It's more natural.

So we are all used to see lots of reports of companies building with AI and just kind of learning. But there's ways to make determinism fully deterministic, fully transparent AI. And it just comes down to the intention of whoever's building it. Contextualization and privacy is really important. It has to be built into these systems. And then the graduated adoption piece is how do you actually enable the adoption of AI without hindering teams that can adopt it right now versus teams that might take a little bit longer to adopt because they have a mission critical data set that cannot be exposed.

So what's Datagrid? So with that, I'm going to talk about what our solution is and what we have built for this industry. And really Datagrid is the AI agent model that is built on the concepts of having knowledge, having tools, and have the ability to execute actions for users. And it is a truly unique stack of agentic automation. And it was built entirely with the idea that data is a key aspect of learning and it should be available as a first dimension of an AI agent.

It's not just about attaching something and having the AI agent having visibility on a PDF. It's really about truly learning from data, from interactions that people had, from processes that have ran through your data pipelines. And using agents in a way that enables agents to then use tools to do things and then perform actions for people that do actual work that normally would have to do it yourself as a human.

So [INAUDIBLE] is really about getting work done, it's really about finding the information that you need and working from that information. It's really about creating content quickly and then using that content on actions like update a record, or sending an email, or sending a follow up, or keeping tabs with something, or generating summaries and tables and reports. It's really about using agents to the best of their ability to do work that normally we have to do. And most companies have to do lots of reports, lots of content has to be created every day, and a lot of that content has to be done manually and verified.

And until if we enable AI agents to truly be intelligent about the data and the context, then we can trust them to do this kind of work for us. So I'll play a brief little video on connecting an agent that we built in data grid and then using this agent to learn the data. And then after the agent learns the data, we can start asking the agent to do things for us. I'll play a brief video now.

So you can see simply connecting the data to a platform. So we can choose to connect to over 100 platforms. In this case, we're connecting to ACC and the agent has begun to learn our office forms, schedules take offs. And this is just a lot of different tables that exist within the ACC environment. And then quickly the agent is able to start answering the questions.

We'll deep dive on each one of these things. But something that happened really quickly here was the ability of the agent to plan on how to do those actions. And then configure follow ups and connections that allow the agent to continue to do that work. So deep dive on the use cases, and then walk you through how to set up one of these agents and how they can be used.

So in the first dimension of knowledge, agents are able to go and access all these data sources and then leverage these data sources for learnings. The second dimension here is really the ability to interact with tools such as going and connecting to Microsoft Teams and then schedule things that happen at a later stage. So this is the ability for the agent to remember an automation and remember recurrent action.

Those actions are remembered and done at that time or whenever data changes. And they are really connected to that data at that point in time. So whenever an update happens, if I want to follow up in a certain system, that update is going to contain the latest information.

And the last step that we walk through is the actions, which is the ability for the agents to plan how to actually do these things, and how to do them effectively. So whenever goals are set and the agent sets off to do its thing, it presents you a plan. And then that plan is actually followed very closely and then the agent can give you updates on the progress that it has made towards the goal.

So we'll look into a few use cases in how companies are starting to leverage. We're working lots of companies that are now ramping up AI across the enterprise. And then there are so many use cases to talk about, but I'll talk about the main ones that we see that are going to power a lot of new workflows and will make people much, much more effective at the work that they are doing today.

So the first one is here is summarization. And summarization of logs and manpower logs or daily logs or any kinds of logs that exist within ACC or any other system that is used for project tracking within the ecosystem of construction or manufacturing. The ability to really ask the agent to do the summaries and to generate the actual logs.

So logs are essentially summaries of things. So if I have an issue, and there is a new change that I need to request, those things are the things that go into the logs. So logs are actually a drain. A lot of people spend a ton of time writing logs that are inaccurate because the actual changes, they're already recorded in the systems. So it's a very, very powerful use case because there's so much data about what changed, what needs to be recorded, and then folks are left with writing a whole log entry, which it's entirely automatable.

So I'll walk through this use case of automating a log by playing a quick video. So in this video, we'll be building actually the builder agent that we have in the images and on the first video. So builder agent is simply a data grid agent where we just gave it a purpose, we gave it a name, we gave it a icon, and we gave it some basic access to data.

So what's going to happen here is the first thing I asked is for the agent to connect to the data. And you'll reach out and actually figure out, OK, I can connect to Autodesk Construction Cloud. Once we log in, we gave access to the agent to start learning and then the agent begins learning in the background about all the data that's accessible. So when I ask what data does the agent has access to, the agent can then go out in the world, right, and learn from all of the data that was connected to it and all the different data sets and things, and come back with an answer about not only the data, but then like what's in there.

So this is a super simple case where I'm just asking, what data is available from project submittals. My construction, real estate contacts database. I have a ton of data sets that I put in the system, and then including financial data, which will walk through the next use case. But basically the entire thing is connected now and available to the agent.

Now I can actually go and go into one of these data sets and then figure out what actually is in here. So in this case, I'm just looking at simply the setup of this data. I came in with original commitment, unit prices and so on. I can enrich and empower the agent to modify this data and improve the quality of it. So one of the main things would be classification of this data. So I'll classify all these budget entries automatically.

So I want the budget entries to be just learned. I don't want to be typing them. I want the agent to just go in every time that a new budget entry comes in, I want it to go and augment that with the classification based on the description of the entry, and what I believe a good class is for this particular entity.

This is a super simple way to look at any automation workflow on the data to empower the agent. And then this can be done, of course, in an automated fashion. So the agent will always be in reaching this data now. So we learned that it should be going through every record and do this work for us. I'll pause here.

Now that the agent was-- that we enriched some of the data that the agent was able to use, we added descriptions to the data. Now we're going to go off and ask the agent to do things for us. We have enriched the data just to show the ability for the agent to learn how the data could be better. We'll go now and talk directly to the agent that we used to connect this data to and ask it to start doing work for us. So please find information about the ERCIP project. This is a project in one of our databases that is a government project that is going out, and we want to learn more about this project.

So instead of us going and actually spending a ton of time summarizing this information ourselves, we're asking the agent to do it. And this is a 600 page PDF document that you would have taken me a ton of time to just go and parse through and understand what's in there. The agent looked over 600 pages. It found 17 pages that contain critical information about the project, including the scope, the disposition of the project, the actual dates, the links to where this project is being posted on the web.

And what we can do with this is then start working with the Writing Assistant to go and write this summary about this particular project. So this just getting details around this project, and then producing tables that allow us to then now track the actual project buildout. So this is a writing agent that is working with our builder agent. So as a separate agent that was designed, it was added in here just to write RFPs. So this agent's planning behavior and actions are all around writing of RFPs. So the builder agent can work with the RFP writing agent to consolidate all the information about the ERCIP project and then produce the documents that we need to then analyze further and then start working from.

So you can see here the plan was pretty basic. We're consolidating and generating a detailed response about this. And this is just getting sort of deeper into that document, like getting deeper into the contents of the document, seeing what the sources are. The agent can go to the web and continuously watch for those sources and see how they change. And then towards the end here, we will be asking this to be broken down into just the task list that the agent can help us with.

So now we'll go ahead and get a list. And then this list is going to be used by the agent and us to fill up the next action. So we will be having to draft them, then writing a bunch of things, summarize a lot of this information, and then log that out. So we'll be doing this with the agent rather than by ourselves. The agent can then continues to read data from the software platforms that we're using and connect to our email. So I'll ask the agent to send these updates to me by email at my specific email account.

And you'll see here the next step. The agent actually asks for access to email, and then once we're authenticated to access email, which is pretty much a couple clicks to provide authentication and allow the agent to send emails, then the agent is able to not only read all the RFPs, but also read them, summarize them and send them by email to us. So as the work is completed by the agent. And once the agent has the ability to send emails, then it will be you'll be communicating with us by e-mail from now on.

You can see here, the agent is now actually planning differently. He's starting to plan many more tasks like email composition, extraction of facts, the email drafting itself, informing us about the status of this writing that you'll be going through. So the agent from through the interactions we're having the agent is learning about how it should be behaving in this situation and what is the specific ways that we're trying to get help here and will start to tune itself to behaving that way.

So that's a little bit about summarization and then creating content around using agents to help with assist with content generation. Now, the next step here I want to talk about is just financial insights, is getting the agent to help us with reporting tasks. In this case, I'll be doing this demo live and just walking through a super basic workflow on how financial data can be accessed.

So here, here's our builder agent. Basically, the configuration of this agent is pretty simple. We can see that the agent has the ability to access all sites we give access to. I have multiple agents for safety. I have some for working on my RFPs. I have an engineering manager agent that just does engineering follow ups. And I have a lot of different data sets that were added here, including engineering data sets, finance data sets from my contracts database, and my invoices. Construction data sets, and then go to market data sets with things that I'm doing on getting new business.

I also connected my email so the agent has access to my email, but also has access to send emails itself. And you can do that in order to respond and give feedback. So now here, if I go and ask about invoices, for example, the builder agent has access to all of that data. If I ask about invoices and I just make that one, what are the invoices you have access to? The agent will go on to, again, to the planning and all its behaviors is going to be figuring out which invoices are accessible to me because not all of them might be.

And then you will retrieve those and start presenting them to me. Retrieving this is going to require the agent to actually do some analysis. So in this case, because this is financial data, the agent will be using an analysis tool instead of just looking at it as from a text perspective. And in this case, retrieving that information, we want to retrieve just the most accurate, latest information about invoices, and that's what the agent is going off and doing.

So we found 30 invoices on my payments patterns and financial status. It gave me access to all the invoices, the payment terms, the invoice dates. I can download this invoice. But we built an entire little table for me with all of it still writing out each one of the records on the table and giving me more details from this. I can certainly go back and into the actual table and the source table and make modifications. But he basically was able to go and find all my invoice data and then retrieve it to me. And this is the latest most accurate information.

The next step here with manpower logs. I want to summarize at 9:00 AM my manpower logs from all of the data. And in that case, I actually gave very specific instructions to the agent that I wanted to number one, summarize the logs, find out products that have not entered any logs and check how much details in each log. I want to make sure I spot any logs that are sparse, that are not fully completed.

And the agent came back with a plan to create an automation to summarize the manpower logs, and he scheduled the automation for 9:00 AM PST and he would do these steps every day at 9:00 for me. And it's telling me a little construction joke at the end, because I configured the agent to do so.

And in terms of looking at information from a project directly, if I'm working on, let's say, a school project or the RFI in the school project, it will be able to actually understand what I mean by school project, because it has been learning while I'm away, while stuff is happening. It actually is starting to understand what school projects are, and then which ones are available to me. So I'm just going to go and look at the RFI for this school project that is the one that I'm assigned to, that one that I'm working on.

So it's a very flexible and capable agent that can do a lot of different work, and it can do these tasks just off of raw data. You don't have to change anything. And it found actually the Alameda School Project Log, which is the one I'm on. It presented me all the RFIs and then it gives me links to the actual source where this information is available and can be used by me and by my colleagues.

So just to summarize, we walked through a use case with looking at daily logs and summarizing content from logs. This was the first use case we looked at. We looked at how financial insights can be managed or accessed with AI agents. You can ask anything like group information in this way or retrieve sums or averages, and it does all of the querying and returns with all the information as tables. We walked through a use case on categorizing data and making sure that the data was enriched and categorized by the agent.

You can, again, identify risks on projects by defining what you think risks are and how to work through those risks. You can take actions by email and via Slack. You can send follow ups and notifications when you ask. So if I ask it to engage with me over Microsoft Teams on this particular topic, you engage with me on that topic, and be able to do tasks write content, retrieve information, give me the latest, and follow up with people in my team.

And we also looked at how it could help you generate things like RFPs by looking at an entire data set of PDFs with hundreds of pages, and then sifting through all that material and creating summarized content off of RFPs. So that's data grid. That's our AI agent platform. It can be used in a number of ways and empower teams to move faster and leverage all their data to build knowledge and use tools to have agents perform actions for people and reduce the amount of manual work they have to do in day to day. Thank you.