How Mott MacDonald and Autodesk Are Using Product Insights to Build Skills

IAN BESFORD: Hi there, everyone. And welcome to our Autodesk University 2022 class about how Mott MacDonald and Autodesk are using product insights to build skills. As this is a joint class with Autodesk, we need to share this Safe Harbor Statement with you before we start. Which I'll give you a few seconds to read through the detail.

OK. So I'm Ian Besford. And I'm digitally with Buildings at Mott MacDonald, in the UK. I'd like to start by asking you all a question. What if we could really help people access training that's relevant for them that helps them upskill, and as a result improves the efficiency and the quality of work that they deliver?

It may seem like an obvious thing to aim for. But the reality is that making it happen is a real challenge for many people. Particularly people who are at the front line of delivering projects.

Take, for example, Josephine. She's an experienced structural engineer and she's worked at Mott MacDonald for nine years. She meets clients to discuss future work.

She line manages people. She does design. She does delivery, and manages projects. She does everything.

And time is precious to her. She also needs to find time to keep up to date with the latest design tools, and the features they're in, and worries that she'll be left behind. How is she going to manage that?

This is Sandeep. Sandeep has just joined Mott MacDonald, as a technician. He's got four years experience at another company as a BIM user. And he's a keen and enthusiastic individual, who wants to develop his skills. He's looking for support to understand how we do things in Mott MacDonald.

What are our standard processes? What are our standard ways of working, whether our template's saved, all that sort of thing. He also wants to get even better at how he uses software tools. But he doesn't know where to look.

This is Mikaela. She's worked at Mott MacDonald for 25 years. And she's one of our most experienced technicians in the office.

She's proficient at lots of tools. And she's keen on helping others improve. She feels she's got a lot of knowledge to offer.

She aspires to become a recognized expert in her field. But she's struggling to work out what her next steps are for development. These are three very different people.

But they've got one common challenge. They all want to develop their tool-based skills and competency, just in very different ways. So I'd like you to imagine a future where our tools and our systems are all intelligent enough to know what these people will benefit from, and to help them achieve their goals.

It's not about skills assessment. It's not about searching Google for the answer. It's about putting personalized development actions front and center, where they're needed. And in this talk, we're going to show about how that future is closer than you might think. We'll share how Mott MacDonald and Autodesk are using data to provide people like Josephine, Sandeep, and Mikaela with the help that they need when they need it and how we're removing the barriers to skills development to make it more accessible than ever.

First we're going to cover the importance of learning to Mott MacDonald, and the value it creates. Then we're going to cover some of the principles we followed and take a look at the way we and Mott MacDonald are using our data to help people learn. Jo is then going to share the work Autodesk are doing in the same space, looking at data to help people learn. But doing it through Autodesk lens.

Then finally, we'll close by explaining how we're bringing these two different ideas together to provide the best outcome for the most important people, our end users. And we'll leave you with some thoughts on how you can get started on your learning journey. Really, the case for learning and development is pretty self-explanatory.

People have a natural desire to want to develop skills and to master things and to self improve. And the Holy Grail for many people is finding a career which they can fulfill their own purpose, whilst also mastering skills that they enjoy. It's important that people enjoy that journey of learning.

Because you spend a lot of time learning and focusing on that, as well as the outcome is the ideal. It's in the interest of every business to have skilled employees too. I mean, if you look at the opposite case, what company wants employees who are unskilled and don't have that drive to improve?

Really, we take the need for development as a given. Richard Branson was spot on when he said this. Why wouldn't we want to train our people?

We need them to be skilled at their job. And we need to make sure that they stay. The real challenge is how we train effectively, and how we get maximum return on the money that we invest in people's training.

And that's one of the reasons we're thinking about this. So to give you an idea of the size of the prize, in Mott MacDonald, we have over 6,000 monthly users of Autodesk tools. That's mainly focused on Revit, Civil 3D, and also AutoCAD.

These people, over the course of a year, spent a quarter of 1 million hours using these tools. So if we can even get a 1% efficiency saving by using the tools better, that's the equivalent of 1 and 1/2 full-time employees freed up over the year. So we recognize the importance of training.

We've got to push to double our investment in technical training in tools over the next 2 years. If we're going to spend that amount of money, we want to spend it right. And our users are telling us that they want this as well.

We did a survey of all our global BIM users. And over 70% of the people that responded said that better access to structured tool training was one of their main priorities. There's clearly a target audience for doing this, and a pull from the end users.

But actually, what real value does it add to the business in pounds or dollars? We think there's three areas where we can add value. One is about allocating our training budget more effectively.

If we under invest in training and we don't invest in particular areas, we don't develop skills. Those skills can't be used. That's just stands to reason. By identifying areas that our current skills are low, we can allocate the budget to those areas, which improves our ability to deliver our products using the tools.

The second area is reduced wastage. It's widely recognized that wastage occurs when people develop skills that either aren't relevant to their role. Or more likely, are relevant to the role.

But they don't do the learning at the time at which they actually need those skills. If they don't apply them in practice, then this money is just wasted. If we an individual's likely skill level and their role and what they're doing, we can give them personalized development goals, reducing that risk of wastage.

The third benefit is around reduced overhead. If we haven't got clarity of the digital skills that people will require in each role, then we waste time, or the individuals waste time, by digging out the right training, preparing training plans. Having to apply for budget to spend the money on training.

All of this is money that is better spent on the actual training itself. If we put some figures to that, for every 100,000 pounds we budget to spend on training, we spend 60,000 of it. Of that 60,000 pounds of learning that we do, then only 40,000 pounds is actually applied on projects.

Effectively, we don't get value from over 60% of our training budget. Which is not exactly a great place to be. What we want to do, is for as much of that under 100,000 pounds as we can, to be spent on the people, at the point in which they need it.

That's the business case. Just as a lead-in, then. One of the great things that we found about Autodesk University in the past, is that it offers you insight from industry sectors way outside your own specialism.

Often there's ideas there that you can apply from other industries to your own benefit. And as we were sharing with Autodesk our thoughts on skills and behaviors, it became really obvious that we were both thinking about skills models used in video games. On Autodesk's part, they were thinking about the structure of skill trees, and how people develop their skills in a hierarchical way.

And at Mott MacDonald, we were thinking about the experience points that people gained by completing activities, and how this level up the measurement of skills. What we're going to share with you now is how we've gone about implementing these approaches. I'll now hand you over to Tom, to talk about the work we've been doing in Mott MacDonald.

TOM HUGHES: Thank you, Ian. I'm Tom Hughes. I'm program manager of product systems at Mott MacDonald. Ian and I have been talking about skill builders for design tools for a number of years.

It's not just Ian and I. There's a whole team that have been working on this project. So Neelam, Ash, Graeme, Ana, and Michelle, have been working closely together to develop, test, and deliver skill builder for design tools.

We've got Mark Shields, our technical excellence leader for Europe Business. And Ed Scrutton, global head of learning development, that have been providing overall steering and governance for the project.

Thinking about Ian's what-if statement, we had an idea. What if we could use data that we have available to do the heavy lifting, and provide proactive suggestions for people to develop their skills and hence, competency that at their role? So we took that idea and started to think about what it would mean for Josephine, Sandeep, and Mikaela.

What if the data that we hold could tell us that Josephine is a member of lots of projects, but rarely uses BIM 360? We could make a suggestion Josephine attends an upcoming BIM 360 accelerator course to pick up the skills she needs.

Likewise for Sandeep, we might see since joining Mott MacDonald, Sandeep's been regularly using Autodesk Revit, but hasn't yet completed a Mott MacDonald course on the Revit structural template. We could make that suggestion, but we could do more. We could think about recommending to Sandeep to make use of our Mott MacDonald Dynamo add-in, or connecting with the wider community of Revit users.

And then Mikaela, we could see that over time Mikaela's been one of our most frequent users of our Autodesk tools we could make a suggestion that Mikaela think about becoming a certified professional in the tools she most frequently uses. Those opportunities are something that we tested, a small scale pilot, looking at skill builder version 1.0.

We took a very, very quick approach, a proof of concept, and put that data in front of 50 pilot users. We found the results to be very interesting. First of all, as a project team, we spent less than 250 hours to develop, test, pilot, prototype, proof of concept.

The predictions that we made about people's skills were, even at prototype, very accurate. 74% of the predicted skills we made were accurate as reported by pilot users. Finally, and perhaps most importantly, all of our pilot users, 72% said that having this quick access to relevant training goals would increase their likelihood of completing learning.

We got some great feedback. So both from the perspective of people who are using the skill builder version 1.0, in terms of their ability to see some of the potential skill gaps and taking courses to improve learning. Likewise, we had some great words from Ed, our Global Head of Learning & Development.

It's all about reducing friction putting those recommendations quickly in someone's line of sight, and giving them that psychological boost to increase their motivation and desire to learn. going I'm going to go now in some of the how we approach this skill builder, and then move on to where we're going next.

Thinking about the challenge and opportunities that we faced. Self reporting of skills is low, inconsistent. Also, skills change over time.

Goals and recommendations need to be relevant for a person's project role and also, their current level of experience. We recognize that advancing technology is changing both the tools we use and how we learn. So the way that your manager might have learned might be different from you.

Finally, we looked at the data we had available and recognized that we'd got sufficient data to identify those key themes and make recommendations.

Thinking about that data, One of the things that we did before processing any was think about these three areas; whether we can, whether we should, and also what could go wrong? Both Ian and I feel it's really important to speak about this today. Particularly for anyone watching the class thinking about doing something similar.

From a "can we?" perspective, we reviewed our employee privacy policy. And from Mott MacDonald, collecting and using systems data for the purpose of learning and development is something we can do. But we didn't stop there.

We thought about should we? It's not just system data. We're going to make some assumptions and generate some predictions about people's skill.

What's that going to feel like for them? It would be really clear right away through this project, predictions are always going to be limited by the data we've got. And they're only there to support learning. They're not the same as an actual skill level.

Finally, what could go wrong? It's often too easy to think about all the good things that could happen and not the bad. So we were thinking about control and access, and making sure individual predictions remained individual to them.

Even where we might have desire to show predictions at a higher level, thinking about how we can keep those suitably anonymized, so that individuals [AUDIO OUT] targeted-bound predictions, but leaders can make decisions, also with our senior leaders, to make sure we set off on the right path.

How do we take that data and think about it in that gamified framework Ian talked about earlier? For those familiar with games and skills experience, you can think about that framework on the right-hand side. Think about the smaller actions and tasks that you do on a regular basis. And think about missions, side quests, that you do less frequently, but contribute more.

Finally, those big-story quests and bosses, you might not do them very often. But they're a big milestone in the character development. We thought about that relative to the data we have on our tools. And we're able then to weight, from smallest to largest, those different events that people were doing to create a numeric model behind our dataset.

What did that look like in practice? Well, we could start by thinking about that data by person, by tool, and over time. We mapped out all of those events. And we also started to look at gaps in the data.

But potential indicators that skills might be going down, rather than up. To begin with, we did this at very small scale, literally with an Excel workbook. But once we'd got a good sense of the data and the parameters that we wanted to control, we moved that work into Python, to start computing XP value at a larger scale.

Once we were able to compute the XP value at a larger scale, this is how we linked it to our existing skill models. We calculated XP for over 1,000 people, for six key Autodesk tools. We looked within those 1,000 people, for people with relatively well-known skills and used those well-known skills to make predictions about people we knew nothing about. You can see on the right-hand side, how we mapped this to the 6 levels of skill we have at Mott MacDonald. But that could be any levels, beginner, intermediate, and advanced.

The other thing we did was think about prediction confidence. So depending on how much data we got, we could then rank those predictions from a very low confidence to very high. That's the theory behind skill builder, but what are we doing next?

We wanted to take all of that thinking that we proved to a small scale pilot, and make it available to those 1,000 people and beyond. We've been looking at skill builder in a few key areas. One was building a resilient and secure data pipeline and warehouse. Looking at API connections to Autodesk, Pinnacle Series, and Knowledge Smart data, and then the infrastructure needed to process that data on a regular basis.

Alongside the data pipeline, we did an analytics service, taking that Python scripting we developed at small scale, optimizing it, and getting an app to run on a virtual machine. Finally, we wanted to update the Skill Builder app. A small-scale prototype, [INAUDIBLE] was great, but it had some limitations. So we've built a specific SharePoint web part to provide Skill Builder in all the places that people are looking for skills.

That's in our intranet, in their practice sites, in their project sites. That allows people to authenticate automatically, only get access to the data that they need. But also, interact with Skill Builder app, setting their preferences and providing us more insight into their skills and goals.

What does this look like? Well, here's skill builder 2.0 in a SharePoint web part. The key things to pick out are the ability for people to switch quickly and easily between different job roles. Really important for us, as people do many things with [INAUDIBLE].

View by Practice. Again, as our technical tools are aligned to our global practices and people can belong to more than one, we wanted to give a quick way for people to see tools relative to what they're doing right now. Probably the main part is looking at recommended level versus predicted level. So based on an individual role and experience, where would the practice recommend they are? And what do the predictions think? Doesn't mean there's necessarily a gap. But if there is, there's personalized goals right there in front of them, one click away, start that activity and develop skills.

One last slide, before handing over to Jo, is really just the value of connected thinking. Mott MacDonald Autodesk Pinnacle Series have been talking about skills and data going back nearly 3 years. I don't think we'd have got where we were without it. But also, it's allowed us to think about how we can all be solving different parts of the same complex problem. Thank you. Over to you, Jo.

JO VERMEULEN: Thanks, Tom. I'll talk about some of the projects we're doing at Autodesk, to improve how people learn to use our software and how they can improve their skills. But before I get started, I just want to mention a bit about myself.

I'm a research scientist in the Human Computer Interaction and Visualization Team, focusing on software learning. Helping people learn our software and improve their skills. I'm based in Toronto, in Canada. I joined Autodesk in August, 2020.

I'm a former professor of computer science. And I'm originally from Belgium. This is our team, the HCI 7 Visualization Group, led by our director, George Fitzmaurice.

What we do in our team is we focus both on near-term research that can have an immediate product impact. An example of that is the ViewCube, that was developed in our team. But we also do longer-term research, that spans 5 to 10 years out.

Like, programming within a VR space or interactive on-body fabrication design. Actually, some of our past research projects are quite relevant to what we're talking about here today. This project is called Community Commands, in which we adopt collaborative filtering algorithms to recommend new tools in AutoCAD that you've never used, but people who are similar to you have used.

We then present them in a small panel in the corner of your workspace. Building on this idea, we then developed a command map, a research project that was released as a plug-in for Fusion 360. Also, presented at AU 2019. It allows our customers to reflect on their skills, their development over time, and how their skills relate to others in their industry.

We are now also beginning to see these kind of innovations more widely available in our products. Like the My Insights feature in AutoCAD, that provides personalized advice based on your usage, helps you identify new commands to try, or even automate some of your tasks. There's also a growing number of these insights available for our users.

But what I really want to talk about today is a project for which our research team collaborated with the team that is developing my insights for AutoCAD, to create a new insight that shows the user their most used skills for a given time period. I'll show this in a demo in a little bit. The core concept underlying this visualization and this prototype is what we call the AutoCAD skill tree.

Now, what is a skill tree? As Ian already alluded to, both Autodesk and Mott MacDonald were thinking about skills as they relate to video games. Video games have actually used this concept of a skill tree, borrowed from the learning sciences.

In a gaming context, the skill tree is about characters within different professional classes, like a druid for instance, that can advance their skills and powers through different levels. The Learning Strategies Team at Autodesk is doing something like this with skill trees for products. Like AutoCAD, for instance.

The advantage of this is that instead of talking about software features, we're talking about skills that map to people's jobs and careers. And that's quite powerful. What does this actually look like?

Here's what the AutoCAD skill tree looks like. At the bottom of the slide here you can see a visualization of the skill tree, so it's quite big. It consists of 2,281 commands that are categorized into Workflow Skills, Sub Skills, and Tasks. We have 15 workflows, 50 skills, and so on.

If you look at a particular command, for instance, the Layer command, you can see how it's actually categorized within this tree. It's within the object creation and organization workflow, within the drawing layer skill, and subdivided into sub skills and tasks as well.

How are we now using this skill tree, now that we have this? Essentially, what we can do now is we can relate different commands together, and also talk about skills at a higher level than just individual commands that you use. That actually provides a lot of interesting and powerful information.

I'm going to show a quick demo of how this report works and what you can do with it.

PRESENTER: What you see here is the most used skills report. This shows the user their most used skills in a particular time frame. In this case, November, 2021. I'll just give you a quick overview of how to read this and what information is presented here. In the left column in the table, you see a list of this users 10 most used skills for November. This is sorted by frequency of use.

This user has used the skill object modification the most. And the skill navigation the least. The next three columns to the right represent breakdown of commands into categories, Core, Occasional, and Rare, where you can look at the legend at the bottom to understand what these mean.

Core commands are everyday commands that form the basis of most users' usage. Occasional commands are more targeted and less-frequently-used commands. Finally, Rare commands meet the need of very specific workflows. The legend also indicates how to interpret the color coding in these three columns.

The darker blue a cell is the more those aspects are used. For example, for the object modification skill, in the first row, since the Core cell is a darker shade of blue than the Occasional cell, that actually means that the user has used more core commands in that skill than occasional commands.

We also have gray cells, as we see here in the Rare column. These gray cells indicate no usage. For object modification, this user has not used any Rare commands. Finally, we also sometimes have hatched patterns in the cells here, as we can see for drawing environment, customization.

This indicates that there are no commands available in that category. For some skills, we may only have occasional and rare commands, because that's already a very specialized type of skill. The next column shows the percentage of total command usage that this skill took up in that time period.

In this case, 29% of users command usage was object modification, in November. The last column shows the number of different commands used within that skill. For instance, for object modification, there are 109 unique commands and the user has used 18 out of those.

This is only a high-level overview. We can break this down further and look at it in more detail. We can open up individual skills. Let's expand object modification here.

What we see here is a breakdown of all the different commands within the Core, Occasional, and Rare categories. The small rectangles that you see here represent an individual command with a two-letter abbreviation. These command rectangles also use the same color scheme and the same color coding to represent frequency of use. So dark blue commands are used more than light blue commands.

Gray commands have not been used at all in this time period. We also see a breakdown of how many commands are available in each of the different command categories. For example, there are 41 core commands in object modification, of which 17 were used.

If we hover over commands, we can actually get more information, such as the full command name. We get the icon that's used in the user interface, how much that command has been used, and a short description explaining what that command does. We can do this for every skill in the table.

For example, if we expand drawing environment customization, we get the same kind of breakdown. However in this case, there are no core commands available for this skill, as indicated by that hatch pattern. We only see the occasional and rare commands.

The advantage of this interface is that it really provides a very nice overview of which skills and commands you've been using the most, which commands you haven't used yet. But which are still available within that same skill. So this helps people really see what else they could try or learn.

We actually provide another feature to actually help people with that. We also highlight recommended commands. We can see that some of these commands rectangles have an orange outline. Those are the recommended commands.

These are commands that haven't been used yet. But based on similar users' usage and past usage of this user, we actually recommend this command to the users. This might be something that they might be interested in trying.

We also have a small batch near the skill name, to indicate that a recommended command is available. We also see here that there's another one for object creation. If we open that one up, we can see that the point command is recommended to this user.

JO VERMEULEN: OK. Yeah, that was a quick overview of how we're using the skill tree to give people more insight into how they're using the software and how they can improve their skills. What does it actually take to enable this capability?

What we do is we capture the types of commands that users invoke when using our software. Then that data can do a bunch of different things. It can help us improve product performance, for example. But it also allows us to deliver these kinds of personalized experiences and insights.

We process this data. We then link it to the skill tree, categorize it based on workflow skills, sub skills and tasks in the skill tree. We calculate the frequency of use.

Then we aggregate all of this to build this visualization, highlighting the core, occasional, or rare commands, and then present that usage data back to you, so you can understand your own usage patterns and identify opportunities to upskill and learn more about the specialized features that our products offer. We may also suggest personalized recommendation and learning pathways, so that you can really jumpstart your learning journey.

This was an effort with multiple people involved. I just want to recognize that. People were involved in Research, Design, and Development. This feature is currently being developed for AutoCAD. Yeah, I just want to acknowledge the rest of the team. With that, I'd like to hand it over to Ian again.

IAN BESFORD: Thanks. and thanks Tom, as well. We explored both these in parallel, with separate teams. But when we were talking about it, we started to think about what the opportunity was to bring these two different approaches slightly closer together.

We're very aware that one of the key limitations of skill builder is that it depends quite heavily on the quantity of usage of a tool. But it's got no real concept of what the quality of that usage is. So an apprentice fresh from college can join us and spend 40 hours a week working on a say, Revit. They could do that for a year so they've built lots of experience over time.

But that's not the quite the same experience as one of our veteran Revit users, who spent an entire year developing parametric families, or modeling a complex, highly-architectural building. Put more simply, it's the difference between somebody sitting in front of AutoCAD and doing the same circle 1,000 times, or using the Array function properly. So what we thought was that the skill tree and the graduation of tool-based skills that they've got in there gives us some of that information.

It splits commands into different levels of complexity. So we can take the blend of the different complexities and assess how advanced a user is. It's this sort of thing.

Although, it's not perhaps entirely true to say that a good user only uses the occasional or rare tool. It's pretty logical that the more specialist tools that somebody uses, the more experienced a user they're going to be. So what we do is we can weight the tool usage data that we have according to the type of tool used, so that if you use more advanced functionality in the software, your rate of accruing experience points is slightly enhanced.

It might not make a huge difference to those particular points. But the whole experience point model is based upon lots and lots of small gains, and over time they add up. So if you use better commands in a tool, over time it will make a big difference.

The other thing that we also thought about is how we can feed the two off each other. We already have some proactive suggestions within skill builder to help the user get better predictions. Within Mott MacDonald, we use an internal system called the Professional Information App, which is a repository for people's interests and technical specialisms and the skill that they have in those specialisms.

The tool that we've developed already uses some of that data to help make its predictions. So we provide a prompt to people to update the data within the tool, if they haven't done so in a while. We thought, we can also do this with other things. For example, with insights, we can do this.

This is a bit of a work in progress, but the idea is the same. We can check in the skill builder tool if a person has signed up to receive insights from Autodesk. If they aren't, we can provide a prompt and a link to help them do so.

That way, the end user gets the benefit of all the things that the skill tree and insights can offer. And we get the benefit of having some extra filter that we can apply to the data, to make their predictions better. It's a win-win for us all, basically.

What does all this mean? Well, if we go back to our friends we mentioned at the beginning, Josephine can get bite-sized insights that she can digest at her leisure. She gets targeted specific courses about new features and tools when they're released. And she gets specific training that's relevant to her role as a checker and a reviewer.

Sandeep, on the other hand, gets guidance about how we do things in Mott MacDonald, front and center. He gets personalized insights in email. He also gets things about tools to help him improve.

We also spot that he's not a member of our [INAUDIBLE] community yet. So this is suggested, for him to join the community. When he reaches the right level of experience, he'll get a recommendation to undertake Autodesk certified professional training.

For Mikaela, she also gets that recommendation to be a certified professional. We can make a suggestion to her that she might want to consider becoming one of our Super Users, by doing the Super User training and then enrolling in the program. She might also get asked to help create and develop more content for skill builder in time. Each one of them is getting bespoke development recommendations which are suitable for their role and their experience.

What can you take away from this? I suppose, fundamentally, people are simple. Putting ideas in people's line of sight is really key. So having good quality help one click away-- or even better, no clicks away-- in the tool while they can use it is great.

The other thing is people learn better with a variety of content. So it's not just about pushing training courses at them. It's about short, bite-sized content that's relevant.

It's sharing videos, encouraging them to share knowledge with others in their team, being a super user, and helping a broader group of people, or getting certification. All the sorts of things that work together to make a blended, all-around skills user of a tool.

There's real value in this chunked-up approach. With our great feedback, we're providing short, sharp activities that people can do and then apply is a really good way of people learning in a pressurized environment. I'd encourage you to think about the data you have and how it can influence learning.

For us, this was all possible because we already had structured tool usage data from another project. That lowered the cost of implementation of our pilot to make it feasible. And Autodesk's ability got data based on tool usage, which allows them to compare it across various different users, to give their predictions.

Think big, but start small. Our proof of concept was done for less than 10,000 pounds. The upscale solution that we're developing to roll it out to the wider business is an order of magnitude more than that.

But we've got the confidence of investing that money in developing the tool, because we did a robust pilot. We thought about the bigger picture and the bigger potential one we did it. But we didn't get stuck into all the nitty-gritty detail.

We produced something really, as a minimum viable product, and focused on the prediction engine behind the scenes, to see whether that was real. Sometimes I think there's a natural instinct to keep ideas to ourselves. So a massive shout out to our Autodesk Account Team, for introducing us to Jo and the Project HALI team, who sparked the idea for how we can integrate these tools.

We didn't know what would happen when we shared with Autodesk what we were thinking early in this program. But by being open about our approach, we've learned something we didn't realize. We've improved our tool, and we've improved theirs.

That's really why we're sharing this with you today. For though skill is an industry-wide thing, we can't wall within to Mott MacDonald. So if everybody can improve through what we've done, then the industry improves as a whole. Thank you from me, for your time and attention. I'll pass you back to Jo, to close, who just has a final ask of you.

JO VERMEULEN: Yes. Thanks so much, Ian. If you're interested in getting involved, we have a number of ways in which you could do that. First of all, my insights are continually being developed.

The team behind this effort are always looking for customers to provide feedback, or participate in user research. Secondly, Autodesk research is also exploring what these in-product insights could look like in the next 5 to 10 years. And we're doing research into intelligent features that help our customers' be more productive and achieve better outcomes. And to realize this, we rely on high-quality customer data.

We're looking for customers who would like to partner with us. If you're interested in collaborating with us on this exciting journey, please reach out to your CSM or join the Autodesk Research Community. I'll put up this QR code as well, which you can scan to sign up for the Autodesk Research Community.

With that, I would like to close and just say, we're super excited to have had this opportunity to share this ongoing work with you. We're also very happy to hear your questions and feedback. Thanks so much.