"My Insights" in AutoCAD: Overview and What It Means for the Way You Work

DAN WHITCOMBE: Hey, everyone. Thank you for coming to our class titled Insights in AutoCAD, What They Mean for the Way You Work. Today, you'll be hearing from Ashish Arora, the engineering manager for product analytics on the Insights team, Brian Souder, senior principal data strategist for the Insights team, Khurram Tahir, the senior product manager on the Insights team, and me, Dan Whitcombe, a product manager on the AutoCAD Desktop team.

Before we get started, I just want to mention that if we mention anything related to road maps or future development plans, they are just plans, not promises. The development, releases, and timing of any features or functionality may change, and you should not rely on our presentation today to make any purchasing decisions.

But with that, I want to give you an idea of what we're planning to talk about today. So we'll start by taking a closer look at insights with Brian who's going to talk about what an insight is, why we care about insights, but most importantly, why you should care about insights. Then Ashish is going to take us on an Insights factory tour where he's going to show us how insights are made and how we go from usage data to an actual insight that you can take advantage of. And then I'm going to focus on insights inside AutoCAD and what the future of insights looks like. And we'll end with a Q&A. And with that, I'll hand it over to Brian.

BRIAN SOUDER: Great, thanks a lot, Dan. It's my pleasure to talk to you guys today about my insights and insights as a whole. My goal is to give you an overview of what our insights. So by the time you walk out of the room today, you'll know a lot more about what we're doing, and hopefully, you'll be very excited to get insights for your very own.

When we think about insights, we think about information that's highly personalized. It's delivered in a timely fashion. It's actionable for you, and it's also very convenient. So these little boxes on the left-hand side are the insight cards, and these are specific for each individual that's receiving them. So let's take a little deeper look at what insights actually are and the themes of insights that we are currently delivering.

Insights, as with all data that we work with on the Analytics and Insights team at Autodesk, starts with data, your data, the data from how you use the product. So specifically, we're interested in the commands you're using. We don't care about the actual drawing itself, and we don't actually know what is in that drawing, but we do know the series of commands.

And we build an anonymous profile. That profile has information about your company. It has information about your usage history over time, including the commands you use, the entitlements like is it a commercial product or a student product, for example, and last of all, information about you. For example, when did you start using this product? That's a great example, but it's not your name and stuff like that because that's not exactly what we're doing here.

This information then flows into, in essence, all of the plumbing that you're going to learn more about when Ashish speaks in a minute. It's made up of a decision engine, different machine learning algorithms, and a publishing engine. And the result of all of that is a series of insights that gets delivered.

So now let's go in and look at the actual themes. After we've seen this sort of high-level overview, let's now look at the themes of different insights. So I'm going to highlight these eight themes. The first one is just usage by category. So this is a reporting type of insight. So it's just showing the user how they're using the product over the last, in this case, 30 days and where in the product they tend to spend their time.

The next one is a command recommendation. So here, what we are attempting to do is to provide the user with the command they might want to learn next in order to advance their skills. And they are given in a very specific way, and Ashish will talk more about that.

Feature recommendations are like command recommendations except these are at higher levels. Have You Tried articles are very popular at Autodesk, and the way we're dealing with them differently is because we look at how you're using the product. We get a sense for whether you already know something or not. So we're not going to highlight a Have You Tried article-- in this case, this is about leaders. If we see that you're busily using leaders all the time, then you wouldn't be targeted for this one. But if you're not doing something, that would be an example of when we might target you.

Upgrade-- so we know if you've upgraded, and then also, we know what you might be interested in about this update. It might be, for example, we're highlighting a specific maintenance release because you've ran into a problem, and we want to highlight this maintenance release to solve that problem for you. Another possibility is you use a specific command a lot, and that command has been updated in a given release. So that would be an example where we might highlight that new release because you are using this command a lot.

Frustration detection, and again, Ashish is going to talk more in detail about this, it's where we watch your work, and we try to determine if you're struggling with something. And if we determine that you are struggling with something, we provide this type of insight with more help.

Most used command sounds very simple. It's another reporting example, but we actually call this an efficiency booster or performance one because in this case, we're highlighting these most used commands because this particular user isn't using the alias. So as many of you, I'm sure, know, if you use AutoCAD a lot, then these aliases, like just being able to touch an individual key to start a command, can really speed your work up. So that's why we use these most used commands.

And last of all, app performance where we're actually talking about, for example, how your startup or file open compares, your average startup or file open, compares to the rest of the Autodesk customer base. So you might determine that, oh, my system is running slower than average or whatever. These are a few examples, and I would ask you to keep in mind that we're just getting started, and we're listening to our customers a lot. And so we're trying to make improvements all the time.

This is what the actual insights look like in their destination. So in this case, we deliver an email once a month, and you can go in and interact with them. This is the Autodesk account area on the website, and again, we post these up about once a week. The timing won't stay consistent. We expect it'll get more and more near real time over time, but right now, it's about once a week.

And then, most recently, we have this new Start Tab area that has the insights. So again, you would expect these to become more and more near real time if you would. And in these cases, there's a few other capabilities. For example, if you're inside here and you click Learn More, well, as you would expect, it goes into the appropriate help information. Also, if you're in here and you click Show Me, it actually has the ability to highlight things in the UI for you.

These are just a couple of examples. This last one is an example of us getting deeper into the actual product with these insights, and Dan's going to talk more about that in just a minute. So that's what I wanted to do today to just give you this little quick overview, talk a little bit about the different types of insights we have, and I look forward to your questions at the end. But for now, I'm going to turn it over to Ashish for his factory tour.

ASHISH ARORA: Thank you, Brian, and hello, everyone. And now I'm going to give you a deeper dive into the behind the scenes of our Insights Factory and show you how these insights are actually created. But before we do that, let's take a quick look at what are the guiding principles that we have that are the core set of values that we use to operate our Insights Factory on a daily basis.

We live in a time of continuous disruption, and data being in the forefront of that disruption is being used across the industry by everyone to maximize profits. But for us here at the Insights Factory, we believe in the power of customer first mindset, which essentially translates to customer needs being in the forefront of our end to end development cycle.

Trust and privacy-- with the great power of leveraging user data comes great responsibility of ensuring that it is secure and protected at all times. And we are continually building processes that help us ensure that we are using this data ethically. And throughout AU, you have been hearing a lot about AI and machine learning and our investment as a company in this field.

And here at the Insights Factory, we are also investing heavily in the field of AI. But what we have learned while doing so is that even AI when augmented with human intelligence, that is of our AutoCAD experts, is where we reap the most benefits and can provide the highest amount of customer value. Now, as we go through the rest of our Insights Factory tour, you will see these different principles come into action on how we create these different insights.

Next, let's look at our raw materials that we use to actually create these insights. Raw materials, essentially what these mean to us is the different type of data sets that we have on our customers. It starts from user and company information. The first thing to note is by user and company information, I do not mean any kind of personally identifiable information on our customers. That's not what we are using, but rather, we focused on things like what is the industry group that they related to, when did they actually start using AutoCAD, when did they become an Autodesk customer.

Other things related to our customers are, for example, the Autodesk site usage that they have how, for example, the engagement that they have on our Autodesk forums. The other obvious things that we have are also the products that they own, the products that they use, and the license types that are associated with those different products.

The other primary thing that we are looking at, and this is the key thing that we look at, is the underlying product usage related to our customers. By product usage, what we mean is usage across time of our customers, the different commands that they are using, the sequences of those different commands, the drawing settings that they have, and the feature usage they have.

Another key important thing to note here is that we're not looking at any kind of intellectual property on our customers, but focus on a higher level usage metrics like the command usages and the feature usages. And now, as I go deeper into the making of an insight, you will start to piece together how these different data sets are very important for us to create those insights.

Now, without any further ado, let me take you through the process of making an insight. And to do that, let's take an example of Rachel, who is a hypothetical user working at a multinational company as a contractor. And she has been working in AutoCAD since the past three years.

And right now, she's focusing on this project where she's working on drafting an architectural design. And the things that she needs to accomplish to be able to complete this project are annotation, 3D modeling, blocks manipulation, and object editing. And Rachel, being someone who is still relatively new to the world of AutoCAD, has a good grasp of basic object editing but is still learning a few of these other skills.

And now as she starts working on the project and starts creating that product usage data, it starts flowing into our data platform and in turn starts enriching and updating the user profile that we have on Rachel which combines all the different data elements that I just spoke about in the earlier slide, which then this data in the user profile gets sent through the decision engine which is the core of our overall platform which combines that AI plus human intelligence of our AutoCAD experts and curates the different insights for Rachel and publishes them back to her in the form of these three delivery channels that Brian spoke about earlier, which are email, account portal, and AutoCAD startup.

Next, let's take a look at what were the few different insights that Rachel received and how were they actually created. The first insight that Rachel received was a frustration detection insight where the aim of this particular insight is to actually help our users that are struggling to use AutoCAD and specific commands in order in AutoCAD. And how we actually do this is by analyzing the different undoing patterns that we have on our customers.

Well, the first thing that comes to mind is undoing that. Undoing is a natural part of our user's workflows where a user would draw something and just undo it right after it because they do not like it. So how do we actually use undoing patterns to be able to detect if a user is struggling with a command or not?

How we do this is we divide this problem into two different questions. The first one that we try to answer is how difficult a particular command is when compared to other commands in AutoCAD. The other one is, what is this user's learning curve when compared to other users?

Now let's take an example of Rachel and see what she has been doing with AutoCAD. She has been using the SLICE command to divide the 3D objects that she needs to draft. And here, we see that her average undo rate is above the 90 percentile as compared to other users like her.

And when we look at the underlying output of this model, we see that Rachel has been using the SLICE command in the past 90 days, and she has used it 51 times where she undid that command right after that 32 times, which puts her undo rate to be about 63%.

Now, this information that was outputted by the different models gets sent to a publishing engine where this information gets enriched by content that our experts have curated, and that's when this insight would get delivered back to Rachel in the form of something like this where it would say "learn how to be more efficient with the SLICE command. We noticed that you frequently undo the SLICE command. We recommend looking at these helpful hints and exercises."

The next recommendation that she received, and this insight is called the Command Recommender insight where the aim for this particular insight is to intelligently highlight the commands that our users might want to learn next. And underneath, what this is, it is a Netflix-style collaborative filtering machine learning algorithm, which is doing something very similar to how Netflix would recommend you movies based on your past viewing experience.

Now, looking at the underlying data that gets fed into this particular model, we see that the data that gets consumed is the user data at a command level and at a given period of time. Now what starts to happen in this model is that these users and their command patterns start getting mashed together, and the users that are similar to each other, for example, the users that have been working on drafting versus users that have been working on commands that are more related to 3D modeling, starts getting clustered together.

And that's where the model starts seeing that, OK, this particular user has these particular few commands that are missing based on other users like them. And now, in the case of Rachel, the three top recommendations that she received were MEASUREGEOM, AUTOPUBLISH, and 3DFACE command.

Now again, the output of this particular model that was generated gets fed back into a publishing engine where this gets enriched with content that are exposed that AutoCAD experts have curated and also give the ability to our experts to filter down this particular output based on certain rules to detect the false positives then that these particular models can generate. And that's when this insight would get delivered back to Rachel in the form of something like this where it would say, "have you considered learning the 3DFACE command? Based on the command you use, others with similar usage also use the 3DFACE command."

Now that Rachel has had some time to actually review these insights and look at them, she gave us some immediate feedback in the form of this thumbs up and down functionality that we have on our insights. And when we look at her feedback, she actually liked the different particular insights that we spoke about, the Command Recommendation and the Frustration Detection as well as another insight which relates to the Update Recommendation specifically regarding the blocks pilot feature enhancements that we did that she has been utilizing a lot.

Now, this implicit feedback that she gave us as well as the explicit feedback in the form of all the different type of usage and adoption that she has had with the product is what gets fed back into our decision engine, and that's where the decision engine starts learning and reinforcing the different events that happened on that insight so the thumbs up and thumbs down, and in turn, improves the different insights that will be created in the next iteration for Rachel as well as others like her. And with that, I would like to end this Insights Factory tour and would like to pass the baton to Dan who is going to talk next about the future of insights.

DAN WHITCOMBE: Thanks, Ashish. So as Ashish mentioned, I'm going to talk about AutoCAD, what insights look like in AutoCAD, and how we're thinking about where we want to go with the future of insights. So we've talked at length about individual usage insights, such as command recommendations, frustration detection tips, usage by category, all of which you can see here, and how each of these can deliver value to you as an AutoCAD user.

We've also talked about how these insights are delivered, both in the form of the online Autodesk accounts portal or even in a monthly email. We've also heard from many of you about how much you're enjoying this feature already, such as this one user that talked about how much it's improved their overall efficiency or another user who talks about incorporating a new command directly into their repertoire or even this AutoCAD user who's been using AutoCAD for 30 years but still was able to learn something new through these insights that we're surfacing.

So here, you can see how this feature and this new capability can deliver value to all different types of AutoCAD users, not just somebody that's new to AutoCAD, but somebody who, like many AutoCAD users, have been using the program for decades. But where are we going next as we think about insights?

So let's return to Rachel, our hypothetical CAD user. So we've talked about how we can deliver individual usage insights to Rachel, but what if we put Rachel in the role of the CAD admin in charge of a bunch of people who are using AutoCAD? How can we deliver insights to Rachel as a CAD admin that would provide value to her in this role as a CAD admin? And that's where this idea of team insights comes into play.

So team insights are available to the primary AutoCAD account admin in the AutoCAD accounts portal and provide a bird's eye view of how your team is using AutoCAD and where there might be some room for improvement. So for example, you might see an aggregate summary insight of the actual individual usage insights that your team is receiving so that you can know where your team might be struggling with certain commands and be able to address those issues directly.

Or you might see an aggregate view of the software performance that your company is receiving, which might trigger you to want to improve that performance or address those issues. In addition, you might see a crash summary so you can see which version of AutoCAD is crashing at what frequency, again, so you can directly address those issues just to keep you more efficient and to make those processes of seeking out those problems a little bit easier to do. Also, you'll be able to see how your current seats are distributed and whether you're using all of the seats available to you to make sure that you're extracting the most value from the subscription that you have.

Looking ahead, we're excited to bring insights to more Autodesk products beyond just AutoCAD, and doing that will also allow us to bring cross-product insights because we know our users are not users of individual products but use many products in the execution of their projects. But before we do that, we want to make sure that we go deeper into AutoCAD to really put insights in front of you in the places that they matter, which we know to be in product.

So we've talked about one of the ways that we're doing that, which is the My Insights Dashboard, which is available now in AutoCAD 2022.1, and it provides an aggregate view of your personalized usage insights. But we're excited to show you about a feature that we're currently working on called the Macro Advisor, and this feature will provide a personalized and, importantly, non-intrusive way of delivering command macro suggestions in the context of your workflow that are relevant to the way that you individually work. So let's take a look at how this actually works.

So let's say Rachel is working in AutoCAD, and she is typing a sequence of commands. In this case, it's PURGE, then ZOOM, then SAVE, but it could be a variety of different command sequences and different command sequence lengths. But once she hits a certain command sequence that activates a trigger, she'll be recommended to take those command sequences and turn them into a new macro through this notification that you see here.

Once she clicks on View the Suggested Macro, she's taken to a brand new command macro palette, in particular, an Insights tab within that command macro palette where she can see more information about that suggested macro. From there, she can try that macro, or she can directly save that macro. She'll also see other suggested macros that she can incorporate into her workflow that might be recommended to her at another time, or she can choose to adopt them right here in the moment.

Once she chooses to save a macro, that macro jumps over to the Saved tab of the command macro palette where you can see other command macros that she's saved. In addition, after she saves a command macro for the first time, she'll also be able to access and edit and manipulate those macros from a new tab in the AutoCAD ribbon.

But within the Save tab of the command macro palette, she can also edit that macro. And she can do that via a new Command Macro Editor dialogue. And in this dialogue, she can change the name, the description. She can assign a command alias or an image or icon to show up in the Save tab. In addition, if she wants to customize this macro to match her individual, more personalized needs, she can edit the syntax of the macro directly from this new palette without ever having to access the CUI.

So how does this feature work behind the scenes? So again, we have Rachel, our CAD user, and she is working on a variety of different projects. In the background, the Insight Engine is looking to determine, what are some frequent command sequences that we see Rachel using often?

And from there, we determine which of these command sequences might be a good candidate to surface to Rachel that she could take advantage of to be more efficient in AutoCAD. In this case, it's looking like AUDIT, PURGE, and QSAVE is a good candidate to turn into a macro for Rachel.

But then this information goes to, as Ashish mentioned, both an AI algorithm as well as gets reviewed by a subject matter expert who's an expert in AutoCAD and is able to say, yes, this sequence is a valid and good sequence to surface to Rachel to save her time, at which point, the next time she executes that trigger, she'll be recommended to turn the sequence of commands into a macro.

And now zooming out, what we do want to mention is how this relates to the larger legacy of what AutoCAD is and what AutoCAD does for its customers from taking the process of manually and tediously drafting over a drafting board to putting lines on a digital screen to then making those lines more intelligent to help you achieve better design outcomes.

And as you may have heard in some of the keynotes, this is all part of the AutoCAD's strategy to deliver you, the user, more features that provide you insights as well as automations to make sure that you're spending more time in AutoCAD doing the things that you enjoy and get value from and make sure that you are as proficient and efficient in the product as you could possibly be.

If you want to learn more about this strategy, definitely check out the keynotes, the general session, the AEC keynote, and the AutoCAD keynote. You can also attend the AutoCAD Markup and Collaborative Feedback class where you'll learn a little bit more about some of the automation features that we're excited to bring to you.

But we also want to hear it from you as we think about the future of insights. And to that end, we ask you to please feel free to go ahead and leave a comment on our AU class page to let us know the types of insights that you would like to see developed into the future. And with that, we want to thank you for attending the class.

BRIAN SOUDER: Thank you.

ASHISH ARORA: Thank you.