Deploy Better Plug-ins Faster Through CI/CD and Unit Testing in Azure DevOps

ANDREA TASSERA: All right. Welcome back to another presentation of both of Autodesk University 2022. My name is Andrea. I work at Woods Bagot. And there, I lead the automation efforts for the design technology team, the global team.

Welcome to Deploy Better Plug-ins Faster Through CI/CD and Unit Testing in Azure DevOps. I realize it's a pretty long name. But we'll see what this means.

And just before we get started, this is not going to be a hands-on session. We're not going to go through steps together and get to a finished product at the end. That would take too long. So I'm just going to show you-- I'm just going to demo what we've done at Woods Bagot. We think it's a pretty cool case to see. And hopefully, with the help of the slide deck and the manual at the end, you'll be able to replicate for yourself.

So let me start with a couple of important questions here. How long does it take you to deploy tools in your office? So you're working in an architecture firm or anything else really. And you create tools for your colleagues to use. So you have an idea. You develop your tool. From that point till when the users are actually actively using it, how long does it usually take?

And the other question that I wanted to ask you is, how do you ensure the quality of the tools that you are releasing? Well, I have some experience in this field. And these are some of the options that I've seen in my career and talking to friends as well.

Sometimes, in terms of how long does it take it to release, sometimes it happens that you develop something, you're done, and you send your files to someone-- to some of the users, you send them over-- I don't know-- Slack teams or whatever. And you tell them please copy them here. Bit hacky, probably not the best of the options, especially if you have many users using your tools.

Sometimes, you keep it to yourself. You're the one that developed it. And you're the only one running it-- probably even less efficient than the first option, of course. Because then a lot of times you're interrupted. Can you please run this instead of actually spending time developing new tools?

Or you could copy them to a server location. And then you reroute Revit or whatever application you're using to look into that folder-- sort of works. But you have to hack into a lot of machines and change settings every time.

So it could work. Yeah, it's not the best. Or something a bit more professional, which would be, for example, creating a package on some sort of deployment system such as, for example, Microsoft SCCM. That works. That's, again, a bit more professional.

But the thing here is what happens when you have to fix a quick bug or you release a new feature? You have to repackage everything up. You have to re-upload on SCCM. You have to ask everyone to install the new version. And that doesn't go smooth very often, let's say.

In terms of quality-- again, I've seen a few approaches. Sometimes you have that experienced Revit guy. And you use him or her to test. But of course, like in companies usually, Revit guys, especially experienced Revit guys, are quite in high demand. So it could take some time, even weeks, to get hold of them and get some time to test what you've done.

You might want to be a bit more thorough. And you get a mixed group of people-- and more experienced, less experienced, interior designers, architects, whatever. Of course, this is becoming exponential. Because it's not only one person that you need to take. But it's more than one. And you need to get their time. It's almost impossible most of the times.

Or this is something that I've seen as well. And it's very frightening. You just release without testing. So these are the scenarios that prompt us into looking into what I'm going to show you today.

A few assumptions before we start so that I don't receive bad looks here, especially if you are a software developer that work professionally at Amazon or Facebook or the likes. Keep in mind that what you're going to see today is something that has been made by architects for architects.

This means we're not professional software developers. And we might be simplifying a few things. And we might not be entirely orthodox. But trust me, this workflow works. And it's something that we use a lot at Woods Bagot. And it saves us tons of time. So it's something worth seeing.

Also, we're going to talk about DevOps, but specifically for Revit plugins. This is what we do. We're not going to talk about web applications. It's more similar to what happens with desktop applications. We're not going to talk about that either just like Revit plug-ins.

And because we are in the Revit environment and how often happens in design environments, we're Microsoft-based. We're using C#. We're using Visual Studio. We're using external and internal NuGet packages. And final assumption is for this specific workflow, we're using pyRevit. At Woods Bagot, we're huge fans of pyRevit. It makes it very convenient for us to do some very important things we're going to see towards the end of this presentation.

Big kudos to Essen and all the contributors for pyRevit. It can be adopted not to use by pyRevit. We use something similar in Rhino as well. So the general idea works. In this case, we're going to talk about what you've just seen in this slide.

So we mentioned it a few times already. What does DevOps means, and why is it important for you? And why should you pay attention to this presentation? Word DevOps is a combined word that joins dev, which means the software development team and ops, which is the IT operations team-- so the people that create the software and the people that usually are in charge of deploying the software.

What you're going to see, we're not just strictly talking about something technical, something practical. DevOps is really a cultural shift. And as you can understand from the name itself, DevOps, we're trying to remove silos. We're trying to make everything more efficient removing the silos that encapsulates the developers and the operations team, the security team. It very much depends on how your company is structured. But we make everything smoother let's say.

And in this way, because you're merging these two historically-separated figures, you're empowering the developers to have a more direct relationship with the users. And, of course, it brings a lot of benefits. Because direct communication means also more fitting tools and that sort of things.

So why is it important? On the right, you see a list of keywords of improvements that DevOps will bring. But basically, what we're saying is that you're setting up a process that is reliable. And it works every time. It's set to run when it's triggered.

You're bypassing the IT team, not the IT team is bad. It's an extra step that will take extra time. So with this process here, you're speeding up everything. Because, again, it's an automated process. It's triggered. It will run. It will deploy to people. And actually, it will deploy to the right groups of people. You might have different tools for different people. This will just work.

Also like from the scenarios that we're describing at the very beginning, if you have to deploy a quick fix or a new feature, this will happen not only more rapidly in less time, but also more frequently. Because this allows you to work on smaller features, on basically smaller bunches of code in your solution and release and get feedback from the users.

So you get better and more advanced tools. Because the users will try it instantly and let you know what they think. And you will gain more transparency towards the user, which brings to more credibility towards the software development team, which, at least to me personally, is one of the biggest challenges-- like being the reference figure when you have users working-- architects, designers, whatever-- if they think of something, they will come to you and say, hey, can we please develop this? Because they know that you are deploying the right tools with the right timing.

And the last thing, which is one of my favorites actually, is improved collaboration. With this process, standardized and reliable, you're creating a comfortable environment of confidence towards the developers team. Because there is automated testing-- we're going to see it in a few minutes throughout the presentation-- it still needs human checks.

It's not just like it just doesn't do everything on its own. It needs some sort of like conscious decision made by a human, which is a good thing. But it drastically reduces time. So you're basically creating a successful environment where to work, which is extremely important of course.

DevOps is a process. It's actually a circular process that repeats itself over and over. But this is how it looks like. So you have the coders, the developers that work on smaller features or bug fixes, whatever it is, on their branches of GitHub or Bitbuckets or whatever it is. When they commit the code that they created that they worked on to this version control host, something will trigger. And it will start running the CI/CD pipeline.

This pipeline will do a lot of things. You set it up. You decide what it does. But, basically what we have it doing is it builds the solution. It creates the DLLS, the library files. It includes all the internal and external NuGet packages. It versions the files. It signs them. It tests them.

If everything goes fine at this point, then deployment will happen. And all the groups, all the users will receive the files. They won't even notice. They will just see either something new in their Revit or something has been fixed, and they didn't even realize it was broken, which is great.

There is one extra step, which is actually fundamental for a DevOps environment, which is monitoring. But we're not going to talk about it today. Basically, you are checking you created something. You deployed it. You're also checking is it being used and how. But we're not talking about it today, it would take too long.

So we mentioned this already-- CI/CD, what does it mean? So it stands for Continuous Integration, Continuous Delivery. Continuous integration is basically the developers merging smaller code batches into the central online repository. And this will trigger the CI/CD Pipeline. So we're going to see how it looks like.

This pipeline, as we said before, it builds. It tests. It does a lot of things when it is triggered. We're going to see what triggered means as well.

In this way, especially because you're testing your plugin, you're finding bugs quicker, which of course, like most of the times you realize you're not even deploying the bugs. You find them before deploying. And you fix them. And you only deploy when the code is fine.

So you're increasing the quality and the reliability of the tool. Again, you gain credibility towards the users, which is fundamental. Yeah, you reduce the time that is spent in this space. Because you don't have to do much anymore. You're just pushing the code to a repository which you will do anyway. And because it's faster, you're increasing the feedback as well.

So again, you have more fitting tools delivered to your users-- great thing, of course. CD stands for-- there are actually two versions of CD. What we do is called continuous delivery. This means that the deployment happens to a test environment or a test group of people as well.

But then it requires, again, as we were saying before, a conscious human decision to deploy to production-- so it's deployed globally to everyone. There is another philosophy, another different section of this. It's called continuous deployment where actually everything is automated. You don't need human interaction for it.

That's not really something very widespread outside in the software development world either. I'm not sure who uses that, to be honest. But definitely, we're not covering it today. So let's have a look at what we're going to see in just a few moments.

This is what pretty much like a CI/CD Pipeline looks like. First of all, you would set up the pipeline. We're going to see. You're going to install the NuGet packages. Otherwise, you're not going to be able to build the solution, which is the third step. Then after this, really, all the other steps are sort of like optional. But I would say it's really best practice. You'll see why.

You're going to version, the DLLs, the library files that you're about to deploy. You're signing the files. You're testing the plugin that you're releasing. And then at the end, deployment will happen.

So just a few things before we start-- what you will need to get this to work. Any sort of remote Git account-- GitHub BitBucket, GitLab, whatever you're using already, that will work. Then an Azure DevOps account-- we are using-- at Woods Bagot we use Azure DevOps.

You don't need to. There are other options. AWS provides options. There are many other, providing different options. But what you will see in this presentation is what Azure DevOps looks like.

Also, you're going to need a self-hosted Windows Virtual machine. And that's because for the automated testing part that we're going to see in a few moments, you need to have Revit installed. We're going to give a little bit more detail in a few moments.

You're going to need a code certificate so that you can sign the code. And we're going to use pyRevit. Again, I've already said that-- big fans. And this is what this presentation is using and what Woods Bagot uses. Again, don't need to. But we think it's really, really helpful and really convenient.

So let's actually have a look at the practical bits. How does it actually look like? So following the steps that were saying before, first of all, we need to set up the pipeline. It's extremely easy. You go to Azure DevOps, the Azure portal. And you create a new DevOps project. There is a button for it. You click it. You insert name and probably some other information. Don't remember exactly what.

And it will create a project for you. Inside of the new project, you're going to create a new pipeline. It's called pipeline like the CI/CD pipeline. And the first thing that the new pipeline will ask you is to connect to a remote repository. So that's why you need the GitHub, Bitbucket, whatever online repository that we mentioned just a few moments ago.

Once this is done, Azure will automatically create for you a YAML file, which is basically a markup file where you're going to define all the steps that you're going to perform. So what you see on in the screenshot on the right is actually what the YAML file looks like. In this first section of the YAML file-- so at the very top-- what you see is you have to define the trigger, which is what we were mentioning before.

This is usually a commit or push action or a merge to a specific branch. So as you can see there, not sure how clear it is, but on line seven, you're defining master or main or develop or release slash. When a push or a merge hits that branch, this pipeline that we're about to see fully, will start and will start performing all the actions that you define here.

Then in pool line nine, you have to define which machine is going to run the pipeline. It can be a Microsoft-hosted virtual machine. That's not in our case. In our case, we're using that self-hosted Windows Virtual machine with Revit preinstalled. This is very important.

And then you define a few variables like where's the solution, what's the build configuration, the Git version that we're going to see in a couple of steps? And that's it. The setup is done.

Then we're starting to define the actual tasks. They're called steps in the pipeline. And the first one that you need to do is setting up NuGet. NuGet is really extremely important of course. Without the external packages, the solution won't build. So that's why, first of all, we installed the NuGet tool, which is the tool that will actually do the installation of the packages.

And then you use a restore command. It's actually called Restore. And that will automatically install all the external packages that your solution is mentioning. You also have-- and this is pretty cool-- the option to add this vstsFeed parameter to the NuGet command. And that way, if you have-- for example, we do have them in Woods Bagot, if you have internal packages, for example, you have created your own objects and classes and methods that you use on multiple projects, and you created an internal package for it, then you can install and build those internal packages just through this vstsFeed parameter. Super useful.

Then it's basically all set up to build the solution. And building is probably the simplest of the steps. It uses MSBuild or VSBuild to build the solution. And it's really identical to what you would do to build locally on your machine. All the setup happens actually in the csproj file. If you're familiar with Visual Studio solutions, you surely you're familiar with csproj files.

The only real difference that we have in this case here is mentioning-- sorry, pointing to the Revit APIs. If you're familiar with Revit development of plugins, when you install Revit, it also installs the API files, the DLL files, Revit API and Revit API UI.DLL locally on your machine. And when you start working on your plugin, you just mention them locally.

This will fail in the pipeline. Because you don't necessarily have the APIs installed. Or maybe they're installed under a different file path. So what we do when we develop our plugins and we know that these are going to go through a CI/CD pipeline, we reference an external NuGet package. We use ModPlus. But on the NuGet package manager, you'll find plenty of those. So any will do really.

Once this is done, these steps are optional as we're saying. But they're quite-- I would suggest they're best practice and quite useful as well. One is semantic versioning, which is an extremely interesting concept that automatically versions your plugins based on the Git history. So it tells you version 1, version 2, version 3, version 4. It's a bit more complex than that. It's more like 0.1.2 whatever.

And basically, you will need to have at least a main or master branch and a develop branch. And in that way, it will just go in and calculate all the actions that happened in your repository and create a version number that is going to be applied. That's the reason why we're using that GitVersion variable at the very top when we're setting up the variables.

Then you set up the GitVersion tool. You basically you install it. And then you run it, which is what happens on line 24. And that calculates the version of the plugin.

Then last step of versioning, which is actually part of the build step in on line 46 when you have that version and var, you're saying, OK, when you build, assign this version that you just calculated to the DLLs that you are creating-- extremely useful. You can keep track of the versions. You can know your user, which version are they using. So very, very handy.

Another thing that's not just handy, it's quite necessary, I would say-- code-signing. And this is just really because it avoids the annoying pop up that you have in Revit when you haven't signed your code that asks you if you want to-- if you want to trust the developers of that tool. If your code is signed, then you don't need to do this.

And of course, it's really useful. Because it's gives a signature to the file so that the users know that from when it's being created to when it's been released, the file haven't been corrupted. And they're not doing malicious actions.

You have a few options. You can load your certificate your code certificate on the pipeline. Or an even better solution would be using Azure Key Vault, which is what we do at Woods Bagot. What you see there on the screenshot on the right is actually how you do it with the local certificate installed on the pipeline. Because otherwise, if I had to show you how to use Azure Key Vault, that would take too long.

But trust me. It's really simple. There is plenty of documentation online. And the problem that Azure Key Vault solves is that you don't have to install the pipeline-- sorry-- the certificate on every pipeline that you create. It's just there is one single source of truth. And all the pipelines are going to reference that certificate every time they need it.

After this-- and this is like probably the most interesting, the most useful, and the biggest time-saver of this presentation-- automated testing. So we're not going into a lot of depth with this topic. Trust me, it's a whole universe. So what I would suggest after this, if I caught your interest and your curiosity, go and find some documentation about some literature about the philosophy and practices of testing, automated testing. But it would really take too long here.

What we're seeing right now with Revit, probably not really 100% unit testing. Because unit testing assumes that you are not relying on anything external. Even if you had a connection to the database, you would fake it. It's called the stub. And you would fake that connection.

Because unit testing needs to be extra fast. So even the 30 seconds that it takes to connect to the database and receive an answer, that's not admissible. So it would be just not called unit testing. And that's why this is sort of like in between unit testing and integration testing. Why?

Because we're using actually Revit to run the test. But once it's inside of Revit, it is unit testing at that point. Let's call it automated testing to not incurring to any confusion. So to do this, we're using the RevitTestFramework that the Dynamo team created some time ago. Thank you Dynamo team for that. That's extremely useful.

That's not your only option. You have other options as well. I know that the Speckle team has created something for unit testing as well. So you could use that. At the time when we're doing this, that wasn't out yet. So we haven't really tried it.

But you have at least a couple of options you have. And what really happens-- what is automated testing? Basically, when you're writing your code-- for example, you're writing a plugin that creates sheets in Revit-- so at some point, you are definitely writing a method to create one sheet.

So what you would do is at the same time, you would go to the testing project in your solution. And you would write some code. It has specific ways of doing it. Again, find some literature online on how to do this. But basically, you're writing a test saying run this method and check if it does what it's supposed to do. Of course, you tell the test what it's supposed to do. And this is it.

Basically, what happens, you build the code. It also builds the tests. And then you're telling-- because you have this Windows VM with Revit, the VM will start Revit-- actually, if you have access to the VM, you can also see that happening. Sometimes, I still do it. It's quite fascinating.

You just have Revit popping up doing all sorts of things and then closing down. At that point, it means that all your tests have been run. What happens now is you end up with an XML file with the results of all the tests. So for example, it will say test one passed, test two passed, test three passed.

But for example, if test four didn't pass, it failed for any reason, then the pipeline stops. And this is great. Because it means that there is probably a bug somewhere that you didn't realize you were creating. And so it doesn't release a plug-in, an application that has a known bug in it.

So what you do is you go back. You check which test didn't pass, which test failed. And you try to understand why. And then you fix it.

And then you run it again. And when all the tests pass, then at that point, it will go to the next step, which would be deployment.

You are also-- because you have that XML file that you're passing back to the pipeline, you're also able to visualize-- as the screenshot on the bottom left of this slide-- you're able to see in a nice graphic way how many tests passed and how many failed and which one failed and error messages and all sorts of things, which is extremely useful. And again, this is really what is saving you tons of time. Because you still need people to test the plugin that you're releasing.

But you're saving them tons of time. Because they're not focusing on the small bugs. They're mostly focusing on user experience and UI and efficiency of the tool overall. So that is really, really great and a really important step in my opinion.

Then the final step is deployment, which is, as you were saying, the step where we would use pyRevit. So our one bar toolbar for Revit is built upon pyRevit. And it makes it extremely convenient to release and to deploy mostly because it's based on Git. So really what happens-- if you're familiar with pyRevit, you know already. But if you're not, what happens is you're writing the code on your-- for example, my Wombat solution. And then when I'm done, I hit Push-- well, Commit and then Push.

It pushes to the repository, which is the sort of like Wombat repository that runs pyRevit. At that point, everyone else in my company that have pyRevit and Wombat installed, they will receive the changes. s-- bugs that have been fixed or new tools that have been released, new features, and everything. Extremely, extremely useful, trust me.

And this step here that you see is really just running a PowerShell script similarly to what we were doing with the test, with the test step. And what you're doing is, first of all, you're checking out the correct branch that you want to deploy to. So you move-- let's say, virtually move to that branch, you pull. Because at that point, you want to pull all the changes that happened. Then you don't want to incur in any conflict let's say.

Then you take all the files that your pipeline has built, and you add them to this commit here-- yeah, to the commit. Then you commit with a meaningful message or whatever you prefer. And then you push to the repository.

One important thing to understand here, there are two repositories that we're talking about. There is the solution that we were just working on-- so for example, some sort of more complex plug-in-- I don't know. You're working on some sort of model establishment plugin that you want to go under Wombat. You have the solution for the model establishment plugin. And then you have the Wombat solution or repository.

So what are you doing is you're taking the files from your plugin, the model establishment plugin let's say, and you upload them or you push them to the Wombat repository. At this point, you're basically done. It just works. pyRevit will work its magic. And everyone will get the new software.

You can set it up to deploy to test groups or to everyone or to different groups. But it basically, at this point, it just works, which is pretty amazing if you ask me.

So to look back to what we've just said, we set up the pipeline. We installed the NuGet packages that allows us to build the solution. We versioned the files. We signed them so we avoid the pop-up message in Revit. We test the code saving again probably weeks of time. And then you deploy. At this point, you also do all the monitoring stuff. But this is what we really do.

So in conclusion-- and if you fell asleep and you're only waking up right now-- I want you to go out of this room with this idea. The key takeaways from this presentation is why is DevOps and a CI/CD Pipeline important to you, important for your company and it's going to help a lot?

Well, first of all, you're improving processes. Because with DevOps practices, you're basically standardizing a whole lot of processes from building to deployment, including testing and everything. So you're saving a lot of time. Because you don't have to do it yourself. And also, you're reducing bugs. Because again, we talked about it, unit testing.

So by saving time, you're really saving money to your company, which I guess you'll all be happy about it. Also, you're establishing better collaboration within-- well, on many levels. But like even within the software developers team, they're going to be much more relaxed, work smoothly. Because they're writing the code.

But if they push the code and there is some error, they will find out before it gets deployed and they get submerged by thousands of emails and teams messages saying why is it not working? Why my Revit is crashing every time?

So they're less stressed, because they have this controlled and reliable process in place. And also, it's smoother for them to work with their users because of the same reason really.

So at the end, you're achieving better results, which is great thing to achieve. And so you're deploying tools, plugins, whatever that are more reliable and also in a faster way with a more rapid feedback circle. So you're getting information. You're getting feedback from your users more often. And this allows you to create a more fitting plugin at the end of the day.

So if you wanted to sum up everything in one word, you're winning credibility. And we talked about it already. I believe this is the most important thing, especially in the world of automation and design technology. So this is it.

Thank you very much for listening to this. And I really hope you found it interesting and useful and that you're leaving this room with something to think about. And you'll try to do it for yourself and for your own company. And I hope I'll see you around the conference. Well, again, thank you for listening, and enjoy AU.