Using Fusion 360 to Design Patient-Specific Prosthetic and Orthotic Devices

MICHAEL PEIRONE: Hi, everyone. Thank you very much for attending my talk for Autodesk University 2021, titled Using Fusion 360 to Design Patient-Specific Prosthetic and Orthotic devices. My name is Michael Peirone, and I am the chief operating officer of the Victoria Hand Project. Today, I'm going to tell you a little bit about how the Victoria Hand Project uses Fusion 360 and the Fusion 360 API to develop our own software solutions to make custom prosthetic and orthotic devices.

During today's class, I will first talk a little bit about Victoria Hand Project's work in the field of prosthetics and orthotics and how we design some of our 3D printed prosthetic hands. Also talk about some of the issues that we ran into when designing these prosthetic and orthotic devices for mass customization. And then, I want to cover the solutions that we developed to help streamline our workflows to make the process of making these custom prosthetic and orthotic devices much easier. And as part of this, I will also be covering how we use the Fusion 360 API.

So there's four main learning objectives that I want to cover during today's class. I want to talk about how Fusion 360 can be used by healthcare professionals in a orthotic and prosthetic context and how they can use workflows in Fusion 360 to generate user-specific devices. I also want to talk about how we model designs parametrically and how we use a user interface to allow these clinicians to do quick and easy customization of the prosthetic and orthotic devices. And then finally, I want to talk about the Fusion 360 API and how we use it to create these workflows.

So first, I want to start talking a little bit about Victoria Hand Project's work in the non-profit space to cover the need for prosthetic care and why we are developing these solutions. So the World Health Organization and the International Society for prosthetics and orthotics estimates that 80% of amputees live in developing countries, but only 5% have access to prosthetic care. This means that there are millions of people around the world in need of care.

The healthcare industry in many of these developing countries don't receive the necessary support from their governments, and that means that many people just end up going without care. This happens for a variety of reasons, such as the high cost of prosthetic and orthotic devices, a scarcity of trained professionals to work with the patients, and a lack of infrastructure to actually produce the devices. This means that many of these people just end up living life without a prosthetic device.

North America, we have a very positive attitude towards limb loss and disability, but this isn't always the case in many developing countries. There's often a stigma associated with limb loss. And for some amputees or disabled individuals, it might be very difficult to gain or keep employment. It's going to lead to low self-esteem and social exclusion from the community.

There's also many daily hardships associated with limb loss and disability, such as difficulty with taking transportation, dressing, cooking, feeding themselves, personal hygiene. These are things that many of us just take for granted. So the Victoria Hand Project has a mission to provide low cost prosthetic care to amputees in developing countries and low income communities. We're also working on developing low-cost 3D printed back braces for individuals suffering from scoliosis.

We're currently operating in 10 countries around the world, as shown on the right. And in each of these countries, we partner with local healthcare professionals and technology experts. And we train them in how to use leading edge tools, such as 3D printing and 3D scanning, to actually produce these devices. This helps empower the partners that we work with and fosters a sense of pride within the community. It also lays the groundwork for sustainable ongoing care.

So this is Bin Amin. Unfortunately, he lost both of his hands and both of his feet in an accident when he was very young. We actually provided him with his own hand in February 2020, when we expanded to a new site in Kenya. This is actually the first time he'd ever written in his life. I was there when he was fit and I just remember how happy he was to receive the hand. He just wanted to play, show off to his mom. Just do things that every kid wants to do.

This is Ijabu. She lost her hand due to an infection, and she had to grow up learning how to go to school, go through her daily life with only one hand. And this is Jacob. He wanted to use the Victoria Hand for writing and at work.

So the Victoria Hand Project's primary focus is on the design and the development of the Victoria Hand. This is a 3D printed prosthetic hand that's specifically made for low-income communities. Has a number of unique features, I showed on the right hand side, which help people while they're at work, at school, or at home.

The whole system, including the hand, the wrist, the socket, and the harness is only 100 US dollars. This greatly reduces the cost of prosthetic care in developing countries. I should note that we also do not charge the patients for the device, due to the great need.

An integral part of the Victoria Hand system is the custom-made prosthetic arm socket. We wanted to develop a arm socket which is comfortable, but it also looks natural on the user. We want it to look-- or to be comfortable because if it's uncomfortable, the patient won't want to wear the device no matter how nice it is. We also want the socket to look very natural so the patient feels more comfortable when they're using the hand in public. And it helps empower them to become more independent in their daily lives.

The workflow for creating the Victoria Hand socket is simple, yet effective. First, the patient's limb is 3D scanned to generate a 3D mesh on the computer. Next, a premade mesh-- a premade socket is selected from based on the patient's anatomical dimensions. The 3D scan is aligned within the socket, and then it's cut away. This leaves a socket with the exact shape of the patient's limb.

Finally, the socket is 3D printed. This whole process could be done in one day by our partners, and it greatly reduces the time required for people to receive their prosthetic device.

So here's a quick video showing how we create these custom arm sockets in a program called Meshmixer. So first, the 3D scan of the patient's limb is loaded in, and then a socket is loaded in, which is selected from based on the user's measurements. The 3D scan will be aligned within the forearm socket. And if the socket is a little too small, it can be removed and a larger socket can be brought in.

It's very easy for the clinicians that we work with to use this program to navigate through 3D space and to align the 3D scan in the socket. They will align the 3D scan in the socket based on their clinical expertise and also the needs of the patient. The clinicians are also able to use a transparent view to see how the 3D scan fits into the socket. They want to make sure that the 3D scan doesn't interfere with any of the features within the socket.

And if they determine that the fit is good, they can perform something called a Boolean Difference, which will cut the shape of the person's limb, the 3D scan, out of the socket, leaving a cavity the exact shape of the user's limb. Finally, before they export the socket, they want to just use the smoothing brush so they can clean up some of the edges, make sure it's not sharp, and it's very comfortable on the user.

I just want to point out some of the features of the socket. There's some slots which are used for attaching the socket to the harness. And on the right hand here, right hand side, you could see the cable guide, which helps guide the cable from the prosthetic hand to the harness so the patient can use it.

So as we've seen in the video, there is a number of pre-made sockets, which are selected from based on the patient's anatomical dimensions. In the early days of the Victoria Hand Project, we designed the socket in a CAD program and we had to manually create each side of each size of socket ourselves. This meant that the patient's dimensions had to be entered into the equation manager, and then the socket is exported.

There's three primary measurements which are used in the socket. There is the wrist circumference, the forearm circumference, and the overall length of the socket. Using these three dimensions and expanding to accommodate various patient sizes, we created a library of thousands of different types of sockets.

The problem was that each of these sockets had to manually be created. This meant that one of our students would need to enter the dimensions into the equation manager, export the socket as an STL, and then they would enter the dimensions for the next socket, export it. This was a very time intensive and labor intensive process. We also avoided doing any updates to the design of the socket, because that would mean that we would need to create each socket again.

To overcome these bottlenecks, we redesigned the socket in Fusion 360. We found that using Fusion 360 gave us a lot more control over the design and allowed for a much more natural looking shape of socket. Using Fusion, we were also able to run a script which would automatically create each size of socket and export them.

This meant that we didn't need to manually enter the dimensions for each socket. It greatly reduced the time to create these batches of sockets. And we also didn't need to worry about updating the design of the socket, because we could just run the script again, export all the sockets, and it would be much easier.

So this process in Fusion 360, it worked well, but it also had its own limitations. We couldn't easily cover the entire range of socket sizes for all the patients. We were often asked by our partners if we could create a custom socket for a patient based on their dimensions, because they fell outside of the ranges of the sockets that we had made. For example, Angel in Nepal is very small, but Samer in Egypt is quite large.

So we could expand the range to fit people like Angel and Samer, but we found there's still people that fell outside this range. There are very tall people with very small, thin arms, and very short people with very wide arms.

We already had thousands of socket sizes and gigabytes of STL files, so when somebody had to find the size of socket that they wanted, they would have to go to online folders that are sorted by size, go through them, find the socket that they wanted, download it, try it out. If it didn't fit, they would have to find another size.

So this was a very time consuming process, and they were also finding the closest-fit socket, rather than something that is the exact size that the patient needs. So what we wanted to do was, we wanted to create a program which would allow the clinicians to create sockets to the exact size that they wanted.

At the same time, we also started working on a research project to make 3D printed back braces for people suffering from scoliosis. So the base shape of the brace is made by entering the patient's anatomical dimensions, and then there's also other adjustments that can be made to the brace.

So due to the variety of the sizes of different patients and all of the adjustments that can be made, it's supposed to be a very difficult problem. For the socket, there's three primary dimensions which are used to make the brace. But for the brace, there's more than 20 primary measurements. And then there's additional adjustments.

There's no feasible way that we could have premade off-the-shelf solutions like we did with the sockets. The library of different sizes of braces would just be way too large, and it would be way too difficult for the clinicians that we work with to find anything.

So we wanted to do was, we wanted to create a program which would allow the clinicians to design custom-made braces. So we wanted these software to be made using the patient's anatomical dimensions. We didn't want to have to rely on 3D scanning to do this though, because 3D scanners are very expensive and we don't have access to them in all the places that we work in.

We also wanted the Fusion program to allow for quick and easy adjustments to the model. It might not be that hard for an experienced CAD user to jump into Fusion, make different changes to the brace, but the people that we work with, the clinicians, they're not CAD users. And we didn't want to have to rely on them to learn CAD to carry through with this project.

It's also not a scalable solution for our engineering design team to be making one-off designs for our partners. We wanted these braces to be made with minimal input from the clinicians that we're working with. For many people in North America and many of the clinicians in North America, they have helpers in the workshop who can do most of the tedious labor.

But the clinicians that we work in developing countries don't always have somebody who can do this tedious labor. What we want to do is, we wanted to create tools which would allow the clinicians to speed up the process. And the tools would do the hard tedious work, allowing the clinicians to help more people.

Fortunately, we were able to get some help on this project from the Fusion team, thanks to the Autodesk Technology Impact Program. Three Autodesk employees, Brandon Cramer, Kevin Acker, and Melissa Kaner consulted us on the different tools that we could use and the design workflow that we could use to create these socket and back brace projects.

They also showed us how we could implement the Fusion API into our workflow, so we could create our own software and programs to create these devices. Their insight allowed us to greatly speed up our timeline, and it allowed us to reach our milestones much more quickly.

So for just a little bit of background on the Fusion API, it's an application programming interface, which means it's an intermediary between two programs. In this case, it works between a script that someone develops, and Fusion itself. The really great thing about the API is we'll automate features within Fusion, such as selecting a sketch to extrude by a certain amount.

It's also a very valuable tool for doing repetitive actions and tedious steps within the workflow. You can start API program and run through repetitive workflows much more quickly with minimal input from the user. It's also really great for non-technical users, because you're able to develop a user interface and a program, which will allow the user to create designs without actually needing to enter the Fusion design space.

So for technical users, people that have experience with CAD, it might not be that difficult to do many of these updates. But then, as I mentioned, the people that we work with are non-technical users. It's also really great because you can create parametric designs. So they have parameters that can be scaled depending on the user's input combined with the user interface, and makes it very quick and easy to adjust the size of the design.

In the next few steps, I'm going to show you some of the APIs that we developed for our socket and back brace programs. There's also a lot of great resources online on the Fusion site, such as code examples, syntax, and user manuals that you can check out to start to create your own.

So this is an image of the Socket Designer program that we created. We have it working with Fusion 360 and our own sort of software to create the sockets. It's an add-in, and we have a user interface that comes up over top, and it makes it very easy for the clinicians that we work with to create the sockets. So now, I'm going to show you a quick video of how a clinician would create the Socket

So within Fusion, along the top bar, there is the tools tab. And then under there, there's add-ins, where a user can select scripts and add-ins they have added, or go to the Fusion 360 app store to buy or download their own. So under the scripts and add-ins tabs, there's a number of pre-loaded scripts. Or you can go down to create to begin to create your own program. Here, I'm running the socket add in that we created. Now it's running in the background.

And then I'm able to start the program that we developed, and it comes up over top of Fusion. Here, the clinicians are able to select if they want to make a new socket or update an existing socket for a patient. And then they are able to select the type of hand model that they want and whether it's a left or right hand. And then this model will open up in Fusion 360 in the background.

So here, the clinician will enter the patient's dimensions into the fields. This will create the socket size, and these dimensions are taken from the patient when they first visit the clinic. If the dimensions that the clinician enters are outside of the recommended range, they will get a warning. But the clinician can also override this warning.

Here, the clinician is able to select if they want to add more space around the elbow, which is sometimes required for some patients. And then finally, they're able to review the dimensions and build the updated socket.

So this is the socket that we developed in Fusion 360. As I mentioned, it's a parametric model, so we'll scale based on the user's inputs. If the clinician is happy with the socket, they can build it, or they can go back to change some of the dimensions. So when they go to Export the socket, it will be saved as an STL onto their local device.

So we also developed a similar program for creating the orthotic back braces. Since this is a ongoing research project, I'm not able to show you a video of how it actually works, but it's very similar to the socket software. Here on this page shown on the right, the clinician will enter the patient's dimensions, and the base shape of the brace will be built. There's also additional pages where other measurements can be added, or adjustments can be done, such as twists, shifts, or bends.

So the really great thing about using the API is, we'll automate adding in features at the end of the workflow. For example, we add in holes into the brace, because users in the past using conventional braces have complained that they get very hot. So what we wanted to do was add a whole pattern throughout to help keep the brace cooler and allow it to be more comfortable.

So what a clinician will do is, they will select the density of holes that they want to add from a dropdown menu. And then there's pre-made sketches within the brace model. These sketches are then updated based on the clinician's input, and then the sketches are used to perform a cut-extrude through the brace. And this will cut all the holes throughout the entire brace.

This greatly reduces the time for actually adding in the holes. We previously had these holes built into the brace already, but each time that the clinician wanted to update the design or move to the next page, the entire brace would have to be rebuilt again. This was a very slow process. It could sometimes be 10 minutes between each page, and this is not something that a clinician wants to have to sit through.

The really great thing is, now with the API, the clinician can enter all the dimensions, do the adjustments that they want, enter the density of holes that they want to add, and then they can just press export and walk away. And the Fusion API will do all of the hard work behind the scenes.

So we've shared this program with some of our research collaborators so far, and they're very happy with it. They like how easy it is to create the braces, how nice Fusion 360 is for creating the shape of the brace that they want, and yeah. We look forward to continuing our work on this research project.

There's also other programs that we want to begin to develop using the Fusion API. So we want to be able to do the entire end-to-end workflow of creating the socket in Fusion 360, similar to what I showed in the video in Meshmixer earlier.

This means that they would be able to add the 3D scan in, align the 3D scan into the socket, which is built based on the patient's dimensions, and then perform the Boolean Difference to cut the 3D scan out of the socket, do some final touch ups, and export it. This would make it much easier for the clinician, because they wouldn't need to go through all the little steps in Meshmixer.

We also want to be able to create a transhumeral socket workflow. So a transhumanal socket, as shown on the right, is for people who are missing their arm above their elbow. And we are currently trying to do this in Meshmixer, but it's posing to be a very difficult program.

This is because the socket must be made using a scan of the patient's upper arm and their shoulder, and then using that scan to offset it and create a shell, which will go over top of the shoulder. And then it will help suspend the socket on the patient.

So this has posed to be a difficult problem, because it requires a lot of user input, and there's a lot of steps required. We want to be able to do this workflow in Fusion 360, even though there will still be some user input required from the clinicians. It will still help automate some of the steps in between, which will make it much nicer and easier for them.

Finally, we want to be able to begin implementing T-spline functionality into the brace program. So T-splines are an advanced surfacing tool which allow for very complex and organic shapes. The way that this is done is, a user will select the faces or the nodes on a quad mesh, and then they can push and pull these areas. And it allows for shapes that can't really be created easily by using surfacing or other sorts of tools in Fusion.

This will be very nice for the clinicians, because they would be able to go through the entire workflow and easily add adjustments that they can't normally do. So unfortunately, right now, the T-splines are not added into the API yet, but we hope that they will be in the near future.

So to summarize, the Victoria Hand Project designs and deploys low-cost prosthetic and orthotic devices. We previously made the sockets and back braces for mass customization using other CAD programs, but it posed to be very difficult, and we transferred the design over to Fusion 360. This worked well, but it still didn't allow for full customization, and it required a lot of user input in the CAD programs.

We also made libraries of different sizes of STLs of sockets, but it wasn't always able to fit the patient correctly or optimally. And then this also isn't a proper solution for creating something as complex as the back brace.

So what we did was, we used the Fusion 360 API to create our own user interface and design software, which would allow clinicians to go through the steps of creating the sockets and the back braces themselves without needing to actually jump into the programs. We're also able to automate some of the features of the workflow, which makes it much easier for the clinicians and helps reduce the time required from them.

I would also like to highlight the work of some of the other team members that worked on this. So Dr. Nick Dechev first started Victoria Hand project as a research project out of his lab, and now it's grown into the company. It's working in 10 countries around the world. Dr. Dechev provided his extensive Python knowledge to help get the team started on this project.

And then Kelly Knights, she is a biomedical systems designer with us, and she taught herself Python so she could help create some of the user interfaces and some of the background backend functionality in the socket program. And then Derek Bell is one of the-- he was the primary software developer on this project. He did most of the work in the API and developing a lot of backend functionality of the brace program and the socket programs.

So thank you very much for listening to my presentation. If you'd like to learn more about the Victoria Hand Project's work, you can visit our website, or some of our social media pages, as shown here. If you have any specific questions about some of the workflows or anything that we did, you can also reach out to us through the contact page on our website.

So thank you very much. And I will also be taking the Q&A after the live-- or after the Autodesk University session. Thank you.