Following a successful presentation at Autodesk University 2023 in Las Vegas, USA, Ziqing Liew, Head of Digitalisation, Development & Delivery (3D) at Sunway Property describes how he and his team were able to integrate ChatGPT with Autodesk Construction Cloud’s search function.
Sunway Group (Sunway) has built some of Malaysia's most significant landmarks, including Legoland Malaysia, the Kuala Lumpur Convention Centre, and Malaysia’s first elevated Bus Rapid Transit system (BRT-Sunway Line). Our construction arm, Sunway Construction, is Malaysia's largest pure play contractor.
From early on, Sunway differentiated ourselves from the competition by investing in building information modelling, or BIM. We have pioneered the implementation of Virtual Design & Construction in Malaysia.
More recently, we adopted Autodesk Construction Cloud as our common data environment (CDE), and began the process of migrating our knowledge portal to the new platform.
Our onboarding of Autodesk Construction Cloud closely coincided with the public launch of ChatGPT. When we looked at its capabilities, we saw an opportunity to augment the search function in Autodesk Construction Cloud with ChatGPT that would improve the way the wider Sunway team interacted with search results.
Our challenge is that we have a large database of knowledge articles. The team sometimes struggles to locate files that are directly related to their search intent. Searching any large database by keyword naturally returns multiple results that, while they are related to the keyword entered in the search bar, don’t relate directly to the intent of the searcher.
For example, searching ‘What is involved in laying interlocking pavement?’ would return results that answered the question, but they can be buried by results that don’t directly address the intent of the searcher.
My colleague William Wong – Sunway Integrated Properties Senior Engineer – told me about a lightbulb moment he had when he found a solution. He realised that using the power of OpenAI, and its ChatGPT API, it was possible to enable semantic – rather than lexical – search within Autodesk Construction Cloud.
This meant that instead of matching characters and words used in the search term to those in the database, the AI could understand the meaning and intent behind the user’s question.
By parsing the semantic meaning, the tool could understand what the searcher wanted to know, and return results that matched the user’s intent more closely.
While the technology was new, the challenges faced by William were familiar. How could he ensure that the results returned by the tool were reliable, trustworthy, and presented the most useful results first? ChatGPT is a ‘black box’ – how it chooses which results to present are not visible to the user, or indeed the team developing this solution for Sunway.
William’s early experiments involved allowing the AI to make all the decisions regarding which results to return to the user. And while this more often than not gave the answer that the user was looking for, the system would sometimes produce interesting results.
So a second iteration was created. The initial search would be carried out by the traditional keyword method present in Autodesk Construction Cloud, with the ChatGPT AI then ranking those results based on its understanding of the searcher’s intent.
The team is already experimenting with including images, graphs, and links to documents to improve the useability of search results. Now, search results pages provide direct links to knowledge articles and PDF documents, augmented by a GPT-generated summary of the combined results.
To ensure accuracy of both search results and summaries, the AI model only uses data from the Sunway knowledge bank for its summaries.
William developed an entirely automated process that automatically prepared Sunway’s knowledge bank for use by the AI. Using a webhook provided by Autodesk Construction Cloud’s API, every time a file is created, altered or deleted, the change is reflected in the AI’s model.
We are now in the process of optimising the system. First up – fine tuning the model.
“We want the answers to be more specific, more geared toward the property and construction industry,” says William.
This can be achieved through a separate OpenAI API, and the team is optimistic about improving the AI-generated summaries even further.
Another improvement in the team’s sights is analysing images. Optical character recognition has already taken care of the text in the database’s PDFs, but of course these documents also contain images that contain valuable information.
It’s a similar story with drawings. William explains the difficulties of teaching a machine to identify elements of plans and drawings.
“Before integrating with Autodesk Construction Cloud, we had a project related to analysing 2D drawings. For humans, one glance at a drawing is enough to understand where a kitchen is, for example,” he explains. When it comes to machines, it becomes very hard for the algorithm to make those same assessments.
“It’s easy for us to identify the kitchen even if it's not labelled, because it looks like a kitchen. AI finds that very difficult.”
And of course, the ultimate target is Revit and Navisworks files – “that would be incredibly useful for the whole construction industry,” says William.
Looking even further into the future, William thinks implementing AI and integrating it with Autodesk Construction Cloud is the easy part.
“The hard part will be convincing people to use it. We have to show how AI is going to change people’s lives and show them that AI is not actually replacing our jobs. AI is definitely the future, so let's embrace it.”
