How machine learning is used in the building industry today

7 min readNov 16, 2021

You can access the original article through this link. Don’t forget to explore our blog for additional informative articles on technology and the building industry!

Last month aec+tech invited industry-leading design technologists, data scientists, and machine learning (ML) experts to discuss the applications of machine learning and artificial intelligence in architecture, engineering and construction (AEC) today and towards the future. Machine learning is a branch of AI — artificial intelligence — that focuses on using data and algorithms to mimic human learning and improve its accuracy over time. Read below to learn more about our speakers and their work, in addition to a summary of the discussion.

Leland Curtis is the former Co-Lead of Computational Design at SmithGroup. Leland implements Machine Learning into his design process through one application of ML called surrogate modeling. Oftentimes, design analysis can only be performed after the design option has been modeled, but with parametric modeling and more computational power, there is the ability to iterate through all possibilities of a design problem when modeled in the right way. In surrogate modeling, users are taking data from their parametric model and creating a representation of all the possibilities. The goal is to have these surrogates, being incredibly light-weight and fast, to then power new design interfaces — “Instead of answering how does design A or design B perform?” designers shift to asking questions such as: “What is my design problem, what are the possible paths, and what are the impacts of various design decisions?” One of Leland’s projects utilizing surrogate models while at SmithGroup, called ‘SG-PV’, leveraged a surrogate modeling process to encapsulate the functionality of the free online Photovoltaic analysis website PVWatts within a grasshopper module. This allows the functionality of PVWatts to be deployed directly within the Rhino-Grasshopper environment, enabling a collaborative, real-time interface that delivers analysis at the speed of design. The surrogate also supports generative design and design space exploration workflows. Check out the PVWatts tool here.

Surrogate Modeling Workflow Diagram | SmithGroup

Aprameya Pandit is a Data Scientist working at Perkins & Will IO Lab. Aprameya has a background in architecture and design; his interest in computer science led him to discover computational design. Through working with computational, data-driven projects in his own practice, competitions, and providing optimization services for other firms, Aprameya seeks to position himself between the space of tool-user and tool-developer. As part of the IO Lab team, Aprameya works in the realm of answering — “what will our industry look like in the near future?”. The team develops web tools consisting of common full-stack workflows backed with an ML engine, providing design professionals with powerful tools with intuitive, elegant interfaces.

Chin-Yi Cheng, a Principal Research Scientist and Manager at Autodesk’s AI Lab, has a background in architecture and design prior to studying human-computer interactions and AI/Machine Learning. Chin-Yi’s research focuses on answering, “how can we apply machine learning to help designers?”. Several of Chin-Yi’s latest publications include Building-GAN and House-GAN++, a model developed with external collaborator Obayashi to help users develop floor plans and building volumes using high-level instructions such as bubble diagrams, and graph neural networks to simulate and design building structures. Autodesk AI also works with users outside of AEC such as manufacturing, entertainment, and animation. Play with the interactive demo for House-GAN++ here.

House-GAN++ automating home layouts based on bubble-diagram | Autodesk AI Lab

Theodore Galanos is an intelligence-driven Design Technologist who envisions Machine Learning models enabling entire design processes that were not possible in the past. Optimization and automation should not be the goal — rather, “What can we do more with the time saved?” given the ability to work faster and more efficiently. For some designers, this may mean completing more projects and making increased profits. For Theodore, automation allows innovation that changes the way projects are designed and built. Shifting away from the traditional methodology of dictating what a project should be, architects and designers are challenged to extract information, analyze and evaluate.

Maksim Markevich, the CTO for Kreo Software Ltd, comes from a background in structural engineering. Maksim first began with automating processes from his daily routine, before moving into BIM and product management. Maksim explains that at Kreo they reformulate almost every object generation problem as an optimization problem. If the resulting problem is a discrete optimization problem, then to solve it they use local search methods (e.g. for scheduling problems and generation of apartment layouts), greedy algorithms (e.g. for the problem of inscribing sections into the site), dynamic programming or open-source constraint programming solvers (e.g. for the problem of converting into modular buildings). If the resulting problem is a continuous optimization problem, then usually they use a variant of the gradient descent algorithms (e.g. for the problem of placing windows on the facade). Maksim states that less than 10% of experiments in the ML space make it into production. When a viable experiment enters production, it still needs to be developed and improved substantially. Kreo’s latest product uses computer vision: convolutional neural networks for segmentation and classification of footprints, spaces, walls, doors, and windows from users’ 2D floor plans. After the initial release: working with clients, and obtaining feedback, Maksim’s team shifted to transformer models and then a swim transformer model.

Kreo computer vision tool | Kreo Software

Rutvik Deshpande is an Associate Data Engineer at Digital Blue Foam (DBF) and completing his undergraduate studies in the field of architecture. His work involves using AI to solve problems in the built environment, and developing data-driven design workflows at architectural and urban scales. Rutvik’s team uses ML & Deep learning to transform unstructured contextual and project data into meaningful knowledge that can be used to improve the quality and sustainability of a design. Digital Blue Foam recently launched the beta version of the software, allowing users to increase efficiency and reduce time drastically in early-stage designing and environmental analyses, using their user-friendly sustainable design platform. DBF’s primary clients include real-estate developers, students, and designers.

Climate data for sustainability analysis | Digital Blue Foam

Artificial Intelligence and Machine Learning are shaking design and the built environment. Some economists predict that 47% of work will be replaced or automated by robotics by 2037. Maksim explains how this figure may be misleading — while current tasks continue to face automation, the roles embodying them are simply evolving. Cost estimators using Kreo’s computer vision software do not need to spend time measuring or drawing, however, now need to perform validation. ML is indeed changing how projects are produced, but jobs are changing as well. Leland adds, “what we are doing today will get automated, and will allow us to move towards higher-level, more valuable work. Thus, it shouldn’t be something we fear.

What opportunities arise from automation?

Theodore asks as he explains that optimization should not be the focus since it is limited to the processes used today. Very few projects have the resources to do projects well. If we transform surrogate models into programs, we can then democratize improved workflows that everyone has access to. Leland asks a follow-up, “Which parts of the process should we do better?”. There are many parts of the design process that designers do extremely well, e.g. conceptual design that relies heavily on human emotion and creativity. Yet there are areas designers struggle greatly with, such as sustainable design. Buildings themselves are not incredibly intelligent and the design process is usually poorly informed — very little output for a lot of input. When solely solving for inefficiencies, there is the implication we are trying to automate the work we are doing now. Rather than trying to make the current path faster, we might ask the following:

How can we achieve goal X in a different way given new technologies and tools?

However, architects and designers are resistant to change. Aprameya explains the difficulty of introducing new technologies and workflows into the workplace, and the necessity to convey its value proposition to existing users. Consequently, tool developers are typically focused on meeting current needs and usage rather than offering a 10x solution. According to Theodore, the disparity between technology and adoption has less to do with marketing and product promotion than the quality of the problem and solution. Theodore explains that the citizens of Greece were highly attached to their cars. However, within a week after introducing the metro system everyone was using the metro. People will adapt if offered a good solution to a real problem. One major remaining roadblock in harnessing ML within design processes is the tangibility of data and current models. Chin-Yi explains that current ML representations do not match the vector, BREP, or BIM models representing geometry. In addition, public datasets and researchers are incredibly limited within the AEC industry as compared to other fields including medical or food & beverage industries. Chin-Yi and Leland encourage designers to upload their research onto open-source libraries like Kaggle, a web-based data-science environment, allows users to explore and publish data sets. Data sets and technology need to be democratized for a better-built environment and future —

Thanks to Kevin So.

GAN-Generated Masterplan | Source: Stanislas Chaillou




Keeping up with the latest innovations & technologies in AEC/O (Architecture, Engineering, Construction & Operation) Visit us @