THINKING ABOUT AI AND ROBOTS on construction sites are just some of the things Dr. Biplab Sarkar, as CEO of Vectorworks, Inc., ponders when bringing his focus to the more bleeding edge of innovation within his company. That and how exactly, new tech terrain like data analytics can help his team focus on delivering better solutions to challenges that Vectorworks users have today.
Earlier this year Dr. Sarkar spoke to Architosh about some of the innovations that are taking place behind the scenes at their HQ in Maryland. In this feature, we break out five major innovation themes happening at Vectorworks.
Many Brains, Little Time
A challenge with any company charged with innovation around its products and services is how to capture ideas and critiques from all of its available brain trust. Before Dr. Sarkar became CEO a few years ago, the company initiated an annual week-long event called Innovation Week.
The algorithms are the same ones that are being used in genetic research, where they are trying to look at the particular configurations of genes in pursuit of a cure for a disease.
“For a week the whole company kind of shuts down so that staff can start thinking about newer processes, methods, newer algorithms, newer products—whatever,” says Dr. Sarkar. The company would vote up the best ideas to pursue, but there was one fundamental problem. It would take them years to implement these ideas.
“The problem was we would all go back to our day-to-day activities,” he added. “It was very hard to carry out some of these ideas to fruition.” To address this, when Dr. Sarkar became the CEO the company created a new group of people who do nothing but pursue ideas that are basically “on the fringe,” as he would say. This today is the Vectorworks Advanced Research Group (VARG). “They work on the side and don’t interface with the Vectorworks development teams all that much, says Dr. Sarkar.” When ideas reach maturity and time well with the general market, they make their way into Vectorworks as major new features.
An Almost Clandestine Team
While a dedicated group to take ideation out of Innovation Week was a smart idea, so too was the formulation of the group itself. The Advanced Research Group is headed by Dave Donley, director of product technology at Vectorworks.
This is quite a strong group of people—lots of Ph.D.’s on the team and very strong capabilities.
“This is quite a strong group of people—lots of Ph.D.’s on the team and very strong capabilities,” he adds. “This is sort of a clandestine team, you don’t hear about them much.” The team doesn’t have many Vectorworks responsibilities related to a particular release; they are just charged with pursuing new ideas, both ideas that come from Innovation Week and those fringe ideas that are out on the Hype Cycle. While the group is highly independent, Sarkar as the CEO gets a monthly report from Donley that keeps him updated on what’s happening within the group.
A Dedicated Analytics Team
Along with a dedicated research team, Vectorworks also has a dedicated analytics team. In some ways, Sarkar tells me, he is more interested in their bi-weekly reports. The new analytics folks are tasked with analyzing the data that comes directly from the use of Vectorworks when users opt-in to sending that data back to the company. This tech was demonstrated during a Design Summit keynote a few years back.
“The team can see the usage data, the crash data, the mobile usage data,” says Sarkar. “I’m very interested in this data and ask them regularly about the new information. And while I am looking at that, I see new things and then ask the team new questions about, for example, what is this new feature that is appearing on the analytics side?”
next page: AI and Machine Learning
AI and Machine Learning
While both types of teams are relatively new to not just Vectorworks but also similar software companies, the analytics data that comes in from user behaviors is mainly of interest for improving user performance.
“One of the ways we can improve the user performance for Vectorworks is by guiding the user’s next action,” says Sarkar. This is an area where the company is employing machine learning frameworks and an area where they are exploring AI (artificial intelligence). The company dumps user log data into several machine learning frameworks that in the end are trying to figure out things like after a user draws a wall or a rectangle, what do they generally do next? “We are interested in predicting what their next step wants to be,” he adds. One of the things the company is already doing is making changes to the user interface (UI) based on this machine learning analysis.
We are also very interested in how we can connect TensorFlow’s deep learning frameworks to the Marionette scripting environment.
The company has already noted in earlier Design Summits their interest in using machine learning to improve graphics in their rendering engines. A focus of late has been using AI algorithms to analyze photos and generate a rendering style from that photo. Such a focus fits the company’s reputation as being one of the most graphics-forward facing CAD/BIM companies, a legacy item of note largely stemming from their deep Macintosh roots. (see image 02, pg 3)
A third way the company is working with AI and machine learning is in exploring the use of deep learning algorithms to generate optimal design layouts in buildings. “These are not just algorithms but genetic algorithms that are aimed at helping the user optimally configure particular types of layouts around various criteria,” says Dr. Sarkar. These investigations can be quite useful for particularly complex program types in architecture, like around hospital design where one is trying to optimize travel time from a nursing station to a patient care unit. “The algorithms are the same ones that are being used in genetic research,” he adds, “where they are trying to look at the particular configurations of genes in pursuit of a cure for a disease.” Such algorithms look at all possible patterns within genes to drive towards medical treatments and cures and the folks at Vectorworks are taking a similar approach but towards optimal layout configurations in buildings.
01 – Vectorworks’ Marionette visual scripting environment is being explored with TensorFlow Deep Learning algorithms to explore new ways of designing using optimization techniques used in genetics and medicine.
To pursue these activities, the company is also tapping the deep learning frameworks from Google called TensorFlow. “We are also very interested in how we can connect TensorFlow’s deep learning frameworks to the Marionette scripting environment,” adds Sarkar. “We want to see what opportunities are there.”
The company is especially interested to see if there is a way to send the feedback loop data from the machine learning algorithms back to the design-forward algorithms in Marionette. In other words, a virtuous feedback loop would begin with a “design goals”-based Marionette process—perhaps generating three-dimensional forms for architecture—that gets affected by TensorFlow deep learning frameworks all in the pursuit of creating optimal building design arrangements that meet complex form and program criteria.
next page: Augmented Reality and Computer Vision: Impacts on BIM
Augmented Reality and Computer Vision: Impacts on BIM
One of the fruits of the advanced research team’s efforts in the recent past has been Vectorworks’s AR (augmented reality) features. Dr. Sarkar feels the debate about VR versus AR isn’t particularly useful because both are important leading-edge technologies with much forward-looking potential.
02 – Vectorworks 2019 includes post-processing image manipulation for renders. But it intends to use AI and machine learning to advance unique render looks based on the analysis of photos and images.
“Immersive experiences will always be important,” says Sarkar, noting that VR isn’t something that goes away because of AR, “but AR adds a valuable way to convey ideas.” Also, Dr. Sarkar acknowledged that AR perhaps adds more value to the construction side of AEC than on the design side. “People will be using AR more for the construction and facilities maintenance part of a building’s lifecycle.”
With that suggestion, I asked Dr. Sarkar to weigh in on the sensor and 3D photography-based robots emerging on job sites, like Doxel’s technology. In particular, if drones and bots can effectively “see intelligently” and evaluate real-life buildings, what needs to happen to BIM models in order to advance those types of operations?
Our next logical step is to capture real-world data and convert it into models.
“The data that resides in today’s BIM models is kind of static,” Dr. Sarkar explains, “we will need to have some kind of dynamic data attached to building elements that you can query.” He noted that he is a big fan of RFID (radio-frequency identification) tagging and if this RFID data is attached to the BIM model then scanning devices can establish two-way communication with structure and BIM software. Indeed, such systems are similarly happening today, including the use of QR codes on building elements tied into CAFM (computer-aided facilities management) software.
Advertisement
The development of augmented reality (AR) systems is somewhat inseparable from computer vision advances and 3D camera technology. The same technology that reads your face for Apple’s FaceID system is fundamentally capable of reading a building under construction and understanding it in three-dimensions at accuracies up to 3 millimeters.
With advances in AR promising major leaps in user-experiences (UX) and how we interact with technology, Dr. Sarkar says Vectorworks is keeping a close eye on these developments, along with VR. “Our next logical step is to capture real-world data and convert it into models,” he says. In terms of the accuracy advancements made by both lasers and computer vision cameras and software, he says, “I think we are already there.”