News » 'DESIGN INTENTS DISENTANGLEMENT' accepted to CAADRIA 2022
To appear at CAADRIA2022
MANUEL LADRON DE GUEVARA, ALEXANDER SCHNEIDMAN, DARAGH BYRNE, and RAMESH KRISHNAMURTI, “DESIGN INTENTS DISENTANGLEMENT: A Multimodal Approach for Grounding Design Attributes in Objects”
Language is ambiguous; many terms and expressions convey the same idea. This is especially true in design fields, where conceptual ideas are generally described by high-level, qualitative attributes, called design intents. Words such as “organic”, sequences like “this chair is a mixture between Japanese aesthetics and Scandinavian design” or more complex structures such as “we made the furniture layering materials like a bird weaving its nest” represent design intents. Furthermore, most design intents do not have unique visual representations, and are highly entangled within the design artifact, leading to complex relationships between language and images. This paper examines an alternative design scenario based on everyday natural language used by designers, where inputs such as a minimal and sleek looking chair are visually inferred by algorithms that have previously learned complex associations between designs and intents—vision and language, respectively. We propose a multimodal sequence-to-sequence model which takes in design images and their corresponding descriptions and outputs a probability distribution over regions of the images in which design attributes are grounded. Expectedly, our model can reason and ground objective descriptors such as black or curved. Surprisingly, our model can reason about and ground more complex subjective attributes such as rippled or free, suggesting potential regions where the design object might register such vague descriptions.