Designing Ecosystemic User Experiences 

In recent decades, design has faced profound challenges and transformations. The traditional approach to crafting and shaping the tangible world has been challenged by the world’s infusion with digital technologies, which have made it smarter, more interactive, and more connected. 

DeSForM, the conference on the Design and Semantics of Form and Movement, was undertaken in 2005 as an attempt to foster discussion in the design community around how to design the meaning, aesthetics, and experience of responsive and dynamic artifacts. DeSForM’s intent was to 

“present current research into the nature, character and behaviour of emerging new typologies of co-designed, content rich, connected and intelligent objects within adaptive systems.” 1

Those ‘emerging new typologies’ of ‘intelligent objects’ have developed and spread over the following years, bringing the rise and formalization of new areas of design research, such as interaction design, user experience, and the aesthetics of interaction. These domains have been widely investigated in the works presented and debated in the past editions of DeSForM.2

However, recent technological developments are causing even more rapid and extreme changes than the ones witnessed at the beginning of this century. The emergence of artificial intelligence and machine learning, flexible electronics, virtual and augmented reality, miniaturized and implantable sensors, and hybrid synthetic-biological materials have not only provided designers with new design ingredients, but also generated new cultural and social landscapes in which they must operate. 

In this context, designers are called to design not intelligent products within adaptive systems, but rather those adaptive systems as a whole. Objects can no longer be interpreted and designed as independent elements, detached from the other components of the complex digital-physical ecosystems they belong to. In such hybrid ecosystems, new distributed intelligences, advanced materials and interfaces, sensing technologies, data, and humans are deeply interconnected and mutually shaped. Their understanding, design, and evaluation demand approaches and tools able to tackle this complexity. Despite this, as these systems become increasingly intelligent, their meanings, aesthetics, and ethics still seem to be overlooked. 

Designing beyond intelligence means that the design of such complex and smart ecosystems should consider issues beyond mere algorithmic thinking and functionality. Scholars and practitioners in the design field are encouraged to reflect on the connections and mutual relations between the performance of these intelligent ecosystems and their physical appearances, meanings, personalities, and interaction modalities. In doing so, they will be able to address the design of ecosystemic user experiences. 

Going ‘Beyond Intelligence’: New Challenges for Design

In the XI edition of DeSForM, hosted by the Massachusetts Institute of Technology, we explore the implications of recent and emerging technological transformations in the practice of design, with a particular focus on the human experience of these complex systems. 

We invited designers, artists, researchers, and industry practitioners to address the need to design for distributed, hyperconnected, and learning intelligent ecosystems, and to investigate how their meanings, experience, and ethics can be approached. 

In doing so, we identified a number of possible challenges that we believe are worth exploring in the upcoming years. They refer to i) the growing complexity of the concept of user experience; ii) emerging forms of interaction with human-like intelligences; iii) the ethical implications of digital-physical systems; and iv) the new roles designers should assume in this context. 

Experiencing Complexity 

As ecosystems of digital-physical solutions become more layered, distributed, and connected, the user experience also grows in complexity. New elements need to be considered, including the meanings of these systems, the multisensory and multimodal interactions they necessitate, and the emotions that such interactions generate. 

The tangible manifestations of the systems users interact with are just a tiny part of a huge underlying infrastructure of data, algorithms, platforms, and digital contents. As functions overlap in the same product and digital contents constantly change, physical objects become just the medium for a plethora of meanings derived from multiple connected platforms. What is the role of aesthetics in such dynamic, digital-physical ecosystems? What meanings can tangible forms convey? What new tools and frameworks are needed to design and evaluate the growing complexity of user experience?

Interacting with New Intelligences 

Artificial intelligence opens up new frontiers for design, where emerging forms of distributed intelligence become design material. Technological advancements in this field now provide the user with the possibility to have increasingly human interactions with non-human artifacts. Users’ interactions with specialized intelligences have progressively taken on the appearance of companionship and assistantship, as seen in the rise of chatbots and social robots. How does artificial intelligence transform artifacts (objects, spaces) and their interaction modalities? How can design give meaning and form to artificial intelligence, when embedded into products? How do artifacts’ aesthetics and experience change through AI at home, at work, or in public spaces? This material should be fully investigated in terms of tangible manifestations, social implications, and impact on the design process and the user experience. 

Societal Impacts 

The above-mentioned transformations surely pose new challenges to design, not only in shaping the tangible forms of these systems, their meanings, and aesthetics, but also in anticipating the consequences they might have on humans at the individual and societal levels. The emergence of AI, robotic solutions, and big data connected with the spaces, objects, and people we interact with everyday will create new landscapes for future generations of designers. This will require designers to adopt new lenses in the design and evaluation of emerging technology, and it will necessitate that designers equip themselves with new ethical paradigms. 

How will algorithmic decision making and autonomous systems impact user experience and behavior? How can we design for transparency and reliability? What are the long-term effects of new digital technologies on society? 

As systems become smarter, more self-governed, and increasingly embedded into our reality, designers should develop new approaches and methods to consider ethical issues in their practice. 

Future Roles of Design 

This evolving context calls for new design skills and ways of thinking that go beyond the traditional field of design. How will this domain change, in order to interface with new fields of knowledge such as biotechnology, computer science, AI, and ethics? What are the future roles of design in shaping the growing complexity of the artificial world, where the boundaries between artificial, human, and natural fade? What role can designers play in the multi-disciplinary teams that will envision future systems which are more and more interactive, interconnected, and even unpredictable? 

While some of these challenges are just emerging, other issues seem to be already compelling, or will likely be in a short time. This conference invites the design community to reflect deeply on the current and future transformations enabled by technology, as well as their effects on design itself, and on society as a whole. It will take time to fully understand this new landscape and its effects on humans, and to approach it with a critical eye. As some of the works included in these proceedings point out, design is just now starting to react to this transitional moment and to equip itself with new sets of concepts, approaches, and methods to face this changing reality. 

DeSForM19 Proceedings

The proceedings from the DeSForM | Beyond Intelligence conference are structured to address some of the emerging questions raised above. 

Design Manifestos. The first section of the proceedings is dedicated to sharing thoughts on the roles of designers and the meanings of design practice through a series of design manifestos. What are new tools, methods, and frameworks that allow designers to forecast and solve the wicked problems of the future? Observing the current landscape of complex systems and varied forms of intelligences - from artificial machines to synthetic biology - some ‘turns’ in design practice are identified, which led us to the current state. These manifestos question the role of the designer and the meaning of agency as design practice becomes collaborative at all stages, especially through the use of algorithmically enhanced design tools and artificial intelligence. These questions are expected to repeatedly rise to the forefront throughout the conference. 

Interacting with Domestic Intelligences. Perhaps we are more accustomed to interacting with artificial intelligence than we think. Virtual assistants and conversational agents are slowly becoming the norm in domestic settings through the use of smart connected products and social robots. The works presented in this section analyze the current product landscape of domestic intelligences and provide an initial understanding of relationships between form, function, and meaning. As we move into the future of embedded intelligences in our everyday environments, how should we design the shape and interaction modalities of artificial intelligence to effectively translate its function and meaning in an intuitive way? 

Interacting with Urban Intelligences. From autonomous vehicles to delivery robots, we will soon, if we are not already, be sharing our urban environment with other intelligent entities. We can no longer opt out from this smart environment experience, and we have no other option but to interact with such systems and provide resources (data) back into their digital networks. Papers collected in this section explore the opportunities and affordances that become available in the design of such environments, as we learn to coexist with various forms of artificial intelligence. From the concept of 'robot citizenship' to that of 'coerced' users, authors suggest new approaches to bring the perspective of urban robots, citizens, and other autonomous systems into the design process. 

Shifting our view to the building scale, various forms of intelligence will also become embedded in our architecture. The rise of synthetic biology and the use of engineered microbes as building blocks in urban architecture is opening up an era of hybrid buildings, which function as metabolic systems. Can we simulate nature and create environmentally performative, intelligent, and living buildings? 

New Interfaces for Complex Ecosystems. In this plethora of complex ecosystemic experiences, what are the new interfaces for control and interaction? Expanding from voice, text, and gesture-based modalities, what novel interactions can we design for? As we start to build a dictionary of universal interactions with smart products, how shall we explore the semantics of interaction language? Here we explore research in the design of interaction metaphors to represent conceptual understanding of situations (and translation of our language) to communicate intuitively with smart devices. But perhaps before we attempt to design new languages, we should think about methods to reveal and recognize the affordances of technologies in relation to our body, and the aesthetics of interactions it brings.

Smart and Multi-Sensory Systems for Behavioral Change. Although the experiencing human body is the constant in this ever-changing environment of complex ecosystems of intelligences, the designed experience of these systems induces behavioral changes in its users. The papers in this section investigate the effects of the merger of digital experiences and physical environment on human behavior, from creating ‘phygital’ activities that affect the cognitive learning abilities of children, to mitigating procrastination through designed interventions in built environment and interactive artifacts. Moreover, attention is paid to the experience of caring in elderly living environments through the use of connected technologies. What experiences can we augment with technologies and what should remain as human-driven? 

Design and Semantics for Health and Inclusion. Focusing on the health industry, we look further into the semantics and aesthetics of interaction in medical and assistive devices. This topic explores semantic strategies and design criteria to overcome social stigmas in the use of assistive devices, and to improve rehabilitation processes as well as overall user experience. 

Designing with Humans, Machine Intelligence, and Data. This final topic brings us back to the discussion on the role of design and the various societal issues designers should consider in their research and practice. What tools and methods can help us to navigate the complexity of data privacy issues in the co-design process? Moreover, as our design software tools become more intelligent and generative, should we rethink the notion of design agency, and invite our software tools to become our creative partners? 

Interactive Demos 

Short papers and their related interactive demos explore five thematic areas: designing immersive experiences, AI and human collaboration, AI curated experiences, sensory augmentation and communication, and processes and tools for design and awareness. 

Immersive Experience. Virtual reality is often used in safety training for hazardous situations or difficult to access environments. What if mixed-reality experiences are used to learn swimming? Can simulated experiences help overcome the fear of water, and bring the experience of swimming to those with limited access to aquatic environments? Along the line of making experiences real, another project explores the ‘aliveness’ of public art installations, to bring continuous life through embedding real-time responsiveness and audience participation in art experience. 

AI and Human Collaboration. Here we explore the design of transparent collaboration between humans and machine intelligences. From graphic design to web contents, and also digital publications, how do we define design agency when algorithms and machine intelligence become active creators of experiences? 

AI Curated Experiences. From music to movie platforms, we are accustomed to algorithmically-curated contents based on individual preferences. How about algorithmically-curated clothing suggestions? Can machine intelligence evaluate aesthetics, cultural nuances, individual preferences, and other design elements? Thinking further about human experiences, can AI understand and foresee users' desired engagement level, and curate a holistic reading experience by, for example, selecting appropriate content and creating an optimal ambient sensory environment? 

Sensory Augmentation and Communication. Human experience is deeply affected by sensory experiences and social interactions. Can an augmented olfaction device strengthen the link between olfaction, vision, and memory? Or better yet, can we translate our emotions and visually communicate these through an interactive wearable device for the face? Can digital technology and algorithms enhance human communication and social interaction, or is this a false hope? 

Process and Tools for Design and Awareness. We close the Interactive Demo section with new toolkits and processes for design and awareness. What are the ways to guide a value-driven design process and the creation of meaningful products? What methods can help unpack complex systemic challenges like climate change, to stimulate discussion and ideate potential interventions? 

Future Perspectives

This edition of DeSForM covers a wide range of topics related to designing with new forms of intelligence in complex human-artificial and digital-physical ecosystems. If the trends we are debating at this venue continue to develop, distributed intelligences could potentially affect any designed reality, as well as the experiences that result from users' interactions with these realities. 

While design as a discipline is required to develop specific frameworks and tools to tackle the growing complexity of our world, such a diversity of application fields also calls for a collaborative approach with other areas of knowledge. Designers will need to operate more and more in concert with technologists, computer scientists, architects, social scientists, psychologists, and ethicists, as well as policy makers and industry players, in a joint effort to reduce the risks and amplify the positive potential of these transformations. This will ultimately support the type of technological development that is truly centered on and beneficial to humans, both at the individual and societal levels.