The technological development over the past decades and the wide-spreading of IoT, smart devices and sensors has enriched the number and variety of intelligent systems embedded in our environment. These systems have the capability to monitor our daily life and provide personalized and context-aware services. In this setting, designing the boundary between human and machine/intelligent environment through natural and seamless human-computer interaction is a hot research topic and within this context spoken multimodal dialogue systems become indispensable to offer the overwhelmed user the possibility to operate the plethora of intelligent devices, sensors and appliances in an efficient and natural manner. Therefore, there is a prominent interdisciplinary potential of both fields to generate sophisticated models that take into account the requirements for complex interactions in pervasive systems.

Interdisciplinary work coupling dialogue systems and pervasive computing technologies could have a significant impact in the adoption of ambience intelligence and ubiquitous computing solutions. On the one hand, dialogue systems can provide a more natural and human-like communication with pervasive systems, making them more transparent and easy to use and also more inclusive. However, although they have become the most featured and cutting-edge consumer interface, most consumer systems are still in an early stage because they predominantly use spoken dialogue only at a shallow level (e.g. for controlling home appliances, playing music, or searching the web). The integration of multimodal dialogue and sensing technologies developed in the pervasive computing area might allow to realize more complex context-aware interfaces and services.

On the other hand, pervasive applications have a huge potential to gain information about the users and their daily routines, providing very valuable information to generate rich user and multi-user models. Such models can be employed to infer relevant information about the users’ knowledge, emotions and intentions as well as the context in which the interaction takes place. The use of these types of models can be relevant for intelligent multimodal systems to react adequately and in a user-friendly manner. Providing spatially distributed information, pervasive computing technologies are realised either in a close contact with their users or smoothly integrated into the environment. Being a part of the smart environment, each smart device serves as a mini personal assistant and requires an ability to interact with the user so as to be able to be more adaptive, user-friendly and effective. This allows for user-adaptiveness and proactiveness in an ongoing interaction and makes the conversation more natural and human-like, e.g. using argumentative dialogue models which take multimodality and user preferences into account and thus, extend the system’s communicative repertoire to cater various new interface possibilities.

While both research fields have recently been of interest for the respective research groups, questions of integration between pervasive computing and spoken multimodal dialogue systems have not yet been sufficiently discussed within each community. In other words, despite the described potential, the interdisciplinarity of both fields have not yet been investigated in detail. An active cooperation between both communities may open the way for more powerful and user friendly pervasive computing systems. We aim to create the first and unique venue for discussion and collaboration between experts from these disciplines. The planned Shonan Meeting will help to explore possible challenges and jointly develop a research agenda for main directions. The meeting will establish a platform for an international collaboration for the next three to five years. As an outcome, we plan to edit a Springer book in the Shonan Meeting Series, in which the participant authors will reflect the main conclusions derived from the discussions.
Topics of this workshop include, but are not limited to:

Research challenges:

• Challenges of spoken and multimodal dialogue interaction for pervasive systems.
• Challenges for specific components of multimodal dialogue systems (natural language understanding, speech recognition, dialogue management, specific modalities…).
• Single and multiple user/speaker identification, modelling and tracking within smart environments.
• Context awareness for multimodal dialogue systems.
• Personalization and user-centered development of dialogue systems for pervasive systems.
• Evolution of the technology and ideas for future research and application scenarios

Development, testing and evaluation:

• Experimental design, user studies and evaluation of pervasive multimodal dialogue systems.
• Engineering approaches to dialogue interaction in pervasive systems: life-cycle, requirement elicitation, robustness to change, standardization and simulation.
• Development models, tools and strategies: middleware, tools, languages…
• Description, development and sharing of resources: corpora compilation, annotation tools and approaches, crowdsourcing approaches…
• Pervasive sensing devices and frameworks, the Internet of Things and its relation to multimodal dialogue systems.
• Coordination of conversational artifacts within smart environments (e.g. service discovery, coordination and argumentation…).

Use cases and industrial applications:

• Specific application domains for multimodal dialogue systems in pervasive computing: smart cities, public space (e.g. interactive digital signage, guidance systems), assistive environments (e.g. elderly care, hospitals)…
• Success stories, functional systems and industrial challenges.

Ethics and societal impact:

• Social responsibility.
• Data protection and privacy by design and default.
• Legal issues.
• Social design and development of conversational pervasive systems.