NO.161 Interactive Visualization for Interpretable Machine Learning ~ Beyond Visualization and Steering of the Parametric Space

Shonan Village Center

November 18 - 21, 2019 (Check-in: November 17, 2019 )


  • Alex Endert
    • Georgia Tech
  • Jean-Daniel Fekete
    • Inria
  • Bongshin Lee
    • Microsoft Research
  • Shixia Liu
    • Tsinghua University


Description of the Meeting

Machine learning techniques are used in most domains where sensors, data, and computing play an integral role. They help individuals (including ML researchers and practitioners, data scientists, and end users) describe, predict, automate, and control increasingly complex phenomena from a broad set of domains, from self-driving cars to stock market fast trading. However, a growing challenge that parallels the growth in adoption and complexity of these models is debugging, understanding, interpreting, and trusting them.

This meeting will bring together experts in the areas of machine learning, human computer interaction, visual analytics, databases, and cognitive science to discuss and explore this research challenge. We posit that the next generation of interpretable machine learning technologies will come from multi-disciplinary research at the intersection of these domains.

Topics of the Meeting

There are a number of societal and ethical concerns that come into play when using machine learning outcomes in the context of specific domains. For instance, how can we inspect machine learning models to ensure they are not exhibiting specific forms of bias in the parameterization and structure of the models? Similarly, how can people ensure that aspects of privacy and other societal concerns are taking into account when making decisions that involve machine learning models. International regulations require guarantees and accountability when using systems based on machine learning [GDPR Accountability].

Interactive visualization can help people debug, understand, and interpret machine learning models and their outcomes. For example, the outputs can be visualized to help contextualize them in the domain and the data. The model structures and topology can be shown to help data scientists gain an understanding of how the model mechanics combine to produce the output. Finally, interaction in these systems can help people adjust parameters or other characteristics of the models when they observe something that is incorrect.

Existing techniques have focused largely on the visualization of the raw model parametric space. While these may be interpretable and helpful for data scientists and for particular models, we will focus on exploring methods that show alternate parametric spaces derived from the raw model but easier to interpret and control. This seminar will explore visual representations that help people make sense of these alternate parametric spaces of models. For example, SVMs are often illustrated using a scatterplot that shows the support vector that divides the space of points. Can we identify similar insightful representations for other models?

This seminar will also discuss new diagnostic methods for people to understand how trained ML models will react to different data cases. Existing ML techniques are typically not optimized towards interpretability, but for benchmarking tasks and gathering standard performance metrics (i.e. precision & recall). Still, some methods for, e.g., extracting decision trees from a trained neural network, provide a way to peek inside the black box and understand locally the behavior of the neural network [DOI:10.1145/775047.775113].

Aims and Objectives

We will bring together experts from the related research communities including machine learning, human computer interaction, visual analytics, databases, and cognitive science to:

  • Develop novel theories and frameworks that help outline the research avenues for cross-disciplinary research to help people understand and interpret ML models and their outcomes.
  • Identify research gaps between machine learning and visual analytics, and enumerate the primary opportunities and challenges for researchers and practitioners.
  • Discuss novel visualization and interaction techniques that make ML models easier to interpret and control.

We will invite three talks to explain what interpretable ML means in three different regions (China, Europe, and US). This will help seminar participants be on the same page before we start our discussion. We will first identify a set of important topics to help us achieve main goals and form breakout groups to cover them. We then plan to have a series of breakout group sessions, having an in-depth discussion on these core topics. A daily schedule is appended below (pp. 9~10).

Related Previous Workshops and Seminars

We acknowledge that a few workshops and seminars have addressed the related topics:

  • Shonan Seminar No. 120: Visual Analytics: Towards Effective Human-Machine Intelligence.
    Even though this seminar mentions algorithm transparency as one of the main topics, its main focus was how to effectively integrate human knowledge and expertise into powerful computational algorithms.
  • Shonan Seminar No. 064: Big Data Visual Analytics
    The main focus of this seminar was on “scalable” visual analytics solutions not on interpretability of the solutions.
  • Shonan Seminar No. 057
    Towards Explanation Production Combining Natural Language Processing and Logical Reasoning. This seminar focuses on natural language processing instead of machine learning and visual analytics.
  • Dagstuhl Seminar: Bridging Information Visualization with Machine Learning
    This seminar is the closest to ours. But, it dates back to 2015, and we want to specifically focus on transparency and interpretability.
  • ICML 2017 Workshop on Visualization for Deep Learning & NIPS 2017 Workshop on Interpreting, Explaining and Visualizing Deep Learning - Now what?
    These two workshops make some initial efforts towards exploring the role of interactive visualization in understanding the inner workings of deep learning models. However, they mainly focus on presenting the existing efforts. Most of accepted papers are from the learning field, which leverage feature maps or saliency maps, instead of interactive visualization, to illustrate the role of neurons in the training or prediction process.
  • NIPS 2017 Symposium on Interpretable Machine Learning: This symposium aims at producing more explainable models, while maintaining high learning performance. All the research work presented in this symposium are from the learning field.
  • 2018 Workshop on Human Interpretability in Machine Learning (WHI): This workshop explores how to make ML more interpretable, using a wide variety of technologies. Only a few papers touched upon visualization.

Our workshop will focus more specifically on how to use visualization as a means to make machine learning more interpretable while facilitating and leveraging cross-disciplinary research across several communities including machine learning, human computer interaction, visual analytics, databases, and cognitive science.

Expected Outcomes

We will write a report summarizing the meeting outcomes including the main concepts and a research agenda. We plan to publish this report in top venues in machine learning and visualization such as ACM Transactions on Interactive Intelligent Systems (TiiS). We will also consider a special issue on interactive visualization for interpretable machine learning.