Large Language Models (LLMs) are undeniably at the forefront of contemporary research, rapidly transitioning from theoretical studies into pragmatic applications. Also, their potential is vast in the health domain, and perhaps their most compelling proposition is in collaborative decision-making. They promise a new paradigm wherein physicians, patients, and artificial intelligence converge, offering an opportunity for assistive and informed decision-making.

However, intricate challenges come with the pace of AI-based applications, especially in healthcare. First and foremost, there's the matter of data. Given its magnitude and sensitivity, accessing and managing health domain-specific data is complex. Ensuring patient privacy and user trust while extracting actionable insights from this data is a tightrope walk. Coupled with this is the necessity for robust hardware and IT infrastructure resources, essential to support the computational demands of LLMs - or the development of methods to make the training and usage of LLMs more efficient.

The communication of results is another pivotal aspect. Explainable AI (XAI) has gained prominence, highlighting the importance of not just generating results but also elucidating them, i.e., answering the “Why?”-questions about how LLMs arrive at their output, in a manner comprehensible to specific and use-case-dependent (end-)users. For example, for physicians and patients to trust and act upon AI-generated advice, they need to understand its rationale.

Yet, the challenges don't end there. As we venture deeper into applying LLMs to the health domain, there's a conspicuous shortage of experts who blend AI acumen with a profound understanding of the healthcare or medical landscape. Furthermore, the worldwide deployment of LLMs trained in mainly a few languages raises questions about their ability to adapt linguistically, avoid or handle biases, and the importance of ensuring availability and inclusivity across diverse populations.

The legal domain, punctuated by frameworks like the upcoming EU AI act, further underscores the need for a vigilant approach, ensuring that the deployment of LLMs aligns with ethical benchmarks and regulatory requirements.

Given this background, our upcoming workshop has delineated clear research challenges:

• Examining the need and/or availability of LLMs specifically tailored for health-related and clinical use cases, defining requirements, and evaluating the performance of LLMs accordingly.
• Understanding how to train and fine-tune LLMs on clinical and medical tasks: What resources are needed? What is the role of synthetic data?
• Delving into the interactive dimension: How can physicians and patients most effectively communicate with and via LLMs? What interfaces (e.g., spoken language, written language, or multimodal approaches), and kinds of argumentation are optimal?
• A deep dive into XAI, aiming to make LLMs (more) transparent and understandable, and its transferability to the medical domain.

Furthermore, the workshop will explore specific use cases and applications, focusing on maintaining privacy and data anonymization, identifying best practices and potent applications, and exploring the ethical and legal boundaries within which LLMs must operate.

For AI-based applications, especially in health, there are challenges and issues:

• How to apply models for health problems:
  • Sensitivity and privacy (e.g., sending data out)
  • Hardware infrastructure (e.g., working onsite)
• Communication of results, XAI
• Transferability to languages and regions, bias, inclusivity
• Availability of sufficient data from the health domain

Aims of the workshop (research challenges):

• How to apply LLMs for cases using sensitive data?
• How to deal with limited domain coverage (e.g. under-resourced languages, medical domain, etc)?
• How to use synthetic data?
• How to make LLMs usable for end users?
  • How to communicate results (e.g. chat, speech interfaces)?
  • How to create trust and explain recommendations (XAI)?

Use cases and applications:

• Privacy and anonymization
• Best practices, tasks and applications
• Ethical requirements
• Legal restrictions and requirements (e.g. EU AI Act)