As we advance into the age of Artificial Intelligence (AI) technologies such as LLM (Large Language Models), LMM (Large Multimodal Models), and VLM (Vision Language Models), we are moving closer to a world of eXtended Reality (XR), which includes both Virtual Reality (VR) and Augmented Reality (AR). Examples include technologies like OpenAI’s ChatGPT and growing affordability of Meta Quest VR headset signal the early 2 emergence of XR for developers. AI is already influencing typical software development tasks [1] , and we expect this trend to become commonplace soon.

We propose “eXtended Reality Software Engineering (XRSE)” as the field to transcend conventional Software Visualization [2, 3, 4, 5], extending into all areas of software engineering in the future: from program comprehension, software maintenance, software analytics, and software evolution to mining software repositories (MSR), and ultimately, the field of data science and analytics. Most topics are self-describing, so we focus on the specific topic of program comprehension and mining software repositories, which we deem very relevant: Program comprehension deals with techniques and methods (often supported by appropriate, custom-made tooling) to understand very large and evolving source code bases. Those code bases are nowadays often available and accessible as repositories hosted on various platforms, with GitHub having become a sort of de facto standard. The field of mining software repositories deals with processing and analyzing those repositories, which comes with numerous challenges, mostly due to the unstructured or semi-structured nature of the raw information.

Our idea is that all the fields mentioned above will potentially see a major upheaval in the context of XR (due also to consumer hardware reaching affordable prices), with two key underlying questions being asked:
1. What are the emerging and unforeseen ways that developers will interact with the data produced while systems are being evolved and maintained.
2. What are the different considerations when it comes to the potential role of gamification in XR?, where game concepts are applied to non-game contexts, such as software engineering.

All this motivates the group of invitees stemming from diverse backgrounds and contexts within the wider software engineering research community. Key outcomes that we expect are structured along the lines of “inform, discuss, enable”. We envision two goals:

Understanding the State-Of-The-Art: The first goal is to reach a shared understanding of the emerging field, by informing the people (many of whom have not yet dealt with XR research-wise) about the state-of-the-art from a technical and conceptual point of view. This would be made possible on the one hand through a small number of presentations by selected attendees, and on the other hand through a number of hands-on “labs” where we would let the attendees play with the currently available gadgets.

A Vision of XRSE: This would lead to the second goal, to discuss the possibilities that are opened up with the current (and future) technology. This would take up most of the seminar time, probably organized in the form of break-out groups that would have focused discussions, reporting back to the plenary group. We also envision that this part could then be formalized as an eXtended Reality Software Engineering Manifesto or white paper, which would be a tangible outcome. The seminar would be structured around Breakout Sessions and Demo sessions, preceded by Vision Keynotes.

A side-result would be to enable collaborations between researchers that would otherwise not have the chance to meet and discuss. Having organized several other workshops, we know that this part is probably the most beneficial to the attendees.

References
[1] Xiao, T., Treude, C., Hata, H., & Matsumoto, K. (2023). DevGPT: Studying Developer-ChatGPT Conversations [Data set].

Zenodo. https://doi.org/10.5281/zenodo.8304091

[1] Hoff, Adrian, et al. "Preparing Software Re-Engineering via Freehand Sketches in Virtual Reality." 39th IEEE International Conference on Software Maintenance and Evolution: ICSME 2023. IEEE, 2023.

[2] Hoff, Adrian, Christoph Seidl, and Michele Lanza. "Uniquifying Architecture Visualization through Variable 3D Model Generation." Proceedings of the 17th International Working Conference on Variability Modelling of Software-Intensive Systems. 2023.

[3] Romano, Simone, et al. "On the use of virtual reality in software visualization: The case of the city metaphor." Information and Software Technology 114 (2019): 92-106.

[4] Moreno-Lumbreras, David, et al. "CodeCity: A comparison of on-screen and virtual reality." Information and Software Technology 153 (2023): 107064.