Societal XR

Measuring 2.00 x 1.40 meters, the Holo Matrix installed in the XR/Games Lab at Offenburg University of Applied Sciences is probably the largest holographic projector in the city of Offenburg and perhaps even in the entire Ortenau region. The Holo Matrix was configured and installed by the Ingelheim-based company Magic Holo using LED rotors called “holofans” developed by the British company HYPERVSN. The HYPERVSN Hologram Wall with 3 x 2 holofans generates a coherent, holographic 3D image that can display both pre-rendered videos and animations as well as live content via HDMI.

Given the growing popularity of holographic displays, their increasing use at trade fairs, for example, and the media coverage of holographic installations such as in “THE SPHERE” in Las Vegas, it can be assumed that holo projectors will become frequently used displays in public spaces over the next five to ten years. For this reason, the Holo Matrix at Offenburg University serves both to further research into displays that can be used according to the Societal XR paradigm and in human-computer interaction. Students at Offenburg’s Media Faculty can use it to learn how to work with this still young technology which is likely to become a natural part of their lives in the future.

The fact that a socially acceptable extended reality (XR) must not only represent virtual elements, but also remove real elements from their users’ perception, just as an ad blocker filters adverts, has been given far too little attention to date. Diminished Reality is a form of XR that is dedicated to precisely this goal: removing unwanted, superfluous or harmful elements from perception.

With DiminishAR, JangHyeon Lee and Lawrence H. Kim have developed an application for the HoloLens that makes distracting objects such as smartphones virtually ‘invisible’ and trialled this application in a study. I am pleased to report that they explicitly consider their techniques to be suitable for Spatial AR and Societal XR. In their analysis of potential applications they assume that their techniques could contribute to Societal XR, for example, by minimising distractions in communal spaces such as open offices, libraries, and classrooms.

On October 1st, just in time for the start of the winter semester, I had been able to put my new XR/Games Lab into operation. It is not yet fully equipped, but with 64 m², around two dozen XR HMD’s and games consoles, eleven workstations and a large collection of XR applications, VR simulations, 360-degree films, video games, analogue games and literature, it not only offers my students and research colleagues the opportunity to work, but also to research, explore, relax and unwind. Equipped with the latest technologies such as holofans and Apple Vision Pro, it also includes technologies related to holography and spatial computing, which should be essential for further research into Societal XR.

Launched in 2001, the “Mensch und Computer” (MuC) is the most important conference series on human-computer interaction in Europe. This year’s four-day conference at the Karlsruhe Institute of Technology (KIT) in Karlsruhe/Germany ended today.

My personal highlight was the Workshop on “Ubiquitous Mixed Reality”, which was held for the first time at MuC 2024 and fit perfectly to the Societal XR paradigm. As the first speaker of the workshop day, I was able to present my vision, including a potential application, put them up for discussion and meet colleagues from German and Austrian universities who are working in the same research field.

A student of mine, Astrid Pittschmann, has created some fantastic illustrations that beautifully bring to life all scenes from the Societal XR story in my vision paper. A huge thank you, dear Astrid! With these illustrations, I can now share the core idea and specific applications of my paradigm even more vividly in talks, lectures, and publications. You can check out Astrid’s amazing work on her ArtStation profile: astridpittschmann.artstation.com

After almost twelve years as a professor in Heidelberg, I have moved to Offenburg University, Germany, on September 1st, 2023. My new Bachelor of Science study program called “Virtual Worlds & Game Technologies” will start there next month, and I am preparing to open a dedicated XR & Game Lab already next year. With new colleagues, new sponsors, the new lab and new opportunities, I will continue to research Societal XR as well as XR and game development in general.

The sheer number of XR publications obscures how little focus is currently placed on social forms and applications of XR. Last year (2022), over 27,000 scientific articles were published on Augmented Reality (AR), and over 20,000 on Virtual Reality (VR). Mixed Reality (MR) was significantly weaker with just under 1,400 scientific publications.

However, if we search for publications on socially focused XR paradigms, there was a maximum of merely 130 on Social VR (in 2021) and a maximum of 36 publications (also in 2021) on publicly experienceable Spatial Augmented Reality (SAR). Other paradigms such as Social XR with just 3 publications each in 2021 and 2022 and Augmented Sociality have hardly been taken up. Others, such as Transmogrified Reality, have already fallen into oblivion.

It is obvious that social forms and applications of XR have been researched too little so far. The vision of the Societal XR is intended to help close this gap.

Since the publication of the vision paper, numerous cooperation talks have been held with companies, mostly from south-west Germany, about potential joint research and use case demonstrations. In addition, several bachelor theses on researching Societal XR have been started, the completion of which is expected in early autumn. Today, Societal XR was presented to a broader audience from industry and research for the first time at the 12th annual XR Symposium in Heidelberg, themed “XR Digitalization World”.