Over the years, several mixed and extended reality devices have been developed with different interfaces. In this post, I'd like to explore the design variations and composition of the different Designs for XR in more detail. The respective designers will be linked in the references section.
I have divided this post into 2 sections, the "how to get started" and the device specific section (Magic Leap, Meta and Microsoft). It is very helpful to build up some basic knowledge that also provides the analytical tools to read the GUI implementations hereafter.
Integrating the Real World with Passthrough
Integrating virtual content into the real world. That's exactly what the concept of mixed reality is all about. In fact it has been a major goal to do that in Virtual Reality as well but there wasn't a real visual representation of the real world yet. So this is the big moment for Augmented Reality with extended sceenspace of a Virtual Reality headset it actually really gets interesting now.
New Possibilities for Spatial Awareness
One of the key benefits of XR devices is their ability to blend the physical and digital world, offering a level of spatial awareness that is not possible with traditional screens. Passthrough technology allows users to see the real world through their device, making it easier to interact with the environment and even other people. This can be quite awesome to realize interactive experiences that integrate those possibilities in the gaming area.
XR Apps for Productivity
In addition to enhancing the gaming experience, XR devices can also be used for productivity. With the integration of passthrough, users can work in mixed reality, accessing digital information and tools in real-time without having to switch between screens. This can be especially useful for tasks that require hands-free interaction, such as drawing, design, or navigation.
Integration of XR Devices as a Monitor Replacement
XR devices can also serve as a replacement for traditional monitors, offering a more immersive and interactive experience. With the ability to display digital information in the physical world, XR devices can help users stay focused and increase their productivity.
Exploring UX of working in Mixed Reality
By integrating the real world with passthrough and XR devices, users have the ability to work in mixed reality, where they can interact with digital information in a way that feels natural and intuitive. This opens up new possibilities for how we interact with technology and how we can use it to improve our lives.
To get a better idea of how this might work I'd recommend to you to connect your bluetooth keyboard to your headset (works great with Meta devices) switch to desk mode and start your device internal internet browser.
How to get started and what about rules?
While interacting, it has proven to adopt positions that are as comfortable as possible, however, depending on the medium of interaction, a different position may be more comfortable.
Understand the canvas and respect the depth
The canvas in XR is defined on the one hand by human perception, i.e. the zoning principle, and on the other hand by the physical space and the objects within it that constrain the space. The idea of content areas is based on human ergonomics and display characteristics of where something should be placed.
There are areas of comfortable and maximum rotation. These result in useful areas to place your content for non-rotating experiences.
A Content Area - The Stage. This is the area where your head can rotate comfortably and where things still provide stereoscopic depth perception. Closer objects have a more distinct sense of "3D optics".
Peripheral Area - The area that is visible at maximum head rotation. The surrounding area is still seen more regularly, but no long-term content should be placed in this area.
Curiosity Zone - At this point, the user literally turns their shoulders and tries with some effort to see what is behind them. Probably to satisfy his curiosity about his surroundings.
No-No Zone - As things approach the face, the user squints and the eyes become strained. Nothing should be displayed in this area around the head.
The radius of 0.5 m is rather large according to experience, so it can be interpreted as a kind of fall-off zone without a fixed line. Skybox zone - after 20 m, the two screens show essentially the same image pixel for pixel and there is no depth perception. To reduce rendering costs, any objects beyond that should be flattened and put into a spherical texture that surrounds the user - the skybox.
Armed with this basic XR-UX/UI knowledge, the placement of buttons, sliders and interfaces should become a conscious and well-planned design decision.
Magic Leap - Lumin OS
Lumin OS 0.98.33 the spatial computing operating system from Magic Leap enables advanced 3D visualization workflows, enterprise device management, multi-user functionality, voice and gesture control. In the post Extended Reality insights & links I've listed links to some of the tools needed to do this, which have been specifically developed by the company.
A custom font, tailored specifically to the device, and used consistently in the GUI
To what extent should you go with the UI implementation and do you really need your own typeface? I think it's all a question of purpose, it is incredible to see, how well it was implemented until the last icon for the home button on the controller - truly impressive!
As a UI-Designer this is your basement. So if your content is supposed to stick out and provide a different layer my recommendation would be to go edgy but organic at the same time.
The pointer interaction beam can be used to navigate through the GUI using the controller orientation. So you have a spatial cursor that shoots a ray into the 3D scene. This interaction helper can be found in almost every XR-UI and is also a fixed part of the hand and gesture based interaction, for sure you can call it a standard XR-UX feature.
As shown in the example above sliders and simple but effective UX is necessary to navigate through the applications content. I'd recommend to stick to bright colors and heavy contrast since the UI provides better readability this way. In terms of depth I'd recommend not to go to far on the Z-Axis in direction to the users. It is tempting to play with the depth of the layers but you should stick to a certain realistic distance. 2D content placed in 3D space tends to look ripped off very easy. Especially if it is not facing towards the user.
Spacial UX/UI by Magic Leap
Since Lumin OS and the UI of the Magic Leap are designed to function as an Augmented / Mixed Reality interface, this implementation is an excellent Mixed Reality experience to date.
The spacial 2D / 3D interface animations are particularly unique. It's exciting how the circular shape was cohesively executed. The takeaway here is to think in motion. If the UI responds to the hover to the cursor it automatically feels responsive and triggers the user to interact (see get started section)
https://www.youtube.com/watch?v=GmdXJy_IdNwWake up in XR, for 2016 already a very impressive GUI
The arrangement, and the adaptation towards the controllers is very effective and makes a super balanced and well-placed impression.
Meta
Since Mixed Reality and the Meta Quest Pro’s high-res, full-color passthrough a new are for spacial UI has started @Meta studios. The ability to identify room surfaces, and anchor virtual objects to the physical world is awesome.
XR productivity apps
The Quest Pro enables easy and comprehensive access to PDFs in VR or mixed reality through Adobe Acrobat Pro as a productivity application. In the promotion rendering, you see an application of the software "standing", which looks good, but is not necessarily the standard user position. In fact, PDFs, which are mainly received from e-mails or downloaded from the browser, are usually placed in the background, i.e. on the 2nd monitor or a tab. Therefore, the 2D overlay feature shown below is great in combination with this application. However, it is questionable why one does not simply use the browser to display PDF files. Nevertheless this app has a lot of different UX / UI features that work amazingly well and covers a great source of inspiration.
2D Overlay Design for XR
Greg Madison's post shows the 2D overlay feature, which can be very handy, especially if you want to pin an extra browser window while in the application, or for multitasking.
I think just such simple features, which are "invisible" to the user, but yet naturally used by everyone, will now appear more often. Shortcuts and the integration of applications for the use of everyday documents such as PDF office applications, emails, etc., which are an essential part of our daily work, will follow.
9/ 2D App Overlay:
Like in the physical world with our phones, you want easy access to the web wherever you are in the metaverse - not just in VR Home.
New for Meta Quest Pro, users will be able to open up the browser or any other 2D app without leaving the VR app they’re in. pic.twitter.com/PXAEJvnmZu— Jacob Rossi (@jacobrossi) October 11, 2022
Embedded JavaScript
Metas all new XR - UI experience
From a contemporary user perspective the GUI Design for XR devices is a new world. The features designed by Meta for VR / XR devices cover some exciting possibilities, but they are not really used much yet, so rather created for the possible future.
In the implementation, one finds a mixture of different, already successfully implemented principles from Microsoft and many others. What has been implemented very well and coherently is the "dock", or rather the "taskbar". To function as the user expects, however, some features are missing. The pinning of apps and other customization features.
However, what has been implemented excellently and is definitely a very strong concept is the responsive scaling of the windows. This also solves the size issue directly and helps the user to optimize his interface as desired.
Meta Quest Pro review - For Creatives, UX/UI and Spatial Designers
What is shown in the review video here is rather less impressive, the recording quality is not so special, but you can also see some weak points that the device has. Especially in terms of resolution, you have to sacrifice quite a bit. It becomes clear that you are wearing a closed headset and looking through the cameras mounted on it, which is not exactly a positive effect compared to the "real" AR glasses. It makes sense to use human vision and not to imitate it.
Sources and references
- Sean Couture Lead / Senior 3D Artist
- Greg Madison Experience Designer @Meta
- Mike Alger @Google
- Alex Chu @Samsung Conference
- StartUX Youtube Channel by Avi Barel
Meta
Magic Leap
Nreal
The contents shown here are subject to the legal restrictions of the respective publishers, this contribution follows the entertaining, informative purpose.
