Humans have a natural desire to store and preserve memories. This drive leads to an even greater need to store data in today's digital age. The rise of technology has made it possible to capture, store and access vast amounts of information, including photos, videos and other multimedia. Today, with the advancement of technology, we can not only store these memories, but also access and relive them with an unprecedented level of detail and immersion.
Relive XR is a simple XR app that is designed to help users relive precious memories in an immersive and interactive way. The concept of Relive XR is based on people's primal desire to preserve their most precious moments and experiences.
This project page is divided into different levels, which originate here on this page.
For the web experience, you can scroll through the onepager on Behance.
Meet Relive XR
Imagine Your Memories with Relive XR
Relive is a fictitious company that operates a platform of the same name for storing and enhancing user data. The software processes and categorizes all available data, completely automatically, using AI algorithms to reassemble the fragments and create immersive and interactive experiences from them. With Relive XR, you can experience your memories as if they were happening all over again, in vivid detail and with all the emotions of the original moment. Whether you're looking to relive a special moment from your childhood, or capture and share a recent event, Relive XR is the ultimate tool for preserving and reliving your memories.
Project Info
Relive is a non-commercial project that I, as part of my Diploma at Design Factory International, have designed to explore the potential of XR in preserving memories, enhancing experiences, and storytelling. The project has been made possible by the high-resolution resources provided, such as the font, 360° moving image content, and the video player. However, I had to do some heavy post-processing and technical conversions to make it all work. In my opinion, this project showcases the potential of XR in creating engaging and immersive content for a wide range of applications, such as games, advertising, and trade fair presentations. By utilizing a hybrid approach that combines motion, video, and virtual reality interactions, the possibilities are endless.
Research and Development
The RnD part was huge and extended far beyond the time frame of the development of this experience. It goes without saying that many different factors are important. I wanted to give a lot of importance to the interface design. But there were also some challenges in converting 360° photos into 3D worlds. Since the project was organized and implemented by myself, I started mapping the process into a tree diagram quite early.
On the 3D side, I worked mostly with Blender and Cinema 4D to create the immersive experiences. To convert and map 360° content into 3D geometry, I used an experimental tool called "Walkabout Worlds".
Today, companies like Matterport have similar implementations on their platform, their AI-enhanced projection and photogrammetry software has greatly improved and produces better results, especially when combined with infrared tracking data. However, my focus was on the spatial effect I was experiencing, and the tools I found at the time gave me ideal results.
Product development process
In the relive XR-Showcase it is possible to relive a random selection of the most exciting moments, this shows the current state of possibilities 2017. The contents are 360° videos drawn on geometry, thus it is possible to easily save periods of time in a memory place. It is possible to move over a certain radius in this world and relive the spatial mood.
How to convert Videos into 3D spaces?
First and best answer would be of course, you can do that with photogrammetry! Or maybe even better - Gaussian Splatting.
However, in 2017, smartphone apps and AI applications really weren't as advanced as they are today. Also, the challenge for me was that I was blessed with a blatant amount of data, not too much computing power, and a maximum implementation time of 2-3 months. So this possibility was unfortunately not up for debate. But there were 360° pictures and videos en masse and partly in outstanding 4k quality. After I had looked at some of them and was unfortunately not impressed. As soon as you try to walk in a 360° view, the experience is disrupted.
As soon as you try to walk in a 360° perspective, the experience is destroyed because the image is not spatial and sticks to the position of the head. The spatial effect is missing which means the viewer does not become part of the world. Thanks to my media design education and my time in the postproduction company INFECTED (yes, that's their real name), i knew my way around codecs to a certain extent and was well aware that videos are actually individual images. So all I had to do was find a frame in the video where all the content was clearly visible and then transfer it from 2D to Walkabout Worlds.
To convert and map 360° content on 3D geometry I worked with the experimental tool "Walkabout Worlds". This approach was relatively new and experimental at that time and you have to do a lot manually. Today, the company Matterport has a similar implementation on their platform and produces better and better results automatically. Especially in combination with infrared tracking data and AI this can be precise and fast.
What I think was a more special use and a really ingenious implementation for the experience design, and ultimately worked great, was playing videos on geometry. To create the real feeling of being in the scene and to increase the immersion, I added different particle effects. In the desert, sand particles. In the waterfall scene water vapor and in the snow scene in alaska lightly trickling snow.
Experience Design Realization
I used Unity 3D for my XR implementation. I've found that there are very few useful wireframes that can be used for this, but certainly limitations on what can be placed where and how. During the implementation process, I often used multiple PNG layers in the functioning UI to test them directly. The GUI was created using Affinity Designer and Affinity Photo in 2D and then assembled in Unity in 3D space.
Virtual Reality UX & UI in relive
A crucial learning for me was the time of conceptual interface development. It makes no sense to overload virtual reality with cyber interfaces just because it is possible. In Golden Krishna's book "No interface is the best interface", as the title suggests, the need for simple, functional interactions in real life is described and investigated. And then you keep running into smart XR people who remember the title of the book. The problem with these bold statements is that, just like all other wisdom from "experienced" designers,
it completely omits context. Inevitable context is: You have to try it out for yourself. So i did! - Back and forth with a lot of way to sophisticated design decision, thanks to the great principles by Mike Alger, Golden Krishna some more scientific reports like Alex Chu but also my own "best practice" principles. With the result:
Especially the touch detection on the buttons and the thumbstick can be used to make UI "responsive". The other buttons on the controllers are hard to find.
User Information
The interaction concept is based on a device-typical controller interaction that claims to be native and responsive. Memory worlds are stored in spheres, which can be detached from the interface and placed on the head.
relive UX/UI Design
The aim was to harmonise the concept with the real environment and create a hand menu that is omnipresent in the Relive XR experience, allowing you to jump from one memory world to another. In fact, I was able to achieve a particularly solid result by combining objects and 2D elements that, when layered in depth, create a 3D effect.
The idea of placing scenes on your head in order to dive into them has already been implemented by Valve in "The Lab", where the spheres were placed on sockets. In my application, I wanted the memory worlds to be available in the diegetic interface. I wanted the instanced placement of the 2D elements to feel more tactile. As with most things in UX design, it was a case of taking proven interaction principles and adapting them to fit the demands of the task at hand.
As an extension of the memory worlds, I considered inserting a depth timeline. Seeing how well the depth elements and sliding interactions worked with the hand, this would have been the next step in the case of many more tens of thousands of memories.
Gameplay & Reviews
I had the opportunity to present relive XR together with Christoph Spinger on the VR-Nerds channel, many thanks for that!
Playtest by Christoph Spinger on the VR-Nerds Youtube channel
relive product design
The entire digital design was based on the vision of creating a company and the right face for it. This applies to the image, the visual language and ultimately also to the medium used to present the company. What distinguishes Relive are the first products and their simple approach.
A ring to store data as personally and directly as possible. A ring is a symbol and acts as an interface to the relive cloud. The ring is a natural interface, as the cloud is primarily responsible for sensor management and networking.
I have spent a lot of time thinking about data and especially where data is stored, so I am convinced that this will become more and more important as people become more aware of it. The result is a simplified 3D logo that can be used in various media.
relive the exhibition
Of course, my diploma was also exhibited in public, for which I made a special stand that, mapped to the column in the virtual space. It was a seamless transition into the virtual world. As a slight innovation, the VR headset could be placed on top of it, because this is often not considered at exhibitions.
The effect the different worlds had on people was incredible, from excitement, sadness, longing, to an urgent desire to see more.
The future of relive
Of course, my idea was stolen and commercialized long ago, maybe seconds after I got the idea. By creating relive XR, I have reaffirmed that value and found a way to express it, even if it is temporary. What will never lose its value, however, is the desire to relive beautiful moments in the most immersive way possible. There are approaches to "reliving" these moments. They have been integrated into applications for the elderly. There are hospitals that use VR for pain processing and pain management.
The idea is still with me, but due to my management position today, I am unfortunately too busy to do anything like test the gaussian splatting approach. Apple's new headset alludes to this once again, so we can look forward to seeing how Apple manages the relive moment.
Here are a few funny approaches that literally use the naming:
- AMDs "Radeon ReLive for VR"
- XR RELIVE by splendour (relive on demand for concerts ect.)
- Last but not least Apples Approach with the vision pro, that almost sounds like the text i added to my website with a simple slogan: "Your memories come alive."
Retrospective
Especially the aspect of the perfect presentation was very challenging and demanded a lot from me. I wanted to put everything on the line so that the result would have a special depth, especially for people who experience it immersively with VR headsets. The technical part could be worked out and perfected, the particle animations in combination with the moving content had a good effect, today with modern, powerful hardware more would have been possible. In retrospect, I would have used a lot more photogrammetry today for close-up objects. From a distance of about 2m, Gaussian splatting can be used.
All in all I got the absolute maximum out of what I could achieve in the time available to me. However, I could have improved my results with more effective planning and more budget.
References
For relive's experience design, I drew inspiration from a variety of sources, these are the most important ones / humans behind:
- Mike Alger
- Golden Krishna
- Alex Chu
remarks and acknowledgements
Fortunately, there were people who accompanied me and believed in me during the intensive time with relive. despite the fact that I was running around with huge eye rings and talking crazy stuff about DNA-based data storage.
- Alexander Meinhard, my coach, a top designer with a unique sense for colors, typography and interfaces. His advice and encouragement pushed me to go far beyond my own imagination. What must have seemed completely crazy to some outsiders he accepted as creative process. I still have unique memories of that time. For example, the look on the cafe owner's face when I draped my Oculus Rift with multiple sensors on one of the tables.
- Prof. Dr. Alexander Kutter the then CEO of VR-Nerds Solutions, where I was freelancing at the time. He supported me and believed in my ideas.
- Patrick Wienzek, the talented and above all patient XR developer who conjured up a lot of magic in VR for VR-Nerds in a uniquely pragmatic way.
- Julian Heineken, the then already experienced tech and shader developer, who helped me especially with the visual implementation of the memory spheres.
- Christoph Spinger for his feature on the VR-Nerds news channel and the refreshingly funny way he presented my project - in the GAMEPLAY 👆🏽
