The visual demands of people have changed dramatically in recent years. We expect to see results that achieve a very high level of realistic representation. This expectation has been significantly increased by the gaming industry and the progress of VFX for film and television production.
Using Stadia's server power, Google presents Immersive Stream for XR
The perhaps most decisive question in advance, is there realistic metaverse implementations? Absolutely, but very few and at a very high price. The most impressive results are known to me from the automotive industry.
Key Aspects of Pixelstreaming
High-Quality Streaming of VR and AR Content
Pixelstreaming uses various compression techniques to reduce the amount of data that needs to be transmitted while streaming virtual reality (VR) and augmented reality (AR) content. These techniques include lossless compression, lossy compression, spatial compression and temporal compression. These methods help in maintaining high resolutions and frame rates while streaming the VR and AR content and provide a smooth and seamless experience for users.
Pixelstreaming means Game Engine from the server to your device.
Pixel Streaming, also known as Render Streaming or Remote Rendering, is a technology that enables the streaming of hosted cloud software to the client. This technology allows users to access and interact with highly detailed and realistic virtual worlds through the use of cloud-based game engines. Game engines are the source of truth for virtual environments, they provide the physics, lighting, and other elements that make the virtual world feel real. With Pixelstreaming, users can experience these virtual worlds in real-time and with a high level of interactivity, without the need for bulky hardware or high-end computers. Since around 2020, several visually impressive virtual worlds have been created using Pixelstreaming and made available to users on the web.
Accessibility over the Web
Pixel Streaming is a technology that enables real-time streaming of virtual environments hosted on cloud-based game engines, to the client. Game engines provide the necessary physics, lighting, and other elements to create realistic virtual environments, making them the "source of truth" for virtual worlds. The use of cloud-based game engines allows for the use of powerful graphics processing units (GPUs) to create high-quality, realistic virtual environments that can be streamed to a variety of devices.
Cloud-based Game Engines and Rendering Software
Pixelstreaming allows for the use of cloud-based game engines and rendering software, enabling users to experience high-quality VR and AR content without the need for powerful local hardware.
Low Latency and High Frame Rate
Pixelstreaming maintains low latency and high frame rate allowing for the creation and streaming of highly detailed and realistic virtual environments.
- For more information on Pixelstreaming using WebRTC, see WebRTC for Unity
- For more information on Pixel Streaming for WebXR, see Pixel Streaming for WebXR
- For more information on technical details of
Applications of XR Pixelstreaming
XR Pixelstreaming is not just for gaming and entertainment. It has a wide range of potential applications in various industries:
- Education: Imagine being able to explore a virtual model of a building before construction even begins, or taking a virtual field trip to a distant location without ever leaving the classroom.
- Healthcare: XR Pixelstreaming can be used to create virtual reality simulations for medical training, therapy, and patient education.
- Architecture: XR Pixelstreaming can be used to create virtual walkthroughs of buildings and interior design, allowing architects and clients to make changes before construction begins.
- Manufacturing: XR Pixelstreaming can be used to train employees on virtual assembly lines and machinery.
- Real Estate: XR Pixelstreaming can be used to create virtual tours of properties, allowing potential buyers to explore a home without ever stepping foot inside it.
- Fasion: Complex cloth animations and simulations of new collections can be realized with XR Pixelstreaming. Avatars can be used and real-time motion tracking is possible.
XR Pixelstreaming is a relatively new technology, however, it is advancing quickly. Companies such as Google and Facebook are investing heavily in research and development, and it is expected that XR Pixelstreaming will become more prevalent in the near future. Despite this, it is already available in some resources and has shown promising results.
XR Pixelstreaming guides and resources:
Getting Started with Pixel Streaming using Unreal Engine
- Getting Started with Pixel Streaming: in Unreal Engine
- Hosting and Networking Guide
- For this guide, you can use the Pixel Streaming Demo provided by Epic
Great pile of resources by TensorWorks.
Some general guides for the hosting servers
- Pixel Streaming on AWS (Linux)
- Pixel Streaming on AWS (Windows)
- Pixel Streaming on Azure (Linux)
- Pixel Streaming on Azure (Windows)
- Pixel Streaming on GCP (Linux)
Other nice resources by TW
- A GitHub repository with UE4 docker containers for both Linux and windows
Conclusion - Unlocking the Potential of Virtual Reality with XR Pixel Streaming
In conclusion, XR Pixelstreaming is the key to achieving realistic virtual assets. As the dominant technology in the market currently focuses on simple streamed assets, it is expensive to render for a large number of users. The AR application discussed in the linked article is a perfect example of this, as it demonstrates how cost-efficient XR pixel streaming can be. However, as the use cases for XR continue to expand and include bigger "virtual scenes" or "metaverses", the cost efficiency becomes a concern.
Epic Games, with their in-game purchase finance model in Fortnite, have shown that it is possible to make these bigger virtual worlds cost-efficient. Google is also a strong player in this field, holding patents even after the failure of their Stadia project. From the perspective of a backend server owner, it is helpful to have big players such as Google in the field as they can play a helpful role for B2C studios and content creators. As an XR experience designer, the idea of using a realtime engines to enhance asset quality is exciting. Humans expect realism for marketing assets and my prediction is that mobile applications will continue to grow, as seen with Fortnite. Additionally, with the advent of 5G and edge computing, I have high hopes for more XR applications that can be connected to phones via XR headsets.
Overall XR Pixel Streaming is a promising technology that will open new possibilities in the field of XR and make it more accessible for users and developers alike.
