Log In Sign Up

A Proposed Framework for Interactive Virtual Reality In Situ Visualization of Parallel Numerical Simulations

by   Aryaman Gupta, et al.

As computer simulations progress to increasingly complex, non-linear, and three-dimensional systems and phenomena, intuitive and immediate visualization of their results is becoming crucial. While Virtual Reality (VR) and Natural User Interfaces (NUIs) have been shown to improve understanding of complex 3D data, their application to live in situ visualization and computational steering is hampered by performance requirements. Here, we present the design of a software framework for interactive VR in situ visualization of parallel numerical simulations, as well as a working prototype implementation. Our design is targeted towards meeting the performance requirements for VR, and our work is packaged in a framework that allows for easy instrumentation of simulations. Our preliminary results inform about the technical feasibility of the architecture, as well as the challenges that remain.


Applications and a Three-dimensional Desktop Environment for an Immersive Virtual Reality System

We developed an application launcher called Multiverse for scientific vi...

Data Visceralization: Enabling Deeper Understanding of Data Using Virtual Reality

A fundamental part of data visualization is transforming data to map abs...

A Perception-Driven Approach To Immersive Remote Telerobotics

Virtual Reality (VR) interfaces are increasingly used as remote visualiz...

Conceptual Design and Preliminary Results of a VR-based Radiation Safety Training System for Interventional Radiologists

Recent studies have reported an increased risk of developing brain and n...

Immersive VR Visualizations by VFIVE. Part 2: Applications

VFIVE is a scientific visualization application for CAVE-type immersive ...

GeneNet VR: Interactive visualization of large-scale biological networks using a standalone headset

Visualizations are an essential part of biomedical analysis result inter...

UrbanVR: An immersive analytics system for context-aware urban design

Urban design is a highly visual discipline that requires visualization f...

1 Rendering and Simulation Libraries Used

The rendering system used in our framework is developed using scenery [gunther2019scenery], an open-source VR library for both volumetric and geometric data. In simulations, volumetric data typically stem from discretized continuous fields and geometric data from surface meshes or postprocessing results, such as isosurfaces or streamlines. scenery supports rendering to both commodity VR headsets and distributed setups, such as VR CAVEs and PowerWalls. It uses the high-performance Vulkan API for rendering, and therefore has the potential to harness the power of modern GPUs better than traditional OpenGL-based solutions, as still predominantly used by in situ solutions like ParaView Catalyst [ayachit2015paraview] and VisIt Libsim [kuhlen2011parallel].

The simulation engine used in our framework is OpenFPM [incardona2019openfpm], an open-source library that enables rapid development of scalable and efficient particle- and mesh-based numerical simulations. OpenFPM largely automates parallelization on high-performance computers, including distributed-memory systems, many-core systems, and GPGPUs. It has been used to implement simulations ranging from Smoothed-Particle Hydrodynamics to Molecular Dynamics to Discrete Element Methods on thousands of CPU cores [incardona2019openfpm].

Integrating our work into OpenFPM helps us reach a wide audience of computational scientists who benefit from interactive in situ visualization and steering, but may lack the time or expertise to write scalable solutions from scratch. We implement in situ functionality into OpenFPM in a manner that allows users to enable it in a fraction of the time and effort it normally takes to instrument a simulation with an in situ solution.

2 In Situ Architecture

We design our in situ architecture targeting a rendering frame rate of at least 60 Hz, rendering latency on viewpoint changes of at most 20 ms, and low interaction latency for steering commands.

Figure 1: Schematic of our framework architecture.

Key to achieving these goals is to minimize data movement. We thus choose a tightly-coupled structure with simulation and rendering sharing resources, running on the same nodes within a cluster. Figure 1 shows a schematic. Each compute node runs the distributed OpenFPM environment for the simulation, along with a scenery-based application that performs local rendering. All simulation data generated on a node are rendered on the node itself. The simulation data are shared between OpenFPM and scenery via a common pointer to shared memory. This enables zero-copy rendering.

Several in situ solutions, including ParaView Catalyst [ayachit2015paraview] and VisIt Libsim [kuhlen2011parallel], adopt a synchronous execution strategy, where simulation and visualization run sequentially. We choose an asynchronous strategy, aiming to reduce interaction and rendering latency. In our asynchronous strategy, neither simulation nor visualization need to wait for the other, which means that steering commands can be incorporated at the earliest, and rendering can begin while the simulation is in progress. Visualization artifacts that may result from this lack of synchronization are expected to be small, since numerical stability requires the simulation to only change slightly between time steps. For our application, we emphasize low latency over these minor visual artifacts.

The rendering results produced on the compute nodes are then composited using a divide-and-conquer strategy similar to the binary swap algorithm [ma1994parallel]. The scenery process on the Head Node is responsible for streaming the composited result to a remote client, as well as for receiving requests for changes in visualization parameters. The remote client also communicates steering commands to the OpenFPM process on the Head Node, which conveys them to all OpenFPM processes via MPI messages.

In contrast to other approaches, we intend to render an explorable representation, such as a Volumetric Depth Image (VDI) [frey2013explorable], instead of a simple image. Such representations of volumetric data can, once generated, be reprojected to a new viewpoint without redoing the ray casting. We intend to render such a representation on the cluster and stream it to the remote client. Small viewpoint changes can then be handled locally by the remote client without involving network latency. This is crucial for meeting the latency requirements of VR, where viewpoint changes occur frequently.

3 Preliminary Results

We have implemented a prototype of our framework, performing particle-as-sphere rendering of a distributed OpenFPM simulation. The simulation runs parallel on multiple CPUs, while visualization is done on the GPU. The shared-memory communication between the processes on a single node is extended by a protocol to handle dynamic memory requests and reallocations. The scenery application reads the simulation data from shared memory and understands the data structures used by OpenFPM. Our implementation is open source. The scenery application can be found at and the OpenFPM project at (in the “insitu_visualization” branch).

Figure 2 shows our prototype applied to a Molecular Dynamics simulation. The simulation code written using OpenFPM is only 161 lines of C++, and the inclusion of in situ visualization requires changes in only two lines.

Figure 2: An OpenFPM-based distributed Molecular Dynamics simulation visualized in situ using our prototype, and the lines of code that needed to be changed to enable in situ visualization. Particles are colored by velocity magnitude.

4 Future Work and Conclusions

We have presented a framework architecture for interactive VR in situ visualization of parallel numerical simulations. We used scenery and OpenFPM to base the rendering and simulation parts of the framework on. We have implemented a first prototype of the system, demonstrating the feasibility of achieving our design goals. In our prototype, enabling live in situ visualization only required minimal changes in the existing simulation code.

Our prototype does not yet meet the targets we set for immersive in situ visualization in Section 2, but it enables us to identify a number of challenges that should be addressed in order to meet them: First, volumetric rendering results need to be in an explorable representation, such as a VDI, as explained in Section 2. Second, we need a solution to efficiently generate and composit such a representation on a cluster. Third, support needs to be added for simulations running on GPGPUs. In addition, we are going to extend our framework to NUI-based steering by instrumenting OpenFPM to receive steering commands on the fly and by expanding scenery’s support for NUI interaction.

Taken together, we are confident that many applications in scientific computing will benefit from a more immersive and more explorative mode of interaction.