Visualisation in IPython with LavaVu¶

A python module for 4D visualisation with interactive IPython notebook integration¶

Owen Kaluza - Monash Immersive Visualisation Platform¶

Monash University e-Research Centre

What is IPython?¶

IPython: interactive shell combining documentation, annotation, implementation(code) and publication
Markdown formatted text, MathML formulae, images, video, HTML controls and components
Lends itself very well to use with remote resources, virtual machines and containers.
Used increasingly widely in research for simulations, data analysis, machine learning and general research data processing

https://jupyter.org/ https://try.jupyter.org/

Everyone here is probably familiar with IPython but I'll give it a quick summary anyway.

IPython is an interactive python shell which combines documentation, implementation and publication features into one resource by bringing python to the web browser.

Python code, Markdown formatted text, mathematical formulae, images, video, HTML are all rendered in a web browser interface.

All the backend hardware and software can be remote, so it's great for use on cloud resources.

It's used increasingly widely in research for building of simulation models, machine learning, general research data analysis.

IPython documents can be converted to plain python scripts or various other formats, including presentations such as this one.

The Jupyter project is the parent project of IPython, now bringing the same features to other languages such as R and Julia.

You can see a quick demo at jupyter.org, most examples make heavy use of its matplotlib integration features for creating figures.

logo

A 3d(4d) Visualisation library : developed for advanced visualisation facilities and next-generation HPC resources¶

C++ backend with python interface wrapper provides a native OpenGL graphics layer to IPython
HTML5 GUI: usable from IPython or stand-alone via the built in web server
Geophysics: Underworld2 simulation code http://www.underworldcode.org/
CAVE2: via LavaVR, cluster rendering and CAVE2 VR mode, Volumes, 4d data, Surface Models
Output: still images, video animations, 3D interactive interfaces, 3D snapshots in WebGL.

Underworld: A parallel, particle-in-cell, finite element code for Geodynamics¶

Underworld is an open-source, particle-in-cell finite element code tuned for large-scale geodynamics simulations. The numerical algorithms allow the tracking of history information through the high-strain deformation associated with fluid flow.
The finite element mesh can be static or dynamic, but it is not constrained to move in lock-step with the evolving geometry of the fluid.
This hybrid approach is very well suited to complex fluids which is how the solid Earth behaves on a geological timescale.

Underworld2: python driven geophysics simulation http://www.underworldcode.org/¶

Underworld2 is a python-friendly rework of the Underworld code base which provides a programmable and flexible front end to all the functionality of the code running in a parallel HPC environment.

Underworld2 is integrated with the literate programming environment of the jupyter notebook system for tutorials and as a teaching tool for solid Earth geoscience.*

*https://github.com/underworldcode/underworld2

The project has undergone a significant redevelopment and been re-released as Underworld 2. The underlying simulation engine is the same but the interface is completely re-worked to be driven by python and a rich set of examples and documentation built using IPython notebooks, designed to integrate with the arrival of cloud based resources.

The speed of compiled C code is retained for all the key algorithms, but the interface and input files are all via python scripts with an inline visualisation system for instant feedback and plotting of simulation results.

This instant visual feedback enables a "human in the loop" approach to analysis.

The greatest gains provided by this next generation of tools simply came out of re-engineering Underworld to better leverage new tech when solving the main problems of Underworld1:

Underworld2 changes:¶

Problems addressed:

Build and deployment: Docker
Lack of flexibility and arcane XML model format: Python
Difficult to learn and insufficient or out of date documentation: IPython
No GUI features for visualisation: HTML5/CSS3/Javascript + IPython

LavaVu: a lightweight rendering library for scientific visualisation¶

https://github.com/okaluza/lavavu

C++ powered A C++ OpenGL rendering engine compiled to a binary library
Python Driven A Python interface wrapper module with underlying SWIG bindings and IPython supporting code
Web GUI - JavaScript,HTML,CSS interactive components, including WebGL output
JSON data for communication of state
Numpy input for fast loading of native data arrays

I eventually rewrote the rendering component of gLucifer as a self-contained 3D scientific visualisation tool : LavaVu.

gLucifer remains as the sampling and parallel data compositing layer in Underworld with its own python interface for plotting Figures utilising the LavaVu library for rendering image output.

Following the pythonisation of Underworld 2, LavaVu underwent the same process, using SWIG to generate bindings and writing wrapper modules to translate from python to C++.

There are now three layers to the software, the C++ rendering engine, python wrapper and Web based GUI. A JSON property dictionary provides communication of state data between the layers.

Modifications to the state data through the browser or python script are passed through to the underlying C++ renderer. New vis features are easily implemented by defining and using a new property in C++, which immediately becomes available to the upper layers without having to write new wrapper functions.

Numpy interfaces allow rapid processing and loading of large amounts of numerical data.

CAVE2: Challenges of Virtual Reality Visualisation¶

Big data visualisation on CAVE2 and clustered display systems:¶

Data processing: we have to pare down large datasets that can't be rendered at interactive framerates via interactively selecting, filtering, resampling, cropping data
Then we can usually easily load a data set, but what next?
Need the ability to control/explore data by modifying vis properties interactively

While all this was going on, the CAVE2 was built and we now had a whole new set of visualisation challenges.

The CAVE2 and other large display systems provide screen resolution to match huge data sets - more pixels provide more visual bandwidth but and also require more raw computing power in the form of a cluster.

So the real challenges are in software...

For initial data exploration, the data is often too large and visually incomprehensible for VR use without significant post-processing and reduction.

Virtual Reality and clustered display systems rely on custom tools to interact with data, and high frame-rates for interactive viewing. Desktop software just doesn't translate well.

When used with well designed and carefully crafted visualisations targetting these systems we can achieve compelling tools for analysis of visual spatial data and wow-factor in communication of research for outreach.

But most researchers already have their favourite software tools, for visualisation these are often all-in-one desktop applications that can't be used on a cluster based system like the CAVE2 or easily integrated into a more flexible pipeline.

The power of python is in the way it encourages the approach of using many smaller tools to put build a custom work-flow and this can be a far more flexible, powerful and agile way of doing things.

Fortunately IPython is making this approach much more accessible.

Why LavaVu?¶

What it does well:

Use with the vast library of tools available in python
Remote rendering and graphics on HPC systems while simulations are running using Mesa (software OpenGL)
Server side render, client side control - via built in HTTP server and IPython.
Correctly render transparency in large point swarms and surfaces
Combined vis elements: volume/surface/point/line

Why LavaVu?¶

What it does well (cont.):

Built for handling time-varying 4D data sets: simulation data
IPython notebooks: analyse / visualise / process with IPython, instant vis results as part of the analysis workflow.
CAVE2 and VR: pathway direct to CAVE2 with our VR module, LavaVR
Scriptable image and video output, save and restore state, record and checkpoint stages of vis workflow
Scriptable custom HTML5 GUI components

Intrinsically supports 4D data, particularly simulation data, with multiple levels of caching in RAM or GPU ram that can be enabled or disabled.

IPython notebook integration and custom user interfaces, using notebooks, vis becomes an intergrated part of the analysis workflow.

VR support: any visualisations created using this library can be loaded straight into the CAVE2 in our VR module, LavaVR to run in the CAVE2.

Completely scriptable including image and video output, state save and restore so once desired images/videos are created they can be saved and re-rendered easily.

There is no standard GUI but scriptable custom GUI components so you can have as much or little GUI control as you like. The down side is you need to work from examples to build and use these interfaces as they can be hard to implement just from API docs.

For a plotting tool to create data vis, charts, or 2d graphs, there are definitely better choices.

Consider it if the goal is scientific 3D vis: that is to render complex 3D and 4D data in order to better visually analyse it and building a workflow using python or IPython is a drawcard.

How to get it?¶

Github:
https://github.com/okaluza/lavavu

Instructions for building on Mac/Linux, including with IPython in the wiki:
https://github.com/okaluza/lavavu/wiki

Windows build: out of date due to lack of demand

Docker: (CPU and GPU versions to come) https://hub.docker.com/r/lavavu/latest/

Docker run on Linux with GPU access (open-source drivers only):

xhost +
docker run -v $HOME:/workspace/home -p 8888:8888 --volume=/tmp/.X11-unix:/tmp/.X11-unix --device=/dev/dri:/dev/dri --env="DISPLAY" lavavu/latest:v1.1

It's all open-source and the github repository contains simple instructions to build the code from source on Unix based systems

Complete instructions including installing IPython are also found on the github wiki. There are very few required dependencies, mainly only those needed by Jupyter.

I used to do a windows build but it has fallen by the wayside due to lack of demand. If anyone requests I'd happily update it.

I've also recently built a docker image, due to the use of OpenGL it can be trickier to get going with GPU support, requiring permission to use the display.

A CPU only docker image that uses the Mesa llvmpipe OpenGL driver will run without these issues and will render using all CPU cores, it only really falls down when you try and use it for volume rendering tasks (dropping below 1 frame per second)

How to use it?¶

Tutorials/Examples:

Other examples from CAVE2 projects:

Underworld simulation models:

Underworld model visualisation:

I'll run through a few examples to show how it works.

Slow run through: particles

Faster:

Surface
Rabbit
DEM
Carbon

Built-in support for data files includes:

Meshes in Wavefront OBJ format
Raw volume data
TIFF image stacks or wildcard JPG/PNG images to load as volumes
DEM height maps
GLDB (vis database format that uses SQLite3, designed for gLucifer/Underworld to store time-varying Geophysical simulation data - models can be exported from LavaVu to this format to package them neatly in a single file)

Support for further formats is provided with example python scripts that process the data from python and pass it to LavaVu, such as:

Point clouds in text and binary formats (X,Y,Z,R,G,B etc)
Tecplot
HDF5

Documentation for all the properties and commands can be found on github, here: http://github.com/OKaluza/LavaVu/wiki

Conclusions¶

Convergence
One puzzle piece in e-Research workflows
Future work?

Thanks¶

MeRC
My colleagues at MIVP
The Underworld 2 team and researchers

So to sum up, it's been an interesting journey so far trying to bring these multiple projects together.

One of our aims at MIVP now is instead of being purely an end-point for visualisation, if our development efforts can provide useful additions to the vis work-flow outside of our facilities, then these facilities become an effortless addition to that work-flow that researchers can easily choose to utilise.

With flexible and modular vis tools accessible from the widely used python ecosystem, that are usable remotely on cloud resources, there is more potential to integrate with other e-Research tools and resources become part of an overall flexible analysis and visualisation pipeline.

If we succeed at this software based challenge, then we can much better utilise the hardware provided by our current and future high-end vis and VR facilities as they will seamlessly become an optional part of a scientific workflow going from Capture, Storage, Collaboration, Analysis & Visualisation through to Publication and Communication.

The work undertaken is already being utilised succesfully by researchers in the Geophysics domain, and greatly aids with our work in the CAVE2 and recent development efforts have been concentrated on attempting to improve accessibility to the wider community.

It's been a little bit more ambitious and time-consuming than I realised however, to try and polish some of these ideas into a finished product from prototypes, so from here it's a question of seeing if it proves worthwhile.

Our future goals include getting this stuff running in head mounted VR and integrating some of the various development efforts at MIVP together.

Thank you for listening, and I'm happy to answer any questions.