Third Party Software

CMake

Qt

OptiX

Cuda

Compiling ElVis

Setting The Compute Capability

ElVis Options

Regression Tests

Setting Up

Windows

• Open the Control Panel then the NVIDIA Control Panel. 
• Go to the Set up multiple displays option to see which GPUs are connected to your monitors.

• Click on System Information (lower left corner) to see which number has been assigned to the GPU connected to the monitor on which you will use ElVis.

• Set the environment variable CUDA_VISIBLE_DEVICES=, where X is the number from the system information dialog.  Note that the environment variable starts at 0, while the system information starts at 1, so you must subtract 1 from the number obtained from the system information dialog.

Running ElVis

Loading Extensions

Visualizing Cut Planes

 

Extension Types

ElVis can be extended with two different types of extensions.  In the following sections, we describe each type of extension and discuss their pros and cons.

Model Conversion Extensions

A conversion extension is used to convert a finite element volume to the Nektar++ format used internally by ElVis.  This type of extension is most appropriate for volumes that have the following properties:

• All elements are represented by hexahedra, tetrahedra, prisms, or square-based pyramids.

• The field is represented by a polynomial.

If the volume contains elements of different types than those listed above, a conversion exetension may still be created, but the elements must be converted into one of the four element types indicated above.  If the field is not represented by a polynomial, then the converted volume will contain error.  In this case, we recommend that a runtime extension be used.

The model conversion extension's relys on the Nektar++ extension for all runtime processing.  Since the Nektar++ extension only currently supports hexahedra, model conversion extensions only support hexahedra as well.  Additional functionality is currently being added to the Nektar++ extensions to support the remaining element types.

Runtime Extensions

Runtime extensions give ElVis the ability to access a simulation's native data, rather than first converting it to the Nektar++ format.  It does this through a collection of high-level OptiX and Cuda functions that must be implemented in your extension.

Runtime extensions are implemented as shared libraries that are loaded by ElVis at runtime.  Each runtime extension consists of three components, described in further detail below.

• C++ Interface.  This interface is responsible for interfacing with the ElVis gui and handling the movement of model data from main memory to the GPU.
• OptiX Interface.  This component is responsible for providing methods to sample a field on the GPU in OptiX.
• Cuda Interface.  This component is responsible for providing methods to sample a filed on the GPU in Cuda.

The OptiX and Cuda interfaces are both written in Cuda.  Cuda does not support linking of pre-compiled object files, so ElVis uses a different methods to provide customized behavior for each extension.

Creating Runtime Extensions

In this section, we will create a fully-functional runtime extension to illustrate the necessary steps and code.  This section, we assume that there is an ElVis source distribution.  We will refer to the location of the source distribution with the variable $ELVIS.  Therefore, the source files are located at $ELVIS/src, the build is at $ELVIS/Build, and the ElVis distribution is at $ELVIS/build/dist. To simplilfy the initial extension development steps, we have included a runtime extension template at $ELVIS/src/ElVis/Extensions/RuntimeExtensionTemplate.  This template will enable the creation of a simple runtime extension that can be loaded into ElVis (although it will have no functionality).  The code that is located in this directory is not able to be compiled directly; rather, CMake must first be used to generate the runtime extension code, customized with your extension's name.

These steps generate the source code for the new runtime extension, which is located at $BUILD_BASE/$RUNTIME_EXTENSION_NAME (which we will refer to as $EXTENSION_BASE).  The folders $BUILD_BASE/Build and $BUILD_BASE/CMakeFiles are not necessary and can be deleted, as well as the files $BUILD_BASE/CMakeCache.txt, $BUILD_BASE/cmake_install.cmake, and $BUILD_BASE/Makefile.

C++ Interface

The C++ interface is responsible for loading high-order data from disk and tranferring it to the GPU for visualization.  It is also responsible for handling user interaction and populating the ElVis gui.  This is primarily accomplished through concrete subclasses of ElVis::Model.  Each extension is required to have a model-specific version of this class available.

When an extension is initially loaded, ElVis looks for the following three methods, which must exist at the global namespace, have C linkage, and be visible outside the shared library.  On Linux/OS X, these methods will be visible by default.  On Windows, however, they are not, and must be explicitly exported (see ElVis/Extensions/JacobiExtension/Declspec.h for an example).  These methods provide the initial interface to the extension.

const char* GetPluginName();

Each plugin must provide a name that is used in the ElVis gui to differentiate it from the other loaded plugins.  There are no restrictions on the name.

const char* GetVolumeFileFilter();

Returns a file name filter for use by the Qt QFileDialog class to open volume.  The format of the string is " (*.)".  For example, "Jacobi Volumes (*.dat)".

ElVis::Model* LoadModel(const char* path);

Loads the model from the given location and returns a pointer to a new Model object representing it.  ElVis will take control of this pointer and handle cleanup.

The runtime extension template provides default implementations of LoadModel and GetPluginName that are unlikely to need modification.  The implementation of GetVolumeFileFilter, however, will need to be modified to correspond to the extension used by the model.

The next step is to fill in the implementation of the model class (located in $EXTENSION_BASE/Model.h and $EXTENSION_BASE/Model.cpp). For this sample extension, we will build a model that can visualize a simple high-order system that consists of axis-aligned hexahedra.

File Format

The file format we will use is a binary format with the following structure:

sizeof(int) : Number of Vertices
sizeof(double) * 3 * Number of Vertices : Vertex Array
sizeof(int) : Number of faces
sizeof(int) * 4 * Number of faces : For each face, an index into the vertex array for each of the face's vertices
sizeof(int) : Number of hexahedra
sizeof(int) * Number of hexahedra : For each hex, an index into the face array for each of the hexahedra's faces.
sizeof(int) : Number of coefficients
sizeof(double) * Number of coefficients : Coefficient array.
sizeof(int) * 3 * Number of hexahedra : Modes

In this example, there is a single scalar field associated with the volume. This example uses a modal basis. This is not a restriction imposed by ElVis and is simply the example used. Volumes using a nodal basis can be used as well.
Loading a volume from disk is performd in the LoadModel method. The default behavior of this method is to call the SampleRuntimeExtensionModel constructor that takes a single parameter (the location of the volume on disk). So the first step is to populate this constructor with the appropriate code to perform this task.

                SampleRuntimeExtensionModel::SampleRuntimeExtensionModel(const std::string& path) :
            Model(),
            m_numVertices(0),
            m_vertices(0),
            m_numFaces(0),
            m_faces(0),
            m_numHexes(0),
            m_hexes(0),
            m_numberCoeffs(0),
            m_coefficients(0),
            m_coefficientMappings(0),
            m_modes(0)
        {
            // Load the model from disk, and populate all of the internal data.  The constructor
            // is not responsible for populating any GPU constructs.  That will be handled by
            // some of the class' virtual functions.

            FILE* inFile = fopen(path.c_str(), "rb");
            if( !inFile )
            {
                // ElVis should never pass an invalid file name to the constructor, but, if it
                // does, ElVis will handle the exception and abort the model load procedure.
                throw std::runtime_error(std::string("File ") + path + " does not exist.");
            }

            // Read all vertices into a vertex array.
            fread(&m_numVertices, sizeof(int), 1, inFile);
            m_vertices = new double[m_numVertices*3];
            fread(m_vertices, sizeof(double)*3, m_numVertices, inFile);

            // Faces are represented by the verteices that make up the face.
            // Each face is therefore represented by the indices into the
            // vertex array.
            fread(&m_numFaces, sizeof(int), 1, inFile);
            m_faces = new Face[m_numFaces];
            for(int i = 0; i < m_numFaces; ++i)
            {
                fread(&(m_faces[i].VertexIndex[0]), sizeof(int), 4, inFile);
            }

            // Each hex is represented by the 6 faces.
            fread(&m_numHexes, sizeof(int), 1, inFile);
            m_hexes = new Hex[m_numHexes];
            for(int i = 0; i < m_numHexes; ++i)
            {
                fread(&(m_hexes[i].FaceIndex[0]), sizeof(int), 6, inFile);
            }

            // Coefficients are stored in a global coefficient array, rather than with each
            // element.  The GPU implementation often works better this way.
            fread(&m_numberCoeffs, sizeof(int), 1, inFile);
            m_coefficients = new double[m_numberCoeffs];
            fread(m_coefficients, sizeof(double), m_numberCoeffs, inFile);

            // Since each element can have a different number of coefficients,
            // we use this array to indicate where in the coefficient buffer each
            // element's coefficients begin.
            m_coefficientMappings = new int[m_numHexes];
            fread(m_coefficientMappings, sizeof(int), m_numHexes, inFile);

            // Each element can have a different number of modes.  We store the
            // modes in this array, indexed by element number.
            m_modes = new int[m_numHexes*3];
            fread(m_modes, sizeof(int)*3, m_numHexes, inFile);

            fclose(inFile);
        }

Reporting Interface

Several of the Model class' virtual functions are reporting functions that inform ElVis about the model.

        int SampleRuntimeExtensionModel::DoGetNumFields() const
        {
            return 1;
        }

        FieldInfo SampleRuntimeExtensionModel::DoGetFieldInfo(unsigned int index) const
        {
            if( index > 0 )
            {
                throw std::runtime_error("Invalid field index.");
            }

            FieldInfo result;
            result.Name = "Density";
            result.Id = 0;
            result.Shortcut = "Ctrl+D";
            return result;
        }

        unsigned int SampleRuntimeExtensionModel::DoGetNumberOfElements() const
        {
            return m_numHexes;
        }

        void SampleRuntimeExtensionModel::DoCalculateExtents(WorldPoint& min, WorldPoint& max)
        {
            min = WorldPoint(std::numeric_limits::max(), std::numeric_limits::max(), std::numeric_limits::max());
            max = WorldPoint(-std::numeric_limits::max(), -std::numeric_limits::max(), -std::numeric_limits::max());

            // Calculating extents this way is only valid for linear geometry.
            for(int i = 0; i < m_numVertices; ++i)
            {
                WorldPoint p(m_vertices[i*3], m_vertices[i*3+1], m_vertices[i*3+2]);
                min = CalcMin(p, min);
                max = CalcMax(p, max);
            }
        }

For some volumes, the behavior of the field around some surface is of interest, and the surface coincides with element faces. This often occurs at the boundary between the elements and some geometry of interest. For 3D volumes, ElVis supports the collection of element faces into groups called Boundary Surfaces. Boundary surfaces can be rendered using color maps and contours.

        int SampleRuntimeExtensionModel::DoGetNumberOfBoundarySurfaces() const
        {
            return 1;
        }

OptiX Interface

The OptiX interface can be found in ElVis/Core/ElVisOptiX.cu.  Each of the methods in the interface section must be implemented by the extension in a filed called ExtensionOptiXInterface.cu.

Cuda Interface

The Cuda interface can be found in ElVis/Core/ElVisCuda.cu.  Each of the methods in the interface section must be implemented by the extension in a filed called ExtensionCudaInterface.cu.