System Requirements
Barracuda Virtual Reactor’s general System Requirements apply to High-Performance Computing (HPC) systems including clusters. Here are a few specific points to keep in mind for such systems:
- Virtual Reactor does not support multi-node calculations. Instead, each Virtual Reactor simulation runs on a single node and uses only that node’s resources such as CPU, memory, GPU card(s), etc.
- CPUs with high clock speed are best for running Virtual Reactor. When possible, choose a node with the fastest clock speed CPU available on your cluster.
- GPU acceleration gives the fastest possible simulation speed for Virtual Reactor. When possible, choose a node with the newest generation and highest CUDA core count NVIDIA GPU card(s) available.
Installation
Virtual Reactor must be installed, or available via a network mount, on each node that will be used to run simulations. On many HPC systems a graphical desktop environment is not installed, so running the installer in headless mode is often required. The most common process for installation on such systems is:
- Download the latest version of Barracuda from CPFD’s Support Site dashboard page.
- Extract the .tar.gz file (assuming a Linux OS environment, which is by far the most common for HPC systems).
- Run the Barracuda installer as root using a command of the form:
barracuda_virtual_reactor-26.0.0-Linux.run install --default-answer --accept-licenses --confirm-command --root /opt/CPFD/Barracuda/26.0.0
In the above command, replace the version number with the appropriate value based on your downloaded version. You can also specify a different –root directory if you have a preferred installation location.
License Configuration
As discussed in Planning the RLM Client-Server Configuration and Configure License (from the Installation chapter of the User Manual), Virtual Reactor can use either an on-premise RLM server or it can check out licenses from RLMCloud.
- In most cases, using RLMCloud is the most convenient solution. It eliminates the need to manage and maintain your own dedicated license server.
- However, if your cluster does not have access to the internet you will need to install an on-premise RLM server. See On-Premise RLM Server Installation Files for details.
To point client machines (i.e. calculation nodes) to your RLM server, define the cpfd_LICENSE environment variable as either a system-wide environment variable or in scripts that launch simulations.
- If you are pointing to an RLMCloud server, set the environment variable to the path of the RLMCloud license file you have received from CPFD. For example:
export cpfd_LICENSE=/path/to/myRLMCloudFile.lic
- If you are pointing to an on-premise RLM server, set the environment variable to the port (27015 by default) and IP address or hostname of your RLM server. For example:
export cpfd_LICENSE=27015@myserver
Running Simulations
The specific workflow for running simulations on your cluster will depend on specifics of how it is configured by your system administrator. However, as a general overview, the most common workflow is:
- Set up Virtual Reactor simulations on a computer with a graphical desktop. This allows you to use the Barracuda GUI to complete the project setup in a convenient manner. Generate the grid and run the project for a single time-step so you can review and verify the configuration of initial conditions, boundary conditions, and other details.
- Once you are ready to start the full simulation, transfer the project setup files to the cluster (often through a network share or via scp or FTP tools).
- Start the simulation either directly using a “cpfd.x” command or by embedding the command in a job scheduling file (if your cluster uses SLURM or a similar job scheduler). See Running from the Command Line for additional details. On clusters it is usually important that the solver NOT prompt the user for interactive input during a simulation run. To ensure that no interactive prompts are presented, the following command-line flags are available to pass to the cpfd.x solver:
[-overWriteFile] [-ow] # Overwrite existing transient data files without prompt [-appendFile] [-aw] # Append to existing transient data files without prompt [-contChem] [-cc] # Continue with simulation even if chemical reactions are not mass balanced [-contThermal] [-ct] # Continue with simulation even if extreme thermal temperature found [-contBC] [-cbc] # Continue with simulation even if boundary condition warnings present [-contIC] [-cic] # Continue with simulation even if initial condition warnings present [-quitMinDiskPercent] [-qmdp] # Stop simulation if minimum disk space limit is reached [-quitLicenseLost] [-qll] # Stop simulation rather than trying to regain a lost license [-quitFE] [-dieFE] [-qfe] # Automatically exits solver if floating exception encountered (FE_INVALID) [-quitOnPrompt] [-qop] # Stop simulation rather than prompting for user input (useful in batch environment) [-gpu] # Use NVIDIA GPU to accelerate simulation [-dauto [method]] # Auto-select a GPU by: (Default: sequential), random, memFrac, memTotal, bandwidth or tlnumber [-device <devID>,...,<devID>] # Use CUDA enumerated devices <devID> for GPU acceleration (specify 'all' for all devices) [-dmulti <numberOfDevices>] # Use <numberOfDevices> GPU devices for GPU acceleration [-fallback [method]] # Action on CUDA error: (Default: auto), wait or quit
The following solver command is a suggested starting point for running simulations on a cluster:
/path/to/CPFD/Barracuda/<version>/bin/cpfd.x -ow -cc -ct -cbc -cic -qmdp -qll -qfe -qop -gpu -d0 -fallback quit <my_project.prj>
Replace the path to your installation, the <version> number, and <my_project.prj> with details appropriate for your system and project.
- While the simulation is running, monitor information from history.log and runtime.log to ensure that no problems have occurred.
- Once the simulation is complete, files can be transferred back to a computer with a graphical desktop, or accessed over the network, if possible, to perform post-processing with Tecplot for Barracuda.