Particleworks/Granuleworks Bubble Generation by Solid-Fluid Collision

Overview

This tutorial will introduce you to a simulation of bubble generation by solid-fluid collision in Particleworks on Rescale. You will be shown step-by-step how to set up and submit a job from scratch, if you follow the directions contained in this document. Alternatively, you can click the Import Job Setup button to clone the job that we have already run for you. You are free to modify any of the settings in the cloned job to become familiar with the Rescale platform.

Import Job Setup
Get Job Results
Figure 1: Image courtesy of Particleworks

Description of the simulation

  • Duration: Approximately 10 minutes for setup, and 2 minutes of run time. Alternatively, you can also click the Get Job Results link above, and review the full setup and results. 
  • The simulation: Modeling of bubble generation by solid-fluid collision
  • Simulation software: Particleworks

Simulation setup

Starting up Rescale

To start using Rescale, go to the Rescale platform and log in using your account information. Using Rescale requires no download or implementation of additional software. Rescale is browser-based, which allows you to securely access your analyses from your workstation or from your home computer.

From the main screen of the platform, click the Create new job button at the top left corner of your screen. This will take you to the first of five setup pages.

Figure 2: Create a new job

Input files

First, you need to give the job a name. Because Rescale’s platform saves all your jobs, we recommend you name it something specific so that you can find the job again later. To change the name of your project, click on the pencil (shown in blue, below) next to the current job name in the top left corner of the window.

Next, upload the input file by clicking the Upload from this computer button highlighted in the red box. Alternatively, you can use any file that has been uploaded previously to the cloud storage by clicking the Add from cloud storage button.

For this tutorial, use the bubble.zip file that you downloaded at the start of the tutorial. Note that the default project exporting file format in Particleworks is .pwzip, which is not currently supported by the Rescale platform for auto decompression. Always export the project file to *.zip, or alternatively rename it to *.zip after exporting. On completion, your Input Files setup page should look like that shown below:

Figure 3: Input set up

Software settings

Now, you need to select the software module for the analysis. For this demo, scroll down to find or search for Particleworks/Granuleworks. 

After selecting the software, the Analysis Options will become available.

Select version 7.1.0 from the version selection drop-down menu.

Next, you need to add the analysis execution command for your project. This is a command that is specific to the software package and input file being used. For this input file and Particleworks, the execution command will be: 



OMP_NUM_THREADS=$((RESCALE_CORES_PER_SLOT/RESCALE_SOCKETS_PER_SLOT));
mpirun -machinefile $MACHINEFILE app.solver.double -p ./scene -f bubble/scene/result/sub.json -n $OMP_NUM_THREADS

The simulation is executed in parallel using CPUs by default. Alternatively, it can also be run with GPU (with GPU-enabled hardware), in which case the execution command will be:



OMP_NUM_THREADS=$((RESCALE_CORES_PER_SLOT/RESCALE_SOCKETS_PER_SLOT));
mpirun -machinefile $MACHINEFILE app.solver.double -p ./scene -f bubble/scene/result/sub.json -n $OMP_NUM_THREADS -k cuda -g 0

Command explanation

-mpirun: runs mpi

-machinefile $MACHINEFILE: loads the machine information registered in the cluster

-app.solver.double: Particleworks solver with double precision (app.solver.float is used for single precision simulation)

-p bubble/scene: working directory. For this demo, the operation is performed in the ‘scene’ folder, a subfolder of bubble, which is the decompressed current location.

-n $OMP_NUM_THREADS: The number of threads to use per node.

-f bubble/scene/result/sub.json: Specifies the path to sub.json. 

-k cuda: execute with GPU

-g 0: GPU_ID

The completed Software Settings should look like those in the image below:

Figure 4: Software settings

Hardware settings

The next step is to select the desired computing hardware for the job. Click the Hardware Settings icon to do this.

On this page, you must select your desired core type and how many cores you need. For this demo, pick the Emerald core type and for Number of Cores, choose 36 cores.

Figure 5: Hardware settings

Alternatively, run the simulation with GPU with the following hardware setting. In this case you must specify in the command input to use GPU for calculation (see command line section).

Figure 6: Alternative hardware setting

Review

For this tutorial we omit the optional post processing step. The Review page provides a summary of the job you have set up. It should look like the example shown below. At this stage, you can save and submit the job.

Figure 7: Review job

Status

Now, you can monitor the progress of your job from the Status tab. The entire process for submitting this job should take about 2 minutes. Once the job is submitted, it will typically take anywhere from three to seven minutes for the cluster to boot up.

Because this analysis is entirely run in the cloud, feel free to close your browser window or shut down your computer. You can check on the progress at any time by logging into Rescale and clicking on the Jobs tab. You will receive an email notifying you when the job is completed.

Rescale’s Live Tailing feature can also be seen below. In this particular example, the file process_output.log can be used to monitor the job’s progress.

Figure 8: Monitor status

Results

The Results tab shows all the resulting files that are associated with your job. On this page you can:

  • Click Download to download all the files associated with this job.
Figure 9: Download files
  • View individual files, below a certain size, in your browser by clicking on the View icon in the Actions menu. An example is shown below:
Figure 10: View files

Finally, you can download the results for further analysis.