This section walks you through steps to write a job submission script file, submit a job, check job status and download files using the Rescale CLI App.
The tutorial is based on an LS-DYNA simulation example available from the Rescale Documentation page. You can obtain the LS-DYNA simulation file (
neon.refined.rev01.k) used in this example here.
To submit a job that runs on a particular software (LS-DYNA in this case), you need to specify the software in your
#!/bin/bash #RESCALE_NAME="LS-DYNA Neon Example" #RESCALE_ANALYSIS=ls_dyna #RESCALE_CORE_TYPE=Emerald #RESCALE_CORES=2 #RESCALE_WALLTIME=1 #USE_RESCALE_LICENSE ls-dyna -n $RESCALE_CORES_PER_SLOT -i neon.refined.rev01.k -p single
Note: When copying the above script, ensure that there are no spaces following each environment variable. You can find the flags for the input script in the Rescale CLI Commands section. Also, the command on the last line is the same as you would use on the Rescale platform; you can find the command on the Rescale Web UI or using the
list-info command (please refer to the Rescale CLI Commands).
Assuming you have your Rescale Application program interface (API) is a set of routines, pr... Key environment variable (refer here), run the job using the command:
rescale-cli submit -i submit.sh
The log will look like this on your terminal:
Executing Command. Parsing Input Files No existing files to include Found Analysis: ls_dyna Zipping File Creating temporary encrypted zip at /tmp/rescale-cli-work3842771237402719984/input.zip Finished writing encrypted file Uploading Files Uploading: /tmp/rescale-cli-work3842771237402719984/run.sh Uploading run.sh: ##############################| 240B / 240B Uploading: /tmp/rescale-cli-work3842771237402719984/input.zip Uploading input.zip: ##############################| 368B / 368B Job: Saving Job Job CeMod: Saved Job CeMod: Submitting Job CeMod: --end-to-end flag not set, polling should be done manually.
The App will zip up all files in the current working directory into a temporary file called
input.zip. It will then upload both
input.zip to Rescale and include them as input files to the job. It will also include any files listed in the
#RESCALE_EXISTING_FILES line of the input script. For example, if your directory looks like this:
And you submit your job, the App will temporarily create the zip file, as well as parse the
submit.sh script into
run.sh. Your directory will look like this:
input.zip will include the following files –
neon.refined.rev01.k, run.sh, submit.sh. The App will upload the
input.zip file to Rescale along with the
run.sh script. The App will then remove the zip file from the local file system and attempt to save/submit the job. If the job passes validation it will be saved and submitted to Rescale. The App will then start a polling cycle and wait for the status of the job to reach completed. At this point, the App will also print out a <job-id> which can be used for reference.
To check the status of the job running, type the following command by replacing
rqRXT with your job ID –
rescale-cli status -j CeMod
The terminal will show the status of the message as shown –
rescale-cli status -j CeModAuthenticated as <user>@rescale.comThe status of job CeMod is Executing
To download a file when the job is running, we use the
download-file CLI command as show below. You can download only one file at a time.
rescale-cli download-file -j CeMod -f shared/d3plot
Here, we are downloading the file
d3plot from the run. The command looks for the file in the cluster working directory and hence the file path relative to the working directory must be specified. Here, the file
d3plot is present in the
shared directory. To obtain the relative path of the file, please check the live-tailing window on the Rescale platform as shown.
To download files after a job has completed, we use the
sync CLI command. You can download all the files of the completed job as shown –
rescale-cli sync -j CeModT
The terminal will show the following log –
rescale-cli sync -j CeModAuthenticated as <user>@rescale.comSyncing output files for job CeModJob CeMod: Downloading files to /Users/johndoe/Documents/CLI/LS-Dyna-Example/rescale_job_CeModDownloading /Users/shashank/Documents/CLI/LS-Dyna-Example/rescale_job_CeMod/download_20180703_224544_d3plotDownloading download_20180703_224544_d3plot:Downloading /Users/johndoe/Documents/CLI/LS-Dyna-Example/rescale_job_CeMod/process_output.log....
You can use output filtering using
-s flags to download only specific files.
-f flag uses “glob” searching (On client side, bash only) while
-s uses “string pattern” searching (On server side). Additionally, you can also use
--exclude to exclude certain file from downloading.
*is a wildcard that will match any string of characters and
? is a wildcard that will match any single character.
Example : The following command downloads all files starting with
d3 while excluding
rescale-cli sync -j CeMod -f d3* --exclude d3hsp,d3dump01.0000,d3dump01.0000
For large jobs, the
-s is recommended since it is on the server side.
-s does only string pattern searching and downloads all files with the given string pattern.