1.1X Rescale CLI Commands

This section provides a list of various commands the Rescale CLI App supports. You can also go through the Rescale CLI Tutorial for additional information on using CLI commands. Please note the following:

  • If you set up the API key as an environment variable Simplifying CLI Commands page, you may omit [-p <api-token>] from the below commands.
  • Rescale CLI defaults to the US platform platform.rescale.com. If you are a user on any of the other global platforms (EU, JP, KR, ITAR), please specify your regional platform using the -X flag in your CLI command. For example:
PLATFORMCLI COMMAND
EUrescale-cli -X https://eu.rescale.com submit -p <api-token> -i <input-script>
Japanrescale-cli -X https://platform.rescale.jp submit -p <api-token> -i <input-script>
Korea
rescale-cli -X https://kr.rescale.com submit -p <api-token> -i <input-script>
ITARrescale-cli -X https://itar.rescale.com submit -p <api-token> -i <input-script>
  • If you wish to get help on Rescale CLI Commands, you can use the -h flag. Example – 
    rescale-cli -h

You can obtain information about Rescale Core Types and the Softwares available using the following commands and flags :

COMMANDrescale-cli list-info -<flag-type> [-p <api-token>]
FLAGS
DESCRIPTION
-cRetrieves information about RESCALE core types
-aRetrieves information about softwares available

Start a job

Creates and launches a job based on the provided flags and input script. For more information on contents of the input script, see Rescale CLI Advanced.


COMMAND		rescale-cli submit [-p <api-token>] -i <input-script-file>
FLAGS
DESCRIPTION
1) -q or --quietSuppresses all logging output that would normally go to STDOUT. This is useful when piping the script output to another script.
2) -E or --end-to-endRuns the job end-to-end : uploading input files, submitting the job, monitoring for the results and downloading the results files.
3) -f <file1> <file2> ... <fileN>Filter output files to be downloaded after the job finishes. This flag can be used with submit only while running an end-to-end job.
4) -J-Djava.io.tmpdir=<directory>Specify output directory for temporary files.

Stop a job

Submits a request to cleanly shutdown the currently running job

COMMANDrescale-cli stop [-p <api-token>] -j <job-id>

Delete a job

Submits a request to delete a completed job

COMMANDrescale-cli delete [-p <api-token>] -j <job-id>

Check job status

Requests the specified job’s status. Place in a loop to monitor continuously. It is recommended that an exponential backoff algorithm is used to monitor jobs that take long.


COMMAND		rescale-cli status [-p <api-token>] -j <job-id>

Download files from a completed job

Downloads all the result files after the job run.

COMMANDrescale-cli sync [-p <api-token>] -j <job-id>
FLAGSDESCRIPTION
-s <file1> <file2> ... <fileN>A search term to filter files, e.g, “png”. This does not support globs. For jobs with a large number of files, prefer this to -f/--file-matcher. Multiple terms can by used by joining with ,.

Since it happens on the server side, searching more robust and faster – Recommended.
 
Comma , separator works as an AND operator. So, when using ,, it matches with files with both patterns. For example:
rescale-cli sync -j <jobId> -s cpu,gpu: It returns all files with BOTH patterns in their name. cpu_gpu_3000.txt  but not cpu.txt.
rescale-cli sync -j <jobId> -s 'cpu gpu': It only matches if the file name includes both cpu and gpu. (quotes with spaces are mandatory).
rescale-cli sync -j <jobId> -s cpu -s gpu: The search option is not set up to support multiple inputs this way so only the second search term  -s gpu will be used.
-f <file1> <file2> ... <fileN>Output file filtering using “glob” (Happens on the client side).
*is a wildcard that will match any string of characters and
? is a wildcard that will match any single character
Note: This works best in bash, other shells may not work.
--exclude <file1>,<file2>, ... ,<fileN>Excludes the specified files from downloading.
-n <job-id>To sync all jobs newer than a specific job. App will create
individual directories as – rescale_job_<job-ID>.
-d <sync-time>For continuous syncing, the user may specify a
sync time. The time in seconds would determine
the delay time between two sync attempts.
-o <output directory>Downloads the files to the specified directory.
--max-concurrent <number>Maximum number of concurrent downloads (default: 5)
--verify <true/false>Verify file integrity using checksums (default: true)

Download files from a running job

Downloads a specific file from a running job. It downloads a copy of the file (that exists at that time) to the user’s local directory in which they executed the command. Second, it will append a time stamp to the file name, the filename will be formatted as “download_{timestamp}_{original_filename}”.


COMMAND		rescale-cli download-file [-p <api_key>] -j <job_id> -f <filename>
FLAGSDESCRIPTION
-f <file1> <file2> ... <fileN>Output file filtering using “string pattern matching”
(Happens on the client side)

Upload files to Rescale cloud storage

COMMANDrescale-cli upload -p <api-token> -f <file1> ... <fileN>
FLAGSDESCRIPTION
-eTo see an extended set of metadata during upload.
--quietTo suppress non-json output from the command, if the CLI is being embedded in a larger script.
-r <file>|-Creates a report in JSON format and stores in the specified file or prints it to standard output if - is provided. The report contains the file IDs of the uploaded files and the result of each operation.

Upload files to Cloud File System (CFS)

Note:

  1. Cloud File System (CFS) is a beta feature. Contact your account manager for more details.
  2. CLI version 1.1.271-65ac2b475f or later is required for this feature.
  3. The file is first uploaded to the Rescale Files Library folder and then copied to the user’s library folder in CFS.
  4. Only one file may be uploaded to CFS using this command at a time.
  5. Files with the same name will be overwritten in successive CFS uploads. However, each file copy will be maintained in Rescale Files.
  6. -r, -d, -T flags are not compatible with --copy-to-cfs
COMMANDrescale-cli upload -p <api-token> -f <file1> --copy-to-cfs
FLAGSDESCRIPTION
-eTo see an extended set of metadata during upload.
--quietTo suppress non-json output from the command, if the CLI is being embedded in a larger script.
×