Stopping a Job

Last updated: September 7, 2023

Overview

This page walks you through how to stop or force stop a job from the Job Status page using the Force Stop button. Note that the Force Stop button is only visible when you click on the Stop button.

Ways to stop a job

There are two ways to stop a job: you can either stop a job normally or force stop the job

Stopping a job

If you want to stop a running job, click the Stop button.

Figure 1: Stop button

You will then be asked to confirm that you want to stop the job.

Figure 2: Stop warning message

Once you initiate the stop, all the job files are saved to your Rescale Cloud Storage, which will then be shown on the Results tab. 

Clicking the Stop button initiates a few actions in the background, in the order listed below:

  1. The application prepares for shut down. An application could be any of the software that you selected under the Software Settings page, as well as your own software using Bring your Own Software. Some applications and jobs write restart files to allow for the simulation to be restarted from the point where it left off. These restart files in some cases can be large files, which could take significant time to be written out and additional time to upload back to the Job Results page. This also means that the license that is checked out for the job will be in use until the shutdown process is completed.
  2. Once the application is shut down, remaining processes are then terminated.
  3. Files are saved to Rescale, which is when you will start seeing files on the Results tab.
  4. The job cluster is terminated.

Each of the steps listed above has to be performed sequentially and therefore add up to the amount of time it takes to terminate a running job. This process is called Graceful Shutdown.

Force stopping a job

In some instances, however, you may want to terminate the job without all of the steps mentioned above. An example is when you don’t need the restart files written out. Or, when you have realized after job submission that the job was set up incorrectly and you don’t need to continue the run or need to keep any of the files associated with the job. 

In these cases, you would use the Force Stop button, which allows you to skip parts of the Graceful Shutdown job termination process.

To force stop a job:

  1. Click the Stop button from the Status tab to initiate a job shutdown.
Figure 3: Stopping from Status tab
  1. Click Stop again on the confirmation pop-up message. If you want to terminate the job the normal way, you can skip the next steps and wait for the job to shut down.
Figure 4: Stop warning message
  1. If you want to skip some of the steps and shut down the cluster, click Force Stop, which is next to the Stop button.
Figure 5: Force Stop button
  1. On the pop-up window, select Save all files and upload to the Job Results page or Terminate the cluster without saving. The Save option will save all the files and upload them to the Job Results page, whereas the Terminate option will terminate the cluster without saving.
workflow
Figure 6: Save all files or terminate the cluster options
  1. When the process is completed, you will see a message indicating Stopping job (User terminated)… in the Job Logs section. Stopping the job can take a couple of minutes, but after that your job will be complete. Note that with a force stop, if “Terminate the cluster without saving” is selected, your files are not saved, so you won’t see anything in the Results tab.
workflow
Figure 7: Job termination status