Performance Profiles

Performance Profiles is currently in Public Beta, available for any customer. Please contact your Rescale Representative for access.

Overview

This page is dedicated to instructing you on how to use Performance Profiles. In order to access the Performance Profiles feature, you must explicitly have access from your Organization Administrator. 

Performance Profiles enables you to run multiple jobs (with different hardware) using a singular workload to benchmark which architecture provides the best workload-specific price versus performance.

Without Performance Profiles, you would need to blindly follow your gut instinct on hardware choice or run adhoc benchmarks using spreadsheets. With Performance Profiles, engineering leaders can run benchmarks in a self-service manner, which will be plotted on a graph of cost vs. time. 

Benefits of Using Performance Profiles

With Performance Profiles, you will have the ability to seamlessly run benchmarks, which will afford you the following benefits:

Use Case	With Performance Profiles	Benefit
Optimized Templates	Simply set your Profile Configuration as an existing template and run a variety of CoreTypes and at various numbers of cores	Speed up or reduce cost by applying new templates to your entire organization
Determining what Coretype is the best go-to choice for general workloads	Run multiple Profiles against the same CoreTypes and compare which CoreType comes out ahead; look for infrastructure on the Pareto front whether you’re optimizing for cost or time	Simplify your stack and choose the best CoreType for your specific workloads whether it be for optimizing for time or reducing cost
Finding the cheapest Coretype for a specific workload	Load up a Profile Configuration as a previous or future job and run a variety of CoreTypes and at various numbers of cores; look for infrastructure that leans toward the bottom left	Know which infrastructure will result in the cheapest cost, which will end up saving on your compute bills

Performance Home

From the Performance Home page, you can view, manage, and configure new and existing Profiles.

A Profile represents a specific workload that you are using to benchmark against multiple architectures, thus reducing the number of variables that you are measuring, giving an accurate representation of how a certain architecture performs for your given workload.

Available Performance filters include:

• All Profiles
  • My Profiles & Shared Profiles
• My Profiles
  • Profiles you’ve created
• Shared Profiles
  • Profiles that have been shared with you
• Archived Profiles
  • Profiles that you’ve archived

There are six columns available for profiles:

• Name
  • Given profile name
• Runs
  • Number of infrastructure benchmarks run against this profile
• Last Updated
  • Last time a benchmark was run in the profile
• Description
  • The given description of a Profile
• Profile Configuration
  • The name of the template or job that was used to create the Profile
• Shared
  • Who you’ve shared the profile with or the person who shared the profile with you

Managing Columns and Filters

On the Performance Home page, select the (…) icon to customize your filters and filter column headers.

Setting up a Performance Profile 

To create a new Profile, select New Profile from the top left of the Performance Home page. Upon selecting New Profile, you will be given a blank Performance Profile.

Naming your Performance Profile

First, you’ll want to give this Performance Profile a descriptive name so that you can understand what this Profile is about. In order to change the Name, click Untitled Performance Profile to give it a unique name, and then confirm your selection by pressing the green checkmark.

Selecting your Profile Configuration

Next, you’ll want to select your Profile Configuration which can be done by clicking Select Configuration. This will open a drawer that gives you the ability to select your configuration from

1. Templates
2. Jobs

Please note that you will not be able to change the Profile Configuration once you have run your first benchmark.

Adding your Infrastructure

After selecting your Profile Configuration, you’ll need to add your infrastructure. To do so, click Add Infrastructure or the Add button from the table, which will open a drawer with a hardware selection. 

Select the hardware you want by selecting the checkbox for each item and selecting the number of cores. If you’re not sure what to select, you can compare CoreTypes using the CoreType Explorer that is linked in this drawer.

Once you’ve selected your hardware, click Add.

If you forgot to add hardware or would like to run the same hardware using different numbers of cores, you can click Add to select additional hardware.

Reviewing Selections

You’re almost ready to kick off your Profile! Review your hardware selections, ensuring that it is the correct hardware and number of cores. Once you’re ready, click the Run button, which will kick off all Pending Hardware Benchmark Runs. 

Now, kick back and wait for your benchmarks to finish. When you come back, you’ll be able to view them on the graph. 

Reviewing your Performance Profile

After your benchmark runs have been finalized, you’ll be able to view the results both on the graph and the table.

Performance Profile Table

The table will show you a list of all of your hardware benchmark runs, the current status, run time, and cost. This table will give you a detailed view of your runs that have been completed. 

Note that the run time defaults to execution time only, which means Rescale excludes time spent in preconditions, starting, stopping, etc. The cost reflects both the total hardware and software cost.

If you have any outliers that you’d like to remove from both the graph and the table, click the checkbox next to that hardware benchmark and click Delete.

You can always add additional infrastructure to the Profile by clicking Add and you will be given the chance to run this new benchmark. When you click Run, only the newly added benchmark will run, and not the previous benchmarks.

Performance Profile Graph

The graph will show your plotted benchmarks of Cost vs. Time to Solve.

If you highlight a benchmark, you will be given a view of the time and cost of the benchmark.

When evaluating the benchmarks, we recommend finding balanced hardware which can be found in the bottom left corner of the graph. With more complicated benchmarks and requirements, we recommend always looking at the benchmarks on the Pareto Front to select the most efficient hardware.

Adding a Description

We recommend adding a description so that you can track which hardware performed the best and what the purpose of this Profile is. To do so, click the (+) to show the description.

Modifying Run Time Metric Definition

By default, Run Time is defined as execution time only, which means Rescale excludes time spent in preconditions, starting, stopping, etc.

You have the option to switch the Run Time to be total run time, which is defined as the total time it takes to run a job (including starting, stopping, etc).

If you could like to modify the Run Time Measurement, go to Policies > Platform Settings > Performance >Run Time Measurement.