Bring Your Own Software With Rescale Software Publisher – Getting Started Guide

Overview

Rescale Software Publisher (“Software Publisher”, or “Publisher”) is an optional product feature that allows organizations to bring their own software to the Rescale cloud platform.  It provides customers the following benefits:

• On-demand publishing to Rescale’s software catalog- fast without waiting for requests to be processed manually
• You can now bring a wide variety of software to use in Rescale- custom, proprietary, commercial, open source, and local tools and utilities you may have developed that are used in conjunction with simulationSimulation is experimentation, testing scenarios, and making... More workflows.
• Easy to use and does not require advanced cloud administrator access or IT networking or infrastructure skills. Rescale handles all the internal storage, access, security, and specific software environment dependencies when customers self-publish software to the platform.
• Integrated into the corean individual processing unit within a multicore processor o... More Rescale platform, providing a common look and feel without having to context switch between other disjointed tools or interfaces.
• Centralized visibility, access, and administration for all your HPCThe use of parallel processing for running advanced applicat... More software applications and packages

The following features and benefits are highlighted for Rescale Software Publisher:

• Rescale Software Publisher can be licensed for use at the workspaceWorkspaces allow rescale customers to create dedicated teams... More level, or at the organization level which provides visibility and access across all workspace accounts within an organization.
• Customers can extend their existing Rescale software catalog to publish and manage private software accessible to users within your organization.
• Self-published software for batch and workstationA workstation is a powerful computer system designed for pro... More jobs appears and behaves just like other available software packages and applications in the Rescale software catalog.
• You can control visibility of self-published software versions by workspace and remove old software versions no longer in use.
• The Sandbox is a new concept introduced by Rescale Software Publisher, a temporary working environment to install and test software prior to publishing. You can select sandbox properties such as which coretypePre-configured and optimized architectures for different HPC... More to use, wall time, size of sandbox, batch or workstation software, MPI options, and optional thumbnail images for display in the Rescale software catalog.
• Self-published software is portable for use across different coretypes and even cloud regions for use.
• There are also new visibility settings when publishing software, that are described later in this document.  Basically when you are ready to share your published software with others, you can share it at the workspace level, or at the organization level which would be visible to all workspaces.
• New purchases of Rescale Software Publisher come bundled with a package of expert HPC consulting hours, to assist new users through the publishing process and troubleshooting any challenges.

The diagram below shows an overview of the publishing process:

Pre-requisites:

• A user will need admin role privileges to access Rescale Software Publisher – either at the Workspace level or Organizational level.
  • Organizational admins can publish and set visibility within any workspace that has Publisher enabled
  • Workspace admins can only publish and set visibility to workspaces they manage, for workspaces that are enabled to use Publisher.
• You will need someone from Rescale to enable your company or organization to have access to Rescale Software Publisher.   Reach out to your Rescale team for details.
• At this time, only software that can run on Linux OS can be published. Windows OS is not supported for self-publishing.

UI access:

As an admin user, go to the admin portal and you will see a “Tools” menu at the top.  Under the Tools menu, select “Software Publisher”.

If you do not see this new Tools menu from the admin portal, your Rescale user account may not have the necessary permissions configured.

The first time Publisher is accessed, you will see an empty list. Below is a sample list of populated self-published software with a variety of attributes, such as batch vs workstation (interactive), number of past published versions, and latest version. There is a search field for finding software by name, and each column displayed can be sorted. There is a second tab named “Sandboxes”, which will display active sandboxes for the workspace.

New Software Tile

To publish new software, you first need to register your software application or package. This is referred to as a software tile. Once your software tile is created, you can then publish as many versions as needed. You do not need to upload any software when creating a new tile.

Select the “Add Software” button above the list table.  You will see this page to create your new software tile:

Complete the required fields for the new software name and description.  Select “Batch Jobs” or “Workstations” for the software you will publish, if your intended use of the published software is via unattended batch jobsBatch jobs are automated tasks submitted to a computing syst... More or via interactive Workstations.  The thumbnail image is optional to provide- the size must be under 100K. The industry category tags are optional and not required. These industry tags control what users see when they filter software by different industry categories in the UI.

You also have the option for publishing software that has license dependencies. At the bottom of the software tile page, you can select licensingLicensing is a legal tool granting users the rights to use, ... More properties required by your software to run. This will allow your self-published software to use one of Rescale’s licensing services via a Rescale License Host or Company License Proxy (CLP):

Finally click the Save button when you are ready to create the new software tile.

Here is an example from the UI after creating the initial software tile, before publishing any versions:

New Software Version

Once you add your software tile, then you can publish one or more versions.  Select the software tile you created from the list, and then under the Software Versions section, select the button “Add Version”. With Rescale Software Publisher, new software versions are created in temporary isolated work areas called Sandboxes.

A new window will appear with three selection options on the left for “Version Properties” , “Sandbox Setup”, and “Hardware Settings”.   Beginning with the Version Properties page, fill in the required fields. 

• Version – The software version field can be any alpha-numeric value, combined with limited special characters
• Command – The command field is what will be displayed to users in the software catalog, when they select this published version for creating and submitting new jobs.
• Mount Base – This field will be the path where your software is located or mounted on the job clusterA computing cluster consists of a set of loosely or tightly ... More when running. Any self-published software will begin with a top level directory named “/software/”. Typically you will specify the software version under the top level directory when providing a new value for Mount Base.
• Environment Variables – Optionally you can provide environment variables that your published software can reference when running within a job.

You cannot duplicate software version values for a software tile, or create duplicate mount base paths.

Now navigate to the “Sandbox Setup” page next.  You can upload your software from this page, specify the operating system, and indicate if your software has any MPI dependencies or not.  The project field is optional, and only required if the current workspace you are publishing from is configured to require selecting a project. For the sandbox size, try to pick a size that is large enough to install and test your software and any test files, but not too large as to waste unused space.

There is an optional command field on this Sandbox Setup page that is different from the command on the version properties page. This sandbox command field is only used if you want to run an automated command in the sandbox environment after it is first created. You will be able to later log into the sandbox environment directly after it is ready for access, to run any commands or perform operations as you build, test, and validate your software prior to publishing.

Currently, only Linux OS distributions are supported for creating the sandbox environment.

Then navigate to the “Hardware Settings” page. Here you can select from a variety of processor core types that are available, and also set the number of cores you want your sandbox to run.

Create

When you complete filling out the sandbox information on all pages, click the “create” button. You have the option to review any setup page before creating the new sandbox. You will see a new “status” entry appear on the left side.  If you select it you can monitor details of the sandbox creation progress:

Active sandboxes will also have an Administration window available. This will be explained in the next section.

Active

When sandbox setup is complete,  you will see the following displayed in the UI:

• Sandbox status – the current sandbox state is displayed at the top- “Active”
• Visibility – Every new software publish session starts off with a visibility value “draft” while the sandbox is active. After publishing a new software version, there are additional visibility settings an administrator can set, which are documented later.
• Connect button– this is one way to access the sandbox environment via remote desktop, described below
• Suspend button– you can suspend your sandbox if you need to pause your activity, then resume later
• Copy SSH command button– this is another way to access the sandbox environment, described below
• Actions – within the SSH section, these buttons provide in-browser SSH access to the sandbox environment
• Cancel Sandbox button– you can cancel your publishing session at any time prior to publishing. Any uploaded files or software to the sandbox will not be saved.
• Publish button– this is when you have completed all tasks to upload, build, test, and validate your software. Publishing a new software version will remove all sandbox contents and shut down the sandbox environment.

There are different ways you can connect directly to your sandbox after it is created:

• DCV desktop emulation by selecting the “Connect” button
• In-browser SSH connection by clicking under the “Actions” section
• SSH connection from your local workstation. This is a more advanced option that is covered in the product documentation under “SSH From Local Workstation“.

There is an administration page available for active sandboxes that can be accessed on the left side when viewing software version details within Publisher. The following functionality is available on the Administration page:

• Upload additional files to the sandbox- by default these are copied to $HOME/work
• Extend the sandbox walltime
• Share your sandbox with another user, or customer support

Any software you wish to publish must be located under the Mount Base path you provided when first creating the new sandbox. Any other software or files that you upload or copy outside the Mount Base path will not be preserved when publishing the new version.

Connecting To The Sandbox

In this example we will choose the “Connect” button to proceed. The virtual desktop environment is using Rescale Workstations underneath.   To access a terminal window for shell / CLI options, select the terminal emulator button on the bottom:

If you chose to upload software when creating a new sandbox, the software will be placed in the local user’s $HOME/work directory.   There is an option in the UI when uploading files to select automatically uncompressing and extracting TAR files, and the contents will be automatically copied to your software Mount Base directory in the sandbox.

In this remote virtual desktop interactive mode, you can log in to your sandbox and perform commands via a SSH or terminal session.

You can run git() directly from your sandbox to retrieve software or other material, such as test data.  Here is an example retrieving Tutorial data for a HPC software package (SU2):

You also have other options to move or transfer software to your Rescale sandbox environment, such as sftp().

Update

You can update an existing Publisher sandbox, including uploading additional files, extending walltime, and you can even cancel an active sandbox if you choose on the sandbox administration page. Refer to the previous section covering the Administration page of the sandbox.

Sandbox Environment

There is a default sandbox environment variable available for use.  The mount base you specify in your sandbox configuration, is available as an environment variable named “RESCALE_MOUNT_BASE”.   Recall this is the top level directory where your published software will exist when selected to run batch jobs or workstations.  You can access this environment variable in any local scripts or tools you use within the sandbox. Here is an example showing access to the RESCALE_MOUNT_BASE environment variable when first logging in to an active sandbox environment:

Another environment feature to highlight for sandboxes is use of the rescale_env.sh file. This file is created and updated automatically by Publisher and the spub helper utilities, which are explained in more detail in the next section.  This file is located under the mount base directory in a sub-directory named “rescale/” : 

You can think of the rescale_env.sh file as a secondary Bash file.  The contents of this file will get automatically sourced at job launch time when your published software version is used.  This file will get automatically updated by some of the spub tools such as spub_validate, but you can also update it directly yourself if needed.

The rescale_env.sh file in a new sandbox instanceAn Amazon Virtual Machine More starts out empty.  If during the process of building, testing, and running your software you make environment changes to the PATH, or add new environment variables, running the spub_validate command will pick up those local changes and write them to the rescale_env.sh file, which will be run later when the published software version is selected for a new job to run.

Here is an example with a new sandbox instance, showing the contents of the rescale_env.sh file start as empty.  Then a change is made to modify the $PATH, and a new environment variable is created.  The spub_validate command is run after these updates, and you can see before and after what is written to the rescale_env.sh file:

You will notice that while the PATH value was adjusted to include the extra directory for /tmp/tstdir, spub_validate preserves the PATH evaluation order so that the software under $RESCALE_MOUNT_BASE/bin is first.

You can also edit the rescale_env.sh file directly to include environment variables, which will be made available to jobs that run the software version after publishing.   Here is an example of adding a new environment variable “LBIRD” to the rescale_env.sh file under $RESCALE_MOUNT_BASE/rescale/ : 

To summarize above, the environment variable and value  LBIRD=33 was added to the rescale_env.sh file, and then spub_validate was run before publishing the software version.

Here is the test command that will run in a job after the software version is published:

After this test software version was published, it was selected for a new batch job to run:

Here is the output result testing the environment variable being set in the job:

This example shows that sandbox level environment variables that are set or recorded in rescale_env.sh are persisted after publishing via the software version metadataMetadata is data that provides information about other data,... More captured, and can be accessed within a job.

Spub Helper Tools

The Publisher sandbox environment includes a custom set of tools and utilities to assist you with self-publishing, referred to as “spub” tools. They are located under /program/sxp-interactive/util . More details about the different spub tools are covered in a later section. To get an overview of these tools, run the spub_help command, and the spub_helper_list command.

You should always run the spub_validate command from your sandbox environment prior to publishing any software.

When you log in to a sandbox instance, you do not have full root or sudo access to the working environment. You will have full read and write access under the Mount Base path (/software/*), but you will not be able to update or write files or content directly into the operating system location or other restricted system level areas. The spub helper tools are one way to assist you with installing dependent software packages, utilities, compilers, libraries, etc. in a way that you do not need root or sudo access.

Publish

When you are ready to publish, you make sure all of the necessary software files and binaries are located under the mount point base before publishing, and clean up to remove any temporary files not needed to run the software. This will help to reduce the total storage size of your software version. Be sure you have performed needed software testing and validationValidation is the act of ensuring that a product, system, or... More with the software located under the designated Mount Base directory. Run the spub_validate command as a last step prior to publishing.

Here is an example testing software within a sandbox prior to publishing:

From the UI, select the “Publish” button at the bottom of the status page.  A confirmation prompt will be displayed that you must first review and accept. Then you will see the status value at the top of the page first change to “Publish Queued” , then a sequence of states including “Publishing”, and then a final state of “Published”.

You can also view the status of all active sandboxes in a workspace, from the “Sandboxes” page on the main Publisher home page:

When your software version publish action is completed,  you will see the version status updated to “Published”. The sandbox is cleaned up after the software version is successfully published, and all sandbox connectivity is then disabled. You will also notice after a software version is published, the visibility is automatically set to “Admin Private”, and the administration page functions to operate on the sandbox are now disabled since the temporary sandbox no longer exists after successful publishing. There is also a new page that displays within the UI for the software version after it is published, “Allowed Workspaces”, which will be described shortly.

You can now access your newly published software version to test and validate by creating a new job or workstation within your local workspace.

Visibility

There is a “Visibility” menu when viewing the details of a published version.   There are multiple visibility settings that administrators can set to promote the software version across stages after it is initially published.  They are as follows:

From the UI, here is how you set the visibility of a published software from its initial state, which by default always starts with “Admin Private”:

Admin Private

When your software is published to the initial state, Admin Private, you as an administrator, and other administrators with the proper access in the workspace published, can access and use this new software tile and version from the Rescale software catalog when creating jobs.   The visibility at this stage is limited to just the admins:

Here is an example of how the new published software version will appear in the Rescale Software Catalog for the workspace(s) the software is visible. It will display and behave just like any other software version that already exists in the software catalog:

After you test and validate this software version is working ok, you can now share it with others in your organization.

There are two visibility options when you want to share your newly published version with others:

• Publish visibility of the version to the workspace  (workspace private)
• Publish visibility across all workspaces in the organization (organization private)
  • Note this option is only available when Publisher is licensed at the organization level

Workspace Private

Depending on how your organization has licensed Rescale Software Publisher, and your role administrator access, there is an option to share published software versions between workspaces. From the software tile page, select the version number you just published.  At the top of the page next to the “Published” status, select the menu and pick “workspace private” to promote the visibility of your published software version to a named list of workspaces.  Then on the left side, select “Allowed Workspaces” and pick the workspaces you wish to share this software version with:

You can only select workspaces that you as an Administrator have access to.

Click on the button “Update Allowed Workspaces” to complete the selection process.

Organization Private

You would follow similar steps as previously described in changing the visibility of a published software version to “organization private”.  Note that you need to be in the role of an organizational administrator for this option to be selectable in the UI, and you must have Publisher licensed at the organization level.  If your user is just granted admin access to a single workspace, you cannot set visibility of published software to organization private.

Delete

You can delete individual versions of software you publish, and you can also delete an entire software tile or package and all of its versions from the UI. To delete individual software versions that are no longer in use, you can select the trash can icon in the version list:

Preserving Sandboxes

By default, sandboxes are removed after a software version is published, and the contents of a published software version are immutable. You have the option to preserve a sandbox prior to publishing a new version, so that the sandbox will persist after the version is published. This feature is useful for administrators who want the ability to make iterative updates and corrections for a recently published software version, without creating new versions. Choosing to preserve a sandbox will keep the instance active and keep any software you have have uploaded and built.

To enable preserving a sandbox, navigate to the administration section and select the button under the section “Preserve Sandbox” :

When a sandbox is enabled to be preserved, you will see the status value of a sandbox of a published version appended with a “(Published)” string, for both active and suspended sandboxes:

When a sandbox is enabled to be preserved, you must manually remove it later once it is no longer needed. Care must be taken to not let preserved sandboxes idle or be suspended for long periods of time, as they will incur costs until they are deleted. It is suggested that the walltime of a preserved sandbox be strictly set to minimize the risk of unplanned costs if an administrator forgets about a sandbox that is preserved. The option to preserve a sandbox is not available after a software version is published.

Software Portability and Helper Functions

When bringing your own software to the Rescale platform, a best practice is to make your software application, package, or tool as portable as possible.  This is because the Rescale platform is architected so that the software which runs on our managed compute clusters is de-coupled from the underlying hardware and operating system.  This allows customers maximum flexibility when choosing core hardware options and target environments to run their high performance compute workloads.    If your HPC software becomes too intertwined with the underlying operating system, or has other hard-coded dependencies, over time your software solution will become more brittle, and simple updates like an OS patch may require you to re-install and test your software each time such changes occur.

This is why Rescale sandboxes by design do not allow super user or administrator privileges such as the root user account or sudo access, as well as for security reasons.  However it is still possible you can install, run, and test your software within the Rescale controlled hosted environment successfully without requiring such elevated access.   With every sandbox that is created, Rescale provides a set of utilities that help you with installing and testing your software so it is portable and flexible.    These helper functions address many common install commands and packages that typically require root or sudo access, from yum install to python pip .  Below is an overview of the Rescale helper functions and utilities that you can use to ensure your published software is portable and follows recommended best practices.

Using Spub Helper Utilities

At a high level, the Operating System (OS) image in the sandbox is representative of clusters that will run the software that is installed.   However it is important to note that the Rescale design for publishing and running software on the platform decouples the software you publish from the underlying OS.   The published software and OS are not stored together as one volume or snapshot.   This design is to maximize the portability of published software in the Rescale Catalog, so software that is published can run on a variety of hardware platforms and environments.

Many users that are accustomed to installing, configuring, and running HPC software packages, applications, and other supporting tools in a dedicated local environment where they have root or admin privileges within the operating system to directly add and modify files in any location, will need to adjust their approach when making software available on the Rescale platform for use.  The main paradigm shift when publishing software in the Rescale platform, is you need to package your software so that any dependencies are self-contained within a designated directory area, which is referred to as the mount point or mount base of your software.  Because software that is published in the Rescale Catalog needs to be portable – it can be selected to be run with jobs across different hardware core types and even different CSPs- as a best practice you should not install or modify any files directly in the OS area.   There are some common software dependencies that Rescale provides automatically such as compilers and MPI libraries which are made available in consistent locations under the mount base directory, to ensure your published software remains portable.  In addition, these following Spub helper tools are also provided to users who are publishing software on the Rescale platform, which are documented in more detail later.

When working in a virtual sandbox to upload, install, configure, and validate software,  the entire VM will not be saved upon publishing.  Instead, a snapshot of the mount base volume and relevant environment settings are captured. This has implications regarding the portability of the installation and must be handled with care. In general the advice is to ensure that all requisite binaries, scripts, libraries, etc. are all located under the mount base you specify for your software.

As an example, if the path “/software/my-great-software” as the mountbase, this may look something like this:

• /software/my-great-software
• /software/my-great-software/bin
• /software/my-great-software/scripts
• /software/my-great-software/lib

To aid ensuring a portable installation and ease some of the burden on the user, Rescale provides a number of helper functions that are pre-configured and set up by default to point installations to the mount base selected in the UI.  These functions are prefixed with spub_* and can be listed and described from a terminal on the cluster with the command spub_helper_list.  The spub_help function will always have the most up to date instruction set as new helpers are added.   
The location of these spub helper functions are installed in a default location for every Publisher sandbox created.   This directory is /program/sxp-interactive/util , referred to as SPUBUTILS for reference in this document.  Here is a directory listing of the contents:

An overview of the available spub helper functions is provided if you view the following file: 

> $SPUBUTILS/spub_helper_list | more

When running a helper function interactively there will often be actions that need to be performed by the installer, such as adding PATHs to the current shell or setting an environment variable. These will be brought to the attention of the user via messages in the shell. e.g. 

*****************************************************************

*********************** IMPORTANT **************************

*****************************************************************

Please run the following command in your shell before proceeding:

    source /program/my-great-software/venv/bin/activate

There are several use cases to highlight where it is necessary to use the built in helpers. 

yum: spub_yum_install 

Typically when using third party package management systems, e.g. yum, dnf, it is designed to install at the system level. This requires both admin rights on the cluster and places the installed libraries at locations that will not carry over to the published analysis. A wrapper that mimics yum install is provided that downloads rpms and installs them under the mountbase.  Rather than running sudo yum install cowsay with the helper the command becomes spub_yum_install cowsay 

You may get a message after running spub_yum_install to export environment variables in your shell.  Follow the instructions as provided:

python: spub_add_python_venv

It is good practice in general to create a virtual environment when running python code that requires a number of external packages (e.g. numpy, pandas, …).  To accomplish this in a Rescale publishing sandbox, the utility spub_add_python_venv should be used.  This will set up a python virtual environment with a choice of version and make sure that any packages installed via pip will be available when the sandbox is published. 

NOTE: always run the source command provided at the end after spub_add_python_venv completes:

>source /{YOUR_MOUNT_BASE}/venv/bin/activate

Note if for any reason you encounter issues with running this helper function, you can back it out by running spub_remove_venv and start over.  You only need to run this command once within a sandbox.

There is a known Python limitation that spub_add_python_venv does not support installing non-standard packages that are manually added to the repo.  The workaround for such Python packages is to build them from the source code in the sandbox.

gcc: spub_update_gcc

The default version of gcc currently provided for Publisher sandbox machine images is quite old.  Use the spub utility spub_update_gcc to upgrade to a newer version.  Currently only versions 7 or 8 are supported with this spub helper function.  If you need a newer version, you have to manually install it within your sandbox environment.

all: spub_validate

It is recommended to run some small testcase while the sandbox is still running to ensure that the software is configured correctly.  If the binaries run and all libraries are found then it should be ready for publishing.  When you are ready for final validation,  spub_validate is provided to help check if there are any environment variables or PATHs that reference the home directory which will not carry over to future Rescale clusters.  These may end up being innocuous, but on the other hand may need to be remedied by an action such as moving the location of a script/library etc. to the mount base. e.g.

[uprod_eqrCUc@ip-10-25-10-6 ~]$ spub_validate 

Extracting and saving current environment settings

::: WARN: You have an ENV VAR defined that references the ephemeral $HOME in this sandbox: 

          Name:  BAD_VAR

          Value: /enc/uprod_eqrCUc/foo

After checking the environment, this sandbox can be published and this environment will be recreated on future Rescale jobs.   The output of running spub_validate is written to the following directory under your designated software install point:  $MOUNT_BASE/rescale .   There are three files created:

• command_history.txt –  captures all the commands run within your shell
• draft_env.sh – intermediate file if session exits prematurely, resumed at next login
• rescale_env.sh – only incrementally added or modified ENVs appear in this file.  Default ENVs that Rescale generates are not included in this file.

Here is an example output of a successful spub_validate run :

In this example, the instructions are to add a new environment variable name and value to your software version you are testing and validating, directly from the Publisher UI:

There are an assortment of other helper functions that are essentially package installers designed to build things in a way that runs optimally on Rescale clusters. These follow a naming convention of spub_add_<pkg>, e,g. spub_add_R.  There will be a default version stated that can be overridden by user input.  In addition to building the packages they will provide instructions on how to ensure that actions such as installing additional R libraries will be saved correctly and available in the published analysis. 

Common Questions

• What if I need a specific version of Python installed that is not the default version in the sandbox?

Using the spub_add_python_venv helper function, you can specify the version of Python you want installed in your sandbox environment.  It will be the first prompt you see:

Upon success you will see a message saying the Python version was installed under your mount base directory:

Installed Python-3.8.16 to /{YOUR_MOUNT_BASE}/pyenv/versions/3.8.16

IMPORTANT: follow the instructions provided at the end of running this helper function, to source this file:

>source /{YOUR_MOUNT_BASE}/venv/bin/activate

• How would I perform a Python pip install without requiring direct root or sudo user access within the sandbox?

First follow the above example on how to create your Python virtual environment within the Publisher sandbox using spub_add_python_venv .  Then be sure to run the source command afterwards:  source /{YOUR_MOUNT_BASE}/venv/bin/activate  .  After you complete these steps, you can verify that the python binaries and tools you are referencing from your PATH now are in this location: /{YOUR_MOUNT_BASE}/venv/bin .

bash-4.2$ source /program/SWBASEDIR/venv/bin/activate

(venv) bash-4.2$ which pip

/program/acecast2/venv/bin/pip

(venv) bash-4.2$ which python

/program/SWBASEDIR/venv/bin/python

(venv) bash-4.2$ 

Don’t forget to also run spub_validate .

Now you should be able to run a pip install command like:

pip install <pkg-name>

• How do I get a newer version of gcc?

Refer to the helper utility above, spub_update_gcc

• While using the spub* helper functions, I got an error message “no space left on device”.  How do I recover?

Running a python pip install can result in reaching the sandbox disk limit size if you created the sandbox with a size of 1GB.  There is no way currently to dynamically increase the disk size of a sandbox.    The workaround is you will need to cancel this publishing session, and create a new software version with a larger sandbox disk size.

You can run the “df -H” command to view available space under your MOUNT BASE directory.

Advanced Use Cases

MPI – 

If MPI is needed for the desired analysis, there are further considerations that need to be made. There are several pre-configured MPI flavors that can be selected through the UI for the build sandbox. Rescale will generally configure the latest or very recent versions. Ideally these will be suitable for most applications, and keep in mind many modern MPIs are backwards compatible. Additionally there is ABI compatibility across different flavors, Rescale offers Intel MPI which can be used with code compiled with an ABI compatible MPI (e.g. MPICH). It is possible to install an MPI from source or through a package management system such as Spack, however, this is not recommended. 

When an MPI version of the sandbox is selected there will be some Rescale provided MPI wrapper scripts that are added to help aid in keeping the installation as portable as possible. The command mpirun is aliased and will by default add the necessary libraries and flags depending on the type of hardware the software is run on in the future. It is recommended to keep this abstraction in place, which should happen unless PATHs are explicitly set pointing to a different mpirun than the default. This also helps maintain a canonical behavior across software published by Rescale HPC and self published analysis.  

If compiling a code from source there will likely be a need to point to a config to the appropriate MPI location. Regardless of the MPI version, the location will be stored in the environment variable MPI_ROOT. A configure step in the traditional make paradigm might look like:  

      $./configure --with-mpi=$MPI_ROOT --prefix=/program/my-great-install … 

Intel Compilers – 

Rescale provides the Intel oneAPI toolkit as part of the sandbox OS under /opt/rescale/oneapi.  You can select this as an option when publishing a new version of software with Publisher – on the sandbox version page, select “Intel MPI 2019” as the MPI version.   To have the Intel compiler components available within your PATH, you need to source the following file within your sandbox:

• source /opt/rescale/oneapi/setvars.sh

Alternatively, you can use absolute path names to the compiler artifacts.

Known Limitations

• Publishing software that runs on Windows OS is not supported
• Large sized software images will take longer when publishing new versions
• The first time a new software version is used to run a simulation job, there may be a caching delay which will result in the job starting slow.
• Command line interface (CLI)A Command Line Interface is a user interface for computers w... More and direct API access is not supported at this time