Rescale Pioneers a New Approach to High-Throughput Computing

High-throughput computing (HTC) is emerging as a powerful capability for engineering and research across many industries. From drug discovery and computational chemistry to electronic design automation (EDA) and environmental research, HTC is helping scientific and engineering teams perform massively parallel processing to quickly analyze vast quantities of information for solving some of the world’s most complex scientific and engineering challenges.

Now thanks to the technical innovation from Rescale, organizations can more easily and affordably harness the power and scale of the latest computing technologies in the cloud to accelerate their innovation efforts. Companies such as semiconductor industry leader Arm are powering their R&D programs with Rescale’s multi-cloud automation, unified data management, and full-stack security. Critically, HTC is supporting EDA processes for Arm to help them create next-generation computer chips faster and more efficiently.

“Rescale is helping Arm usher in a new era for chip design,” says Mark Galbraith, Arm’s vice president of productivity engineering. “Arm-powered cloud computing combined with the intelligent automation of the Rescale platform brings many benefits to our design and verification processes by not only helping Arm engineers create the world’s most advanced IP but also enabling our ecosystem to take full advantage of multi-cloud resources for accelerating R&D.”

The Power of Cloud Automation for HTC

The possibilities of using HTC for R&D are rapidly expanding, thanks to the growing availability of supercomputing clusters offered by cloud service providers.

But despite this democratization of high performance computing, HTC systems are complex to set up and maintain. To fully capture the benefits of HTC, organizations need sophisticated and robust management capabilities to securely run HTC workloads across multiple cloud services and computing architectures. Rescale HTC makes this possible.

Rescale HTC offers multi-cloud automation, unified data management, and full-stack security to help organizations more efficiently execute their HTC projects.

“Across industries we are seeing rapid adoption of high-performance computing and high-throughput computing, upon which customers are running simulations, applied AI, and machine learning for engineering and scientific workloads.” says Joris Poort, CEO of Rescale. “Arm is a great example of how Rescale is working with leading companies to accelerate innovation with the power of intelligent computing across clouds.”

Organizations can use Rescale HTC’s cloud-native container-based architecture to quickly access the best hardware infrastructure for their computational jobs across any of the leading cloud service providers, including AWS, Google Cloud, Microsoft Azure, and NVIDIA DGX Cloud.

Rescale’s global multi-cloud coverage combined with its platform intelligence delivers best-in-class performance for any workload, anywhere, while offering visibility and control to help ensure HTC operations are efficient and secure. It provides improved throughput, cost efficiency, and reliability beyond what’s possible from a single cloud provider.

Rescale HTC Results: Unparalleled Scale and Reliability

The performance and scale of Rescale HTC is impressive: it’s is capable of running millions of compute tasks simultaneously across hundreds of thousand vCPUs, multiple cloud providers, and multiple global regions, launched in just a few minutes. In stress testing over an eight week period, Rescale HTC successfully ran more than 50 million core hours on Arm architecture instances demonstrating 99.99 percent job reliability.

Rescale’s platform also supports the seamless adoption of the latest computing architectures, such as new Arm-based processors, including AWS Graviton3, Google Cloud Tau T2A, Microsoft Azure Dpsv5, and NVIDIA GH200 Grace Hopper. Through Rescale’s AI-based Compute Recommendation Engine, workloads can automatically and seamlessly adopt the best hardware architectures for specific types of workloads.

Understanding HTC Use Cases and Benefits

High-throughput computing is essential for tackling complex scientific questions that involve processing large amounts of data in parallel to quickly explore many possible design options.

Certain R&D projects require large-scale design studies, in which a massive number of computing tasks (e.g. simulations) are needed to validate design decisions. With HTC, more than a million workloads can be run independently and concurrently. This computing approach is becoming more valuable for R&D as scientists and engineers work on increasingly complex problems.

As researchers collect more and more data from experiments and observations, HTC can help analyze that data quickly and efficiently. By breaking the data into smaller units and running them in parallel, HTC can rapidly parse through massive datasets. This has led to breakthroughs in fields such as genomics, where researchers use HTC to process and analyze vast amounts of genetic data in search of new treatments for diseases.

Automating the computing process provides significant benefits for engineers and scientists across industries. Rescale HTC automates time consuming steps of submitting many computing tasks and managing all the data inputs individually while also automating hardware and software setup with the best-fit configurations. Knowing job submissions will run consistently and reliably, means users can spend more time focused on asking bigger R&D questions and less time configuring and troubleshooting their computing environment.

Another advantage of HTC is its ability to digitally replicate complex systems. Researchers can use HTC to create simulations of physical or biological systems, allowing them to study the behavior of these systems in ways that would be difficult or impossible with traditional experimental methods. For example, HTC has been used to simulate the behavior of proteins and other molecules, which has led to new insights into the structure and function of these important biological components.

One of the top use cases for HTC is in drug discovery. Researchers can use HTC to screen large libraries of compounds to identify potential drug candidates. This process, known as virtual screening, involves running simulations of how different compounds interact with target proteins, which can help researchers identify compounds with the potential to treat a specific disease.

Another area where HTC is making a big impact is in materials science. Researchers are using HTC to simulate the behavior of materials at the atomic and molecular level, which can help them understand the properties and potential applications of these materials.

HTC is also being used in environmental research to simulate the behavior of complex ecosystems. For example, researchers are using HTC to model the effects of climate change and to develop strategies for countering these issues. HTC is also being used to study the effects of pollution on ecosystems and to develop new technologies for cleaning up contaminated environments.

And physics researchers are able to simulate the behavior of complex systems such as solar systems. These simulations allow scientists to analyze massive telescope image sets, test theories, and make predictions, such as identifying exoplanets and asteroids.

With the ability to analyze large amounts of data and examine complex problems, HTC is enabling breakthrough discoveries that were previously impossible. As the technology continues to evolve and become more accessible, it is expected to play an increasingly important role in scientific research in the future.