5 Winning Strategies to Accelerate Engineering: Cloud HPC is Automated (Part 5 of 5)
- Editors note: This post is the introduction to a 6 part blog series. Read the full report here.
5. Cloud HPC: From Manual to Automated
Traditional HPC is Manual
The traditional HPC approach is usually script-based and therefore manual for managing engineering workflows and IT operations. There is no easy and fast way for engineers to automate end-to-end workflows for efficiency, nor for IT to automate control of multi-cloud infrastructure, applications, and security with specific policies, without having to custom build IT management for separate systems, or to handle data protection and compliance manually.
HPC Built for the Cloud is Automated
IT/HPC Management Automation
A built-for-the-cloud approach to HPC management starts by recognizing that the cloud as a dynamic environment can provide a broad range of possibilities, from positive (e.g., on-demand access to scale) to negative (e.g., cloud overspend). Additionally, most HPC organizations are new to cloud, and don’t have the in-house talent or expertise to manage cloud-based infrastructures.
Therefore, for IT/HPC to be successful with cloud, unprecedented levels of automation are required. This should include a unified experience for multi-cloud operations, built-in security and compliance with automated encryption, and adherence to all relevant leading security and compliance standards. Admins need to define policies and have confidence their policies will be applied, from managing spend, approving software and architectures, as well as data access and sharing policies.
Rescale’s intelligent control plane provides a single pane of glass for visibility, control, and automation, for both IT and HPC service owners. Organizations can have company-wide administrators, as well as workspaces, defined to individual business units, each with their own administrators. Policies can be defined on security configs (e.g., password policies, MFA), approved architectures, software packages used, as well as which users are able to define job templates to help ensure best practices among engineers.
Additionally, Rescale provides service level guarantees for the successful completion of compute jobs, and automates load balancing between cloud provider regions and hardware architectures. Many customers have shared that without Rescale, they would not be successful in transitioning their HPC workloads to cloud.
Complete workflow
management
End-to-end visibility on
engineering usage and spend
Single point of control for
multi-cloud
Data protection & security
management
Engineering and R&D Automation
Supporting computational science and engineering objectives is often not about running a single compute job, but rather about enabling an entire computational pipeline. In such a pipeline, the outputs of one job may be incorporated directly into another. In some cases, each of these compute jobs might use different software from different vendors, and leverage different architectures.
A built-for-the-cloud approach to HPC takes advantage of cloud’s scale, optionality, and connected nature, and applies automation. With this approach, the user can easily access any software needed along with the data required. The best suitable architecture is available to the user on demand. And users are able to save, share, and version control an entire computational pipeline. By automating the entire analytics process, engineers and scientists can accelerate their design iteration cycles.
Computational Pipeline Examples
Rescale’s intelligent control plane approach provides an architecture for ecosystem integration, which enables organizations to automate entire computational pipelines that may combine different applications, compute architectures, storage devices, and even cloud providers. Rescale supports automation of these pipelines and helps ensure that the compute job at each step is running the most performant and cost effective architecture. Through integration with leading product lifecycle management (PLM) or simulation process & data management (SPDM) solutions, this process can also work with an organization’s existing R&D tools.
HPC Built for the Cloud Enables Digital Transformation and Delivers Measurable Impact
By reimagining the engineering and R&D experience using what’s possible from cloud and automation, a built-for-the-cloud approach starts with the goal of optimizing R&D throughput, beginning with the user, not the hardware they use. This approach is highly consistent with the goal of digital transformation, where the goal is to change how people and teams interact and eliminate bottlenecks.
Based on Rescale customer experiences, a built-for-the-cloud approach to HPC has demonstrated improved R&D throughput, enabling R&D to explore larger problems to solve, accelerate run time of compute jobs, and improve collaboration. Additionally, intelligent automation has been proven to optimize cost/time-to-solve tradeoffs, help customers take advantage of the latest computing architectures, all while providing secure and compliant operations.
Cloud transformation for enterprise computing has created new ways for businesses to engage customers, and disrupted commerce across industries. While a lift-and-shift approach to HPC cannot deliver the full benefits of cloud, a built-for-the-cloud HPC approach can be transformative, delivering new levels of productivity for engineering and R&D.
Businesses that use a built-for-the cloud approach to HPC can dramatically enhance their computational science and engineering capabilities, and create new ways to capture value and develop new revenue streams.
Accelerate time to market – Combining multiphysics simulations with rapid prototyping to quickly fine tune designs and accelerate time to value of new product innovations.
Adopt digital twins – Simulating a product’s interaction with real-world scenarios when physical prototyping is impractical, or combining simulation with product sensor data to predict product performance.
Combine AI/ML with simulation – Using simulations to predict how a product may perform across a full range of potential designs, and building surrogate models based on simulation data to quickly identify potential designs based on product objectives.
New computation-enabled product or services – Using cloud’s scale and connected nature to create new possibilities for science and engineering. Examples include cloud-based collaboration platforms for seamless interaction between product designers and manufacturers, and using simulation to do everything from verifying designs to validating manufacturing.
By employing a built-for-the-cloud approach to HPC, enterprises can not only commercialize new innovations faster, but also better manage business risk and improve operations. HPC built for the cloud is therefore a foundational element to digital transformation for engineering and R&D-driven organizations.
Customer Spotlight
“Technological complexity in our clients’ demands is growing, so we need to have the computational capabilities to support them. Having access to HPC on demand through Rescale was a game changer – on our largest jobs, we see a 5-10x increase in speed of computing. Now we can take on more and larger client projects and don’t have to make compromises to meet our clients’ needs.” “One of my favorite things about Rescale is that I don’t have to worry about waiting in a queue or wondering if or when my job is going to run. I can also tailor the computational resources for the client’s problem at hand, enabling me to run jobs 10x faster or explore 10x more design parameters.”
