Cloud Adoption is Driving HPC Toward Digital R&D
Originally posted on www.bigcompute.org.
High Performance Computing is a massive industry ($55 billion by 2024) that is the lifeblood of research and science innovation. From supersonic jets to developing cures for pandemics – research engineers rely on HPC for algorithmically complex workloads running on massive datasets.
Today this HPC industry is still running mostly on-prem in supercomputers and Linux clusters in private datacenters. But the rate of specialized architectures, the diversity of options for simulation software, and the general pressure to bring more agility to research engineers to rival the “developer experience” their enterprise software developers face – has created a massive surge in moving HPC workloads to the cloud.
Big Compute sponsor, Rescale, analyzed the aggregated data of more than 300 customers to generate insights leading high performance computing practitioners. This inaugural Big Compute report reveals the drivers behind the massive cloud transformation wave that is overtaking the HPC domain.
Big Compute is Driving the HPC Industry to the Cloud
Big Compute is the use of high performance computing (HPC) hardware and specialized software at scale to achieve breakthroughs in science, engineering and business. The report — “Big Compute: 2021 Cloud HPC Ecosystem” — reveals key trends in how HPC hardware and specialized software are accelerating cloud adoption across science, engineering and business.
An Explosion of Simulation Software and Specialized Architectures
The simulation software market is on fire – an industry expected to reach $21 billion by 2025. Across our library of 600 software packages that Rescale supports, we’re seeing most organizations typically run three or more simulation software packages. ANSYS Fluent is the most dominant simulation software provider across most HPC use cases, but within every industry, domain-specific challengers are ever- growing. For example, CFD++ (Aerospace), LS-DYNA (Energy), CONVERGE 3.0 (Automotive). It’s a great time to be a science engineer, in terms of the diversity of simulation software choice, and this report catalogues some of these emergent simulation softwares in specific industries.
Multi-Cloud is Uniquely Valuable for HPC and an Operational Reality
The combination of a fast growing catalogue of HPC software packages, up against different public cloud providers offering different advantages in underlying architecture is driving a multi-cloud trend. This report shows that more than 50 percent of HPC users are using more than one public cloud provider, up from 35% in 2018.
HPC Leaders are Seeking to Close Talent Gaps and Deliver on Digital R&D
Business leaders understand how cloud changed software development and are now looking to apply cloud transformation to R&D. Research engineers are specialists in the science domains they serve. Cloud providers are specialists in cloud infrastructure. But the universe of engineers who are specialized in both the applied science domains and cloud infrastructure is very small, exacerbated by a talent shortage in the HPC industry. This report found that for most HPC execs, they are looking to automation to deliver more holistic R&D solutions, including workflow automation to help their organizations get the most out of the cloud.
