In this Big Compute Podcast episode, host Gabriel Broner interviews Marek Michalewicz, Director of ICM, the HPC center at the University of Warsaw. With the advent of HPC Cloud platforms, the conversation centers around the opportunity and the benefits to empower all users with simple and flexible access to systems across multiple centers and in the cloud.
Register for Future Big Compute Podcast Episodes
Overview and Key Comments
Rethinking HPC in Academia
HPC in Academia has been primarily on-premise. A system of a particular architecture is usually kept for five years while relative performance declines over time. Utilization is high, so user jobs wait in queues. With the advent of HPC cloud platforms, it may be time to rethink HPC for Academia. Instead of one institution – one system, we may enable access to systems on premise, across multiple centers, and in the cloud. At the same levels of spending as today, we may be able to accelerate time to science and enable new areas of research by having access to multiple architectures, to the latest technologies and by reducing time waiting in queues
Cloud Platforms for HPC
Michalewicz believes we are inevitably moving to a cloud approach for academia, and it is a natural progression of the evolutionary steps that have occurred:
“We are going directly into cloud HPC solutions. I actually view it as a refresh and expansion of technologies that already existed. A few years ago the Cyfronet Research Center in Poland developed the PLGrid grid system. It is rigid, not as scalable and flexible as cloud, but users are able to use resources from various centers. Moving into much more sophisticated and flexible technologies that are enabled through HPC cloud solutions is a natural progression. I can’t see a way out of this.”
“We had to wait for the time when technical and business solutions had to be found. People have been talking about utility computing for a long time. It’s happening now. Crays are available as instances, so are FPGAs or GPUs. That is exactly what should be happening.”
“New architectures such as quantum computing are very exciting but we don’t have access. It would be fantastic if the great variety of computing architectures were available to do research.”
Challenges in the Transformation
Michalewicz warns us about the challenges in the transformation journey, putting special emphasis on people aspects:
“There are various kinds of obstacles. Especially psychological, the human factor, as people are afraid of losing control. I saw people being very skeptical about cloud ten years ago. The technical obstacles have been conquered. People are afraid of losing their position. The reality is wherever you provision HPC resources, you will need expertise. People should think instead that a boom is coming, a time to share expertise with new users.”
“There is also the barrier between HPC and traditional IT. People who live in the enterprise space may not truly understand the needs of HPC. When HPC people try to set up systems as cloud, IT people may feel we are encroaching in their territory, but we need to remove those barriers. We need to remember it is about removing barriers for the users, not the operators.”
HPC for everyone
Michalewicz believes the simplicity of HPC cloud platforms will make HPC not restricted to a small group of people:
“Euro HPC will provide incredibly large resources, but some of those resources will be more appropriate for extremely seasoned users. I care about the group of “newcomers” in the academic environment who have not tried HPC yet. For this group, it’s not necessarily about large scale, but about the “feel” and the ability to have resources that can expand with no limits.”
“I actively look for people who have not used HPC. I started a student team when I was in Singapore and they won the Supercomputing cluster competition in America. More recently I started a Warsaw team and they will go to their sixth final. The two teams are competing against each other. It’s great fun to see new people come to HPC and become proficient after a few years of training.”
“At the University of Warsaw we have 50,000 students and only 1,000 registered HPC users. I want to see everybody in the University have access to extensible resources from 1 to 10,000 cores. I want to completely obliterate the obstacles to access computing resources at any scale.”
The thinking of going from the traditional on-premise model to a cloud of multiple centers has evolved over the years. The difference is that the advent of cloud and cloud platforms for HPC may have removed obstacles to make it a reality. This large compute capacity and variety of architectures available will benefit existing HPC researchers. At the same time, the simplicity and flexibility will make HPC accessible to all researchers and students.
Marek Michalewicz is the Director of ICM, the High Performance Computing center at the University of Warsaw. With many years in the industry, Marek has headed academic and research HPC centers including the A*STAR Computational Research Center in Singapore.
Gabriel Broner is VP & GM of HPC at Rescale. Prior to joining Rescale in July 2017, Gabriel spent 25 years in the industry as OS architect at Cray, GM at Microsoft, head of innovation at Ericsson, and VP & GM of HPC at SGI/HPE.