Cloud Bursting for HPC and AI
Engineering and Scientific Workloads

What is Cloud Bursting?

Cloud bursting is a cloud computing technique that allows companies to scale compute capacity quickly to handle spikes in demand. The main premise of cloud bursting is that a company uses its own private cloud or on-premises infrastructure to run applications at a baseline capacity. When additional compute power is needed due to a usage spike, public cloud resources like AWS, Microsoft Azure, Google Cloud Platform (GCP), and Oracle Cloud Infrastructure (OCI) can be dynamically called into action.

The private and public clouds are interconnected, usually over a direct network connection or VPN. Orchestration software and policies are set up so that workloads can be automatically shifted to the public cloud when they require additional capacity. This provides a flexible and scalable pool of resources. Companies only pay for public cloud capacity when they actually use it.

Once peak demand has passed and the spike in usage drops back down, public cloud resources can be released again. The application will revert to utilizing just the baseline private cloud footprint. This optimizes cost and prevents over-provisioning, since organizations don’t have to maintain large amounts of unused capacity just to handle occasional peaks. 

Key aspects of cloud bursting include:

• Companies use private cloud or on-premises resources as the baseline infrastructure for steady-state applications. This is often called the “private cloud footprint.”
• When additional compute capacity is needed, typically during peak periods, public cloud resources like AWS, Azure, GCP, or OCI are “burst” into. These provide additional capacity to handle usage spikes.
• Once peak demand subsides, the additional public cloud resources can be released back to control costs. The application returns to using only the private cloud footprint.
• Cloud bursting requires the private and public clouds to be interconnected and compatible. This allows workloads to be shifted between them as needed.
• It provides elastic scalability and is pay-as-you-go, since companies only pay for the additional public cloud resources when bursting.
• Cloud bursting is mostly automated using orchestration software and policies that define when to scale out to public clouds.

What Are the Different Types of Cloud Bursting?

There are a few different types and approaches to cloud bursting, which allow companies flexibility in how they implement it:

Vertical Cloud Bursting 

With this approach, additional capacity is added by provisioning larger virtual machine instances in hyperscale and specialized clouds. For example, if the private cloud runs workloads on small VMs, then bursting might provision more powerful large VMs to handle the spike in usage. It scales up to more powerful resources.

Horizontal Cloud Bursting

Here, additional capacity is added by provisioning additional instances of the same size VMs that run the application. So if the private cloud runs the app on a cluster of small VMs, then more of those same small VMs are spun up for bursting. It scales out to more resources.

Hybrid Cloud Bursting 

This combines both vertical and horizontal approaches. For part of the additional capacity, larger and more capable VMs are spun up, and additional small instances are also added to scale out. This provides flexibility to burst using both scaling up and scaling out.

Cross-Cloud Bursting

With this type of bursting, a company provisions resources from multiple different public cloud providers when additional capacity is needed, so they could burst from their private cloud to AWS and Azure simultaneously. This avoids vendor lock-in and allows leveraging services and pricing differences between providers.

Cloud bursting provides a toolbox of techniques to dynamically expand compute capacity. Companies can choose between vertically scaling up, horizontally scaling out, using a hybrid approach, or leveraging multiple public clouds. This provides great flexibility in implementing cloud bursting.

Why Is Cloud Bursting Considered Hybrid?

Cloud bursting is considered a hybrid cloud architecture because it involves seamlessly integrating both private cloud/on-premises infrastructure and public cloud resources:

• Hybrid – Cloud bursting spans across and connects private infrastructure with public clouds. This is the very definition of a hybrid model.
• Private Cloud Footprint – The baseline compute for applications starts in the company’s own data centers or private cloud. This is the private infrastructure piece.
• Public Cloud Scaling – When extra capacity is needed, it is provisioned on demand from a public IaaS cloud like AWS, Microsoft Azure, Oracle Cloud, or Google Cloud. This brings in the public cloud piece.
• Unified Management – Orchestration and policies manage the private and public infrastructure as a single unified pool of dynamic resources, not as separate silos. This is a key hybrid attribute.
• Flexible Scaling – Compute can flexibly shift between the private and public environments to match application workload demands. This demonstrates the elastic nature of hybrid.
• Optimized Costs – Companies only pay for public cloud bursting capacity when they actually need it. The hybrid model allows optimizing costs.

What are the Challenges of Cloud Bursting?

Here are some common challenges and potential pitfalls to be aware of with cloud bursting:

Application Portability 
Apps must be architected to be portable across both private and public clouds for bursting to work smoothly.

Data Transit Costs
Large data transfers between clouds can incur substantial network egress fees. This should be optimized.

Vendor Lock-In
If leveraging proprietary features of a public cloud, it can create technical lock-in and reduce portability.

Authentication and Permissions
Identity and access controls need to bridge across on-premises and cloud environments.

Orchestration Complexity
The automation and policies of cloud bursting can be quite complex to set up and manage.

Testing at Scale
Applications should be tested to ensure they can scale up and down smoothly without issues.

Cost Predictability
Bursting patterns may be irregular, making costs harder to forecast compared to steady state usage.

Hybrid Connectivity
A stable, high-bandwidth connection is required between private and public clouds.

Compliance and Security
Data must remain compliant and secured properly as it moves between cloud environments.

Skillset Requirements
IT teams may lack experience with hybrid cloud, automation tools, or multi-cloud architectures.

How Cloud Bursting Works With Rescale

Here are some common and emerging use cases for digital twin technology across various industries:

• Predictive Maintenance – Digital twins of equipment like turbines or assembly lines can identify issues and failure risk before downtime occurs.
• Virtual Commissioning – New facilities, production lines, and other processes can be simulated and tested virtually before physical installation.
• Remote Monitoring – Digital twins enable remotely tracking assets, fleet vehicles, and more to analyze performance.
• Product Development – New product designs can be refined and optimized through simulating in a digital twin first.
• Training – High risk scenarios like medical procedures or disaster response can be simulated in a digital twin for training.
• Smart Cities – Digital twins of city infrastructure, traffic patterns, and air flows can optimize transportation and services.
• Logistics – Modeling supply chains digitally can identify optimizations, bottlenecks, and prepare for disruptions.
• Healthcare – Digital patients based on scans and data can customize treatments or medical devices.

The applications span from individual equipment optimization to organization-wide initiatives. Digital twin use cases will rapidly grow as the technology and associated benefits become more understood.